system

Abstract

We provide the system. [Solution] A data collection method for sensing the movement of objects or people by utilizing signal reflection from existing wireless communication devices, A data transmission means that transmits data obtained from a data collection means to a server, An analysis means that analyzes the transmitted data and determines the movements and location of a person, An anomaly detection means that detects anomalies based on analysis results and generates notifications, A user notification means that provides the generated notification to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional home and office security management systems, the movements of residents and workplace users cannot be fully grasped, and there are limitations in monitoring the elderly and detecting intruders. Also, since manual monitoring is the mainstream, it is difficult to respond quickly and accurately, and the risk when an abnormality occurs is high. Due to these, there is a problem that the safety and convenience of users are reduced.

Means for Solving the Problems

[0005] This invention provides a data collection means for sensing the movement of objects and people using signals from existing wireless communication devices. Furthermore, it includes a system that analyzes the sensed data on a server, automatically detects anomalies, and notifies the user, enabling a quick and accurate response. This can improve the safety of the elderly and residents and enhance the comfort of home and office environments.

[0006] A "wireless communication device" is a device that uses radio waves to send and receive data, and includes Wi-Fi routers and smartphones.

[0007] "Signal reflection" is the phenomenon in which radio waves emitted from a wireless communication device hit an object, bounce back, and are captured by the original or another receiving device.

[0008] A "data collection means" is a system that uses signal reflections acquired by wireless communication devices to sense the movement of objects or people and collect related data.

[0009] "Data transmission means" refers to a function or protocol for transmitting collected and processed data to a server via a communication network.

[0010] "Analysis means" refers to a function in which a program or model on a server receives transmitted data and analyzes it to determine its specific meaning or information.

[0011] An "anomaly detection method" is a process that identifies unusual behavior or patterns based on analyzed data, and continuously monitors and detects those anomalies.

[0012] A "user notification system" is a system designed to quickly inform users of detected anomalies, and includes functions for sending alerts and notifications. [Brief explanation of the drawing]

[0013] [Figure 1]This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention provides a system that utilizes signals generated by wireless communication devices to improve safety and convenience in home and office environments.

[0035] The terminal senses signals emitted from wireless communication devices installed in homes and offices, captures the characteristics of those signals when reflected by objects or people, and obtains motion and location information. This data is transmitted to the server in real time.

[0036] The server uses machine learning algorithms to analyze the received data and perform pattern recognition. This allows it to identify, for example, whether a resident is in the living room or bedroom, and detect anomalies based on their normal activity patterns.

[0037] If an anomaly is detected, the server immediately generates an alert and notifies the user through a user notification system. The user receives the notification on a device such as a smartphone or PC and takes action as needed. Because this process is automated, a quick and effective response is possible.

[0038] For example, if an elderly person exhibits unusual behavior at night, the user can receive an immediate notification and take appropriate action to ensure their safety. Furthermore, in a workplace environment, it's possible to monitor employee movements and optimize energy consumption.

[0039] This system is a cost-effective solution because it utilizes existing wireless communication devices, eliminating the need for additional hardware installation. Furthermore, AI-powered analysis enables real-time situation assessment and rapid response, significantly improving user convenience and safety.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The terminal receives signals emitted from wireless communication devices within a designated area. This collects basic data for detecting changes in signals caused by the movement of objects or people.

[0043] Step 2:

[0044] The terminal analyzes the characteristics of the received signal and records the amplitude and phase changes of the reflected signal as numerical data. This data contains information about the movement and position of objects and people.

[0045] Step 3:

[0046] The terminal sends the analysis results described above to the server in packet format. The transmitted data includes timestamps and location information.

[0047] Step 4:

[0048] The server receives data sent from the terminal in real time. The received data is immediately stored in the database.

[0049] Step 5:

[0050] The server analyzes the received data using machine learning algorithms. Here, it compares the patterns of signal changes with a model to estimate human movement and location.

[0051] Step 6:

[0052] The server detects abnormal behavior or patterns based on the analysis results. If an anomaly is detected, it generates an alert based on pre-configured conditions.

[0053] Step 7:

[0054] Users receive alerts generated by the server on their devices. These alerts include the type of anomaly, the time of occurrence, and recommended actions.

[0055] Step 8:

[0056] Users take necessary actions based on the alert information displayed on their devices. This includes, for example, checking their home environment and taking security measures at work.

[0057] This process allows the system to operate efficiently, providing users with real-time situational awareness and immediate responses.

[0058] (Example 1)

[0059] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0060] There is a need to provide a means of accurately detecting the behavior of entities or people in the environment in real time, without requiring additional hardware, by utilizing existing wireless communication equipment, and to quickly notify users of any anomalies. Conventional systems require additional sensors or equipment to respond to changes in the environment, which presents challenges in terms of cost and installation. Furthermore, there is room for improvement in the speed and accuracy of anomaly detection and notification.

[0061] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0062] In this invention, the server includes information gathering means for detecting the behavior of entities or people in the environment by utilizing signal reflection from wireless communication equipment; information transfer means for transferring the information obtained from the information gathering means to an information processing device; and analysis means for analyzing the transferred information to determine the behavior and location of entities or people. This makes it possible to detect the movement of objects and people in the environment without additional hardware, and to accurately grasp and quickly notify of anomalies in real time.

[0063] "Wireless communication equipment" refers to devices that use radio waves to send and receive voice, data, and other signals, and is a device that exchanges signals within an environment.

[0064] "Signal reflection" refers to the phenomenon where radio waves transmitted from wireless communication devices hit an object or person and return, allowing information about the object's location and movement to be obtained.

[0065] "Information gathering means" refers to a process or device for detecting entities or human behavior in the environment using signal reflection from wireless communication devices and acquiring data related thereto.

[0066] "Information transfer means" refers to a process or device for transmitting data obtained by information collection means to other devices, particularly information processing devices.

[0067] An "information processing device" is a device for analyzing data transmitted by an information transfer means, and includes devices that perform computational processing, such as servers.

[0068] "Analysis means" refers to a method or device that performs processing to determine the behavior and position of an entity or human being based on data received in an information processing device.

[0069] An "anomaly detection means" refers to a process or device that detects events that deviate from normal activity patterns based on results obtained from analysis means, and generates warnings or notifications.

[0070] "User notification means" refers to a process or device for providing users with generated warnings or notifications and prompting them to take necessary action.

[0071] This invention is a system that uses signals emitted from existing wireless communication devices to sense objects and human movements in the environment and detect anomalies. The server, terminal, and user play three main roles.

[0072] The server provides the computing resources necessary to analyze signals from wireless communication devices. As an information processing device, the server executes a generative AI model to analyze the received data. Here, frameworks such as TENSORFLOW® and PyTorch are utilized to train learning algorithms, thereby building a model that learns normal behavioral patterns and detects anomalies.

[0073] The terminal senses signals from wireless communication devices installed in the environment. For example, it receives reflected signals from Wi-Fi routers and Bluetooth devices and acquires them as data using information gathering means. This data is transmitted from the terminal to the server via information transfer means.

[0074] Users receive notifications from their devices and servers and take necessary actions. For example, if unusual movement is detected in an elderly person's room, the user will receive a prompt message such as, "The system detected unusual activity during the night. Do you want to check on the elderly person's safety?" After receiving this notification, the user can use their smartphone or PC to check the situation and take appropriate action.

[0075] The key features of this invention are its cost-effectiveness, as it utilizes existing devices and requires no additional hardware, enabling rapid and accurate anomaly detection and notification. This can enhance safety in the home and efficiency in the workplace.

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] The terminal receives signals transmitted from wireless communication devices and collects them as data. Inputs include radio waves from Wi-Fi routers and Bluetooth devices, which are reflected when they strike objects and people. The output is data indicating the characteristics of the reflected signals. This data includes information such as signal strength, arrival time, and phase. Specifically, one scenario involves the terminal detecting the movement of a person in a living room.

[0079] Step 2:

[0080] The terminal sends the collected data to the server for information processing. The input data is signal characteristic data measured by the terminal, which is transferred to the server via the internet connection. The output is the signal characteristic data received by the server. This allows the information processing device to perform data analysis. Specifically, the terminal sends signal data to the server several times per second.

[0081] Step 3:

[0082] The server analyzes the movements and locations of entities or people in the environment based on the received data. The input is signal characteristic data sent to the server, which is then analyzed using a generative AI model. Specifically, TensorFlow and PyTorch are used as the software. The output is information about movements and locations in the environment obtained through the analysis. For example, the server performs an analysis to determine that a person is in a bedroom at a specific time.

[0083] Step 4:

[0084] The server uses the analysis results to detect anomalies in behavioral patterns and generates alerts as needed. Input includes information on movement and location obtained from the analysis system, which is compared to normal patterns to determine anomalies. Output is an alert indicating the anomaly. This alert is sent to the user via a notification system. For example, a message such as "Unusual movement detected in the living room" might be generated.

[0085] Step 5:

[0086] The user receives alerts sent from the server on their device (smartphone or PC). The input is alert information from the server, which is displayed on the receiving device. The output is a notification to the user, providing them with information to take action. Specifically, the user can check the notification and then take appropriate action, such as checking the safety of the elderly person, as needed.

[0087] (Application Example 1)

[0088] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0089] The present invention aims to improve safety and convenience by using wireless communication devices to sense the movements of objects and people in a space in real time, enabling anomaly detection and rapid notification. In particular, it aims to support emergency response and enhance a sense of security in the living environment by quickly identifying abnormal behavior in the elderly or in specific environments.

[0090] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0091] In this invention, the server includes information acquisition means, data transmission means, and analysis means. This enables precise motion detection using a wireless communication device and rapid detection of abnormal behavior.

[0092] An "information acquisition means" is a device that uses signal reflection from existing wireless communication devices to detect the movement of objects or people.

[0093] A "data transmission means" is a device for transmitting information obtained from an information acquisition means to a data processing device.

[0094] An "analysis device" is a device that analyzes transmitted information to determine a person's movements and location.

[0095] An "anomaly detection device" is a device that detects anomalies based on analysis results and generates an alert.

[0096] A "user notification device" is a device for providing generated alerts to the user's terminal.

[0097] "Application means" refers to software installed on a user's portable information terminal that displays warnings in real time when an anomaly is detected.

[0098] The system for carrying out this invention is configured as follows.

[0099] The server receives signals from wireless communication devices and processes the data obtained by the information acquisition means. This data is transmitted to the server using the data transmission means. The server analyzes the transmitted information using the analysis means to determine human movement and location. The anomaly detection means detects anomalies based on the analysis results, and if an anomaly is detected, the server immediately generates an alert. This alert is communicated to the user terminal by the user notification means, and the user receives the warning through the application means installed on their mobile device. This application notifies the user of anomalies in real time.

[0100] Specifically, the server uses machine learning algorithms for data processing and utilizes software such as TensorFlow and scikit-learn. This makes it possible to detect abnormal behavior compared to everyday behavioral patterns. For example, if abnormal movement is detected in an elderly person at night, the application is designed to send a notification to their smartphone stating, "Unusual movement has been detected. Please check your safety."

[0101] An example of a prompt message would be: "Develop a machine learning model for detecting abnormal behavior using wireless signal data within a home. In particular, we need a model that can distinguish between normal and abnormal behavior in the living room and bedroom."

[0102] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0103] Step 1: The server receives signals transmitted from the wireless communication device. The input is wireless signal strength data, and the output is the data collection result from the information acquisition means. In this step, the reception of the signal and the collection of data based on it are specifically carried out.

[0104] Step 2: The server transmits numerical data obtained from the information acquisition means to the server using the data transmission means. The input is the data collection result, and the output is data in a format that can be processed within the server. This step involves data format conversion and network transmission.

[0105] Step 3: The server analyzes the data using analytical tools to determine human movement and location. The input is data in a processable format, and the output is the identification result of movement and location. In this step, machine learning algorithms are used to analyze movement patterns using TensorFlow or scikit-learn.

[0106] Step 4: The server uses anomaly detection means to detect anomalies based on the analysis results and generates an alert. The input is the identification result of movement and location, and the output is alert information indicating an anomaly. In this step, a learning model is used to detect deviations from normal behavior patterns.

[0107] Step 5: The server sends the generated alert to the user terminal via a user notification mechanism. The input is the alert information, and the output is the notification information sent to the user. This step utilizes a network protocol for sending notifications in real time.

[0108] Step 6: The user receives a notification through an application on their mobile device. The input is notification information, and the output is a visually displayed warning message. In this step, the application visually and intuitively presents the notification content to the user.

[0109] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0110] This invention combines a safety management system that utilizes signals from wireless communication devices with a user emotion recognition function, aiming to make home and office environments safer and more comfortable.

[0111] The terminal receives signals emitted from existing wireless communication devices and senses the movement of objects and people. This allows it to collect data on the movement and location of the target. This data is transmitted to a server in real time and analyzed appropriately. Machine learning models are used in this analysis to detect human movement and anomalies.

[0112] Furthermore, the server is equipped with an emotion engine that can evaluate the user's emotional state by analyzing their voice and facial expression data. The emotion engine acquires voice and video data through interfaces such as cameras and microphones and determines whether the user's emotions deviate from the normal range.

[0113] For example, the system can sense whether a resident is relaxed or stressed. Based on this emotional state, the server adjusts notifications and actions to provide the user with more relevant information. For instance, if a user is stressed, the system might provide information to encourage relaxation.

[0114] Users can receive these notifications via their devices and take the most appropriate action based on their environment. This system enables flexible and effective management of living and working environments based on detailed information, including the user's emotional state. Furthermore, if an anomaly is detected based on data analysis, the system quickly alerts the user and prompts them to take the necessary action, thereby ensuring user safety.

[0115] This invention offers high security and convenience while being an inexpensive device, making it suitable for a wide range of applications in homes and offices. Furthermore, its emotion-based recognition capabilities could contribute to health management and mental support. This would allow users to enjoy an environment where they can focus on their lives and work with greater peace of mind.

[0116] The following describes the processing flow.

[0117] Step 1:

[0118] The terminal receives radio waves emitted from wireless communication devices and detects signals reflected by surrounding objects and people. This allows it to collect basic data for recording environmental changes in real time.

[0119] Step 2:

[0120] The terminal analyzes the received signal data and converts the movement and position of objects and people into numerical data. This data includes information such as speed and direction of movement, enabling detailed motion analysis.

[0121] Step 3:

[0122] The terminal sends the analyzed data to the server. Security protocols are used for communication, ensuring data confidentiality during transmission.

[0123] Step 4:

[0124] The server receives data sent from the terminal and stores it in the database. The received data is immediately ready for processing.

[0125] Step 5:

[0126] The server analyzes the transmitted data using machine learning algorithms to determine the person's movements and location in detail. It also detects movements that deviate from normal patterns as anomalies.

[0127] Step 6:

[0128] The server collects and analyzes the user's voice and video through an emotion engine. This allows it to determine the user's emotional state, for example, whether they are feeling stressed.

[0129] Step 7:

[0130] The server integrates and analyzes user behavior and emotional data, and generates notifications as needed. If a user is experiencing stress, it might prepare a notification to encourage relaxation.

[0131] Step 8:

[0132] Users receive notifications sent from the server on their devices. These notifications include appropriate actions for the user and provide audio and visual guidance.

[0133] Step 9:

[0134] Users can take necessary actions based on the notification content. At home, this might involve adjusting lighting or playing background music; in the office, it might provide clues for deciding how to take breaks.

[0135] This entire process allows the system to monitor both user behavior and emotions in real time, providing a safer and more comfortable environment.

[0136] (Example 2)

[0137] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0138] In modern living environments, there is a need for systems that utilize wireless communication technology to provide safe and comfortable living and working environments. However, existing systems have limitations in detecting the movement and anomalies of dynamic objects, and may not be able to provide appropriate responses that take into account the user's emotional state. Furthermore, there is a lack of effective means to comprehensively manage user safety and mental comfort.

[0139] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0140] In this invention, the server includes information gathering means, information transmission means, analysis means, anomaly detection means, emotion recognition means, and user notification means. This makes it possible to analyze the behavior and emotional state of dynamic targets using wireless communication devices in real time, evaluate anomalies and the user's emotional state, and provide appropriate notifications.

[0141] "Information gathering means" refers to devices that receive signals from wireless communication equipment and sense dynamic objects or human movements.

[0142] "Information transmission means" refers to a function for transmitting data acquired by information collection means to a computing device (server).

[0143] "Analysis means" refers to processes and algorithms for determining the movement and position of a dynamic object based on transmitted information.

[0144] An "anomaly detection method" is a technology that detects anomalies that deviate from normal operating patterns based on analysis results and generates warnings.

[0145] "Emotion recognition means" refers to technology that analyzes a user's voice and video data to evaluate their emotional state.

[0146] "User notification means" refers to functions or interfaces for providing users with generated warnings and sentiment evaluation results.

[0147] This invention is a safety management system that combines wireless communication technology and emotion recognition technology to make home and office environments safer and more comfortable. The terminal senses dynamic objects and human movements by receiving signals emitted from existing wireless communication devices. Specifically, it uses wireless technologies such as Bluetooth and Wi-Fi to acquire data on the position of objects and the movement of people. This data is transmitted to a server in real time, and the security of the information is ensured using a secure protocol.

[0148] The server analyzes the received data using machine learning algorithms to determine the behavior, location, and anomalies of dynamic objects. The machine learning models used are trained on platforms such as TensorFlow and PyTorch. Furthermore, the server acquires user audio and video data via cameras and microphones and evaluates their emotional state through an emotion engine. This evaluation makes it possible to provide appropriate emotion-based feedback while ensuring user safety.

[0149] As a concrete example, in an office environment, if the system detects that a user is experiencing stress, the server will notify the user via their terminal with music to promote relaxation. Furthermore, if abnormal activity is detected, an immediate warning can be issued, prompting a quick response.

[0150] Examples of prompts for a generative AI model:

[0151] I want to develop an emotion recognition system for use in the office. It would use wireless communication devices to sense movement and analyze the user's emotional state from their voice and facial expressions. This system would send notifications to users who are feeling stressed, encouraging them to relax.

[0152] This allows users to live and work in a safe and comfortable environment, enjoying a high level of security and convenience.

[0153] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0154] Step 1:

[0155] The terminal receives signals emitted from wireless communication devices and senses the movement of moving objects and human actions. Specifically, it uses Bluetooth and Wi-Fi signals to acquire location information of people and objects. This input information is managed within the system as motion data. Sensed motion data is generated as output.

[0156] Step 2:

[0157] The terminal sends the collected operational data to the server. The data is securely transmitted using protocols such as HTTP and MQTT, and received by the server. The input to this transmission is operational data, and the output is the server's reception of the data.

[0158] Step 3:

[0159] The server analyzes the received operational data using machine learning algorithms. Specifically, models trained using TensorFlow or PyTorch analyze the operational data and detect anomalies. The input is operational data, and the output is the anomaly detection result.

[0160] Step 4:

[0161] The server acquires user audio and video data using a camera and microphone, and analyzes it with an emotion recognition engine. The input is audio and video data, and the emotional state is output. Specifically, it evaluates the user's stress level and relaxation state.

[0162] Step 5:

[0163] The server generates notifications and feedback based on anomaly detection results and emotional states. The inputs are anomaly detection results and emotional states, and the output is a notification to be provided to the user. Specifically, it sends a message encouraging relaxation to users who are feeling stressed.

[0164] Step 6:

[0165] Users receive notifications from the server via their devices. This allows them to take actions such as playing music to reduce stress or adjusting their environment. The input is the notifications from the server, and the output is the user's actions and changes in their environment.

[0166] (Application Example 2)

[0167] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0168] While motion detection technologies utilizing existing wireless communication devices exist, they have not yet achieved simultaneous improvements in environmental safety and occupant comfort. In particular, security management that takes into account the user's emotional state is insufficient, potentially leading to delays in responding to emotional changes, stress, and dangerous situations. A new system that balances safety management and comfort is needed.

[0169] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0170] In this invention, the server includes: information gathering means for sensing the movement of objects and people by utilizing signal reflection from existing wireless communication devices; information transmission means for transmitting information obtained from the information gathering means to the server; analysis means for analyzing the transmitted information and determining the movement and location of people; anomaly detection means for detecting anomalies based on the analysis results and generating notifications; user notification means for providing the generated notifications to the user; and emotion recognition means for analyzing the user's voice and facial information to determine their emotional state and generating notifications to promote relaxation. This makes it possible to adjust the environment to be comfortable according to the user's emotional state while maintaining safety.

[0171] "Information gathering means" refers to a function that uses signal reflection from existing wireless communication devices to acquire information about the movements of objects or people.

[0172] "Information transmission means" refers to a function for transmitting information acquired by information collection means to a server.

[0173] "Analysis means" refers to a function that analyzes information transmitted to the server to determine a person's movements and location.

[0174] An "anomaly detection means" is a function that detects anomalies based on the results of the analysis means and generates notifications as necessary.

[0175] "User notification means" refers to a function that provides generated notifications to users and prompts them to take appropriate action.

[0176] The "emotion recognition means" is a function that analyzes the user's voice and facial expression information to determine their emotional state and, if necessary, generates notifications to promote relaxation.

[0177] This system consists primarily of a wireless communication device, a server, and a smartphone or similar terminal. The wireless communication device constantly monitors the surrounding environment in a home or office setting, collecting information on the movements of objects and people using signal reflection. This information is acquired through the information collection means and transmitted to the server via the terminal.

[0178] The server uses software such as Python and TensorFlow to analyze the received information. From the analyzed data, it determines a person's movements, location, and emotional state. This analysis activates anomaly detection mechanisms, making it possible to detect deviations from normal behavioral patterns and high-stress states. If an anomaly is detected, the server immediately generates a notification, which is presented to the user through the notification mechanism.

[0179] The emotion recognition system uses the smartphone's camera and microphone to collect emotional data from the user's voice and facial expressions. This allows it to determine the user's emotional state and provide information to promote relaxation. For example, if the user is feeling stressed, a selected relaxation method (e.g., playing relaxation music) is suggested.

[0180] For example, if the system determines that a user working in an office is experiencing high stress, a notification will be sent to their smartphone saying, "Play some relaxation music to change your mood." An example of a prompt message for the generating AI model would be, "The emotion engine has detected a high-stress state. Please suggest that the user play some relaxation music."

[0181] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0182] Step 1:

[0183] Wireless communication devices collect motion information of objects and people from the surrounding environment using signal reflection. Signal data is input and output as motion information. This motion information is received by information collection means within the terminal.

[0184] Step 2:

[0185] The terminal transmits operational information obtained from the information gathering device to the server. Operational information is input and output directly to the server as transmitted information. This process utilizes wireless communication technologies such as Bluetooth and Wi-Fi.

[0186] Step 3:

[0187] The server analyzes the received motion information. Here, the input motion information is analyzed using Python or TensorFlow to determine the person's movements and position. Through this analysis, abnormal patterns are generated as output.

[0188] Step 4:

[0189] The server determines an anomaly based on the detection of anomaly patterns and generates a notification as necessary. An anomaly pattern is input, and a notification is output to attract the user's attention. In this step, the anomaly detection means uses a learning algorithm to determine deviations from normal behavior.

[0190] Step 5:

[0191] The server utilizes the camera and microphone of the smartphone or device to collect the user's voice and facial expression data. This data is input, and information is gathered to determine the user's emotional state.

[0192] Step 6:

[0193] The server's emotion recognition system analyzes collected voice and facial expression data to evaluate the user's emotional state. The user's current emotional state is output based on the input emotion data. A generative AI model is used in this process.

[0194] Step 7:

[0195] The server generates and provides notifications to the user to promote relaxation, based on the user's emotional state. The user's emotional state is input, and relaxation suggestions are output. Specifically, it generates a message such as "Play relaxation music" and sends a notification to the terminal. This notification is provided through a user notification system.

[0196] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0197] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search)<url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0198] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0199] [Second Embodiment]

[0200] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0201] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0202] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0203] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0204] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0205] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0206] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0207] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0208] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0209] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0210] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0211] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0212] This invention provides a system that utilizes signals generated by wireless communication devices to improve safety and convenience in home and office environments.

[0213] The terminal senses signals emitted from wireless communication devices installed in homes and offices, captures the characteristics of those signals when reflected by objects or people, and obtains motion and location information. This data is transmitted to the server in real time.

[0214] The server uses machine learning algorithms to analyze the received data and perform pattern recognition. This allows it to identify, for example, whether a resident is in the living room or bedroom, and detect anomalies based on their normal activity patterns.

[0215] If an anomaly is detected, the server immediately generates an alert and notifies the user through a user notification system. The user receives the notification on a device such as a smartphone or PC and takes action as needed. Because this process is automated, a quick and effective response is possible.

[0216] For example, if an elderly person exhibits unusual behavior at night, the user can receive an immediate notification and take appropriate action to ensure their safety. Furthermore, in a workplace environment, it's possible to monitor employee movements and optimize energy consumption.

[0217] This system is a cost-effective solution because it utilizes existing wireless communication devices, eliminating the need for additional hardware installation. Furthermore, AI-powered analysis enables real-time situation assessment and rapid response, significantly improving user convenience and safety.

[0218] The following describes the processing flow.

[0219] Step 1:

[0220] The terminal receives signals emitted from wireless communication devices within a designated area. This collects basic data for detecting changes in signals caused by the movement of objects or people.

[0221] Step 2:

[0222] The terminal analyzes the characteristics of the received signal and records the amplitude and phase changes of the reflected signal as numerical data. This data contains information about the movement and position of objects and people.

[0223] Step 3:

[0224] The terminal sends the analysis results described above to the server in packet format. The transmitted data includes timestamps and location information.

[0225] Step 4:

[0226] The server receives data sent from the terminal in real time. The received data is immediately stored in the database.

[0227] Step 5:

[0228] The server analyzes the received data using machine learning algorithms. Here, it compares the patterns of signal changes with a model to estimate human movement and location.

[0229] Step 6:

[0230] The server detects abnormal behavior or patterns based on the analysis results. If an anomaly is detected, it generates an alert based on pre-configured conditions.

[0231] Step 7:

[0232] Users receive alerts generated by the server on their devices. These alerts include the type of anomaly, the time of occurrence, and recommended actions.

[0233] Step 8:

[0234] Users take necessary actions based on the alert information displayed on their devices. This includes, for example, checking their home environment and taking security measures at work.

[0235] This process allows the system to operate efficiently, providing users with real-time situational awareness and immediate responses.

[0236] (Example 1)

[0237] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0238] There is a need to provide a means of accurately detecting the behavior of entities or people in the environment in real time, without requiring additional hardware, by utilizing existing wireless communication equipment, and to quickly notify users of any anomalies. Conventional systems require additional sensors or equipment to respond to changes in the environment, which presents challenges in terms of cost and installation. Furthermore, there is room for improvement in the speed and accuracy of anomaly detection and notification.

[0239] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0240] In this invention, the server includes information gathering means for detecting the behavior of entities or people in the environment by utilizing signal reflection from wireless communication equipment; information transfer means for transferring the information obtained from the information gathering means to an information processing device; and analysis means for analyzing the transferred information to determine the behavior and location of entities or people. This makes it possible to detect the movement of objects and people in the environment without additional hardware, and to accurately grasp and quickly notify of anomalies in real time.

[0241] "Wireless communication equipment" refers to devices that use radio waves to send and receive voice, data, and other signals, and is a device that exchanges signals within an environment.

[0242] "Signal reflection" refers to the phenomenon where radio waves transmitted from wireless communication devices hit an object or person and return, allowing information about the object's location and movement to be obtained.

[0243] "Information gathering means" refers to a process or device for detecting entities or human behavior in the environment using signal reflection from wireless communication devices and acquiring data related thereto.

[0244] "Information transfer means" refers to a process or device for transmitting data obtained by information collection means to other devices, particularly information processing devices.

[0245] An "information processing device" is a device for analyzing data transmitted by an information transfer means, and includes devices that perform computational processing, such as servers.

[0246] "Analysis means" refers to a method or device that performs processing to determine the behavior and position of an entity or human being based on data received in an information processing device.

[0247] An "anomaly detection means" refers to a process or device that detects events that deviate from normal activity patterns based on results obtained from analysis means, and generates warnings or notifications.

[0248] "User notification means" refers to a process or device for providing users with generated warnings or notifications and prompting them to take necessary action.

[0249] This invention is a system that uses signals emitted from existing wireless communication devices to sense objects and human movements in the environment and detect anomalies. The server, terminal, and user play three main roles.

[0250] The server provides the computing resources necessary to analyze signals from wireless communication devices. As an information processing device, the server runs a generative AI model to analyze the received data. Here, frameworks such as TensorFlow and PyTorch are utilized to train learning algorithms, thereby building a model that learns normal behavioral patterns and detects anomalies.

[0251] The terminal senses signals from wireless communication devices installed in the environment. For example, it receives reflected signals from Wi-Fi routers and Bluetooth devices and acquires them as data using information gathering means. This data is transmitted from the terminal to the server via information transfer means.

[0252] Users receive notifications from their devices and servers and take necessary actions. For example, if unusual movement is detected in an elderly person's room, the user will receive a prompt message such as, "The system detected unusual activity during the night. Do you want to check on the elderly person's safety?" After receiving this notification, the user can use their smartphone or PC to check the situation and take appropriate action.

[0253] The key features of this invention are its cost-effectiveness, as it utilizes existing devices and requires no additional hardware, enabling rapid and accurate anomaly detection and notification. This can enhance safety in the home and efficiency in the workplace.

[0254] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0255] Step 1:

[0256] The terminal receives signals transmitted from wireless communication devices and collects them as data. Inputs include radio waves from Wi-Fi routers and Bluetooth devices, which are reflected when they strike objects and people. The output is data indicating the characteristics of the reflected signals. This data includes information such as signal strength, arrival time, and phase. Specifically, one scenario involves the terminal detecting the movement of a person in a living room.

[0257] Step 2:

[0258] The terminal sends the collected data to the server for information processing. The input data is signal characteristic data measured by the terminal, which is transferred to the server via the internet connection. The output is the signal characteristic data received by the server. This allows the information processing device to perform data analysis. Specifically, the terminal sends signal data to the server several times per second.

[0259] Step 3:

[0260] The server analyzes the movements and locations of entities or people in the environment based on the received data. The input is signal characteristic data sent to the server, which is then analyzed using a generative AI model. Specifically, TensorFlow and PyTorch are used as the software. The output is information about movements and locations in the environment obtained through the analysis. For example, the server performs an analysis to determine that a person is in a bedroom at a specific time.

[0261] Step 4:

[0262] The server uses the analysis results to detect anomalies in behavioral patterns and generates alerts as needed. Input includes information on movement and location obtained from the analysis system, which is compared to normal patterns to determine anomalies. Output is an alert indicating the anomaly. This alert is sent to the user via a notification system. For example, a message such as "Unusual movement detected in the living room" might be generated.

[0263] Step 5:

[0264] The user receives alerts sent from the server on their device (smartphone or PC). The input is alert information from the server, which is displayed on the receiving device. The output is a notification to the user, providing them with information to take action. Specifically, the user can check the notification and then take appropriate action, such as checking the safety of the elderly person, as needed.

[0265] (Application Example 1)

[0266] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0267] The present invention aims to improve safety and convenience by using wireless communication devices to sense the movements of objects and people in a space in real time, enabling anomaly detection and rapid notification. In particular, it aims to support emergency response and enhance a sense of security in the living environment by quickly identifying abnormal behavior in the elderly or in specific environments.

[0268] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0269] In this invention, the server includes information acquisition means, data transmission means, and analysis means. This enables precise motion detection using a wireless communication device and rapid detection of abnormal behavior.

[0270] An "information acquisition means" is a device that uses signal reflection from existing wireless communication devices to detect the movement of objects or people.

[0271] A "data transmission means" is a device for transmitting information obtained from an information acquisition means to a data processing device.

[0272] An "analysis device" is a device that analyzes transmitted information to determine a person's movements and location.

[0273] An "anomaly detection device" is a device that detects anomalies based on analysis results and generates an alert.

[0274] A "user notification device" is a device for providing generated alerts to the user's terminal.

[0275] "Application means" refers to software installed on a user's portable information terminal that displays warnings in real time when an anomaly is detected.

[0276] The system for carrying out this invention is configured as follows.

[0277] The server receives signals from wireless communication devices and processes the data obtained by the information acquisition means. This data is transmitted to the server using the data transmission means. The server analyzes the transmitted information using the analysis means to determine human movement and location. The anomaly detection means detects anomalies based on the analysis results, and if an anomaly is detected, the server immediately generates an alert. This alert is communicated to the user terminal by the user notification means, and the user receives the warning through the application means installed on their mobile device. This application notifies the user of anomalies in real time.

[0278] Specifically, the server uses machine learning algorithms for data processing and utilizes software such as TensorFlow and scikit-learn. This makes it possible to detect abnormal behavior compared to everyday behavioral patterns. For example, if abnormal movement is detected in an elderly person at night, the application is designed to send a notification to their smartphone stating, "Unusual movement has been detected. Please check your safety."

[0279] An example of a prompt message would be: "Develop a machine learning model for detecting abnormal behavior using wireless signal data within a home. In particular, we need a model that can distinguish between normal and abnormal behavior in the living room and bedroom."

[0280] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0281] Step 1: The server receives the signal transmitted from the wireless communication device. The input is the intensity data of the wireless signal, and the output is the data collection result by the information acquisition means. In this step, the signal reception and the data collection based on it are specifically performed.

[0282] Step 2: The server transmits the numerical data obtained from the information acquisition means into the server using the data transmission means. The input is the data collection result, and the output is the data in a form that can be processed within the server. In this step, the data format conversion and network transmission are performed.

[0283] Step 3: The server analyzes the data using the analysis means to determine the actions and positions of people. The input is the data in a form that can be processed, and the output is the identification result of the actions and positions. In this step, the machine learning algorithm is utilized, and TensorFlow or scikit - learn is used to analyze the action patterns.

[0284] Step 4: The server uses the anomaly detection means to detect anomalies based on the analysis results and generate alerts. The input is the identification result of the actions and positions, and the output is the alert information indicating anomalies. In this step, the learning model for detecting deviations from the normal behavior pattern is utilized.

[0285] Step 5: The server transmits the generated alert to the user terminal by the user notification means. The input is the alert information, and the output is the notification information transmitted to the user. In this step, the network protocol for transmitting notifications in real - time is utilized.

[0286] Step 6: The user receives the notification through the application means on the mobile information terminal in use. The input is the notification information, and the output is the warning message visually displayed. In this step, the application performs the operation of presenting the notification content to the user visually and intuitively.

[0287] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0288] This invention combines a safety management system that utilizes signals from wireless communication devices with a user emotion recognition function, aiming to make home and office environments safer and more comfortable.

[0289] The terminal receives signals emitted from existing wireless communication devices and senses the movement of objects and people. This allows it to collect data on the movement and location of the target. This data is transmitted to a server in real time and analyzed appropriately. Machine learning models are used in this analysis to detect human movement and anomalies.

[0290] Furthermore, the server is equipped with an emotion engine that can evaluate the user's emotional state by analyzing their voice and facial expression data. The emotion engine acquires voice and video data through interfaces such as cameras and microphones and determines whether the user's emotions deviate from the normal range.

[0291] For example, the system can sense whether a resident is relaxed or stressed. Based on this emotional state, the server adjusts notifications and actions to provide the user with more relevant information. For instance, if a user is stressed, the system might provide information to encourage relaxation.

[0292] Users can receive these notifications via their devices and take the most appropriate action based on their environment. This system enables flexible and effective management of living and working environments based on detailed information, including the user's emotional state. Furthermore, if an anomaly is detected based on data analysis, the system quickly alerts the user and prompts them to take the necessary action, thereby ensuring user safety.

[0293] This invention offers high security and convenience while being an inexpensive device, making it suitable for a wide range of applications in homes and offices. Furthermore, its emotion-based recognition capabilities could contribute to health management and mental support. This would allow users to enjoy an environment where they can focus on their lives and work with greater peace of mind.

[0294] The following describes the processing flow.

[0295] Step 1:

[0296] The terminal receives radio waves emitted from wireless communication devices and detects signals reflected by surrounding objects and people. This allows it to collect basic data for recording environmental changes in real time.

[0297] Step 2:

[0298] The terminal analyzes the received signal data and converts the movement and position of objects and people into numerical data. This data includes information such as speed and direction of movement, enabling detailed motion analysis.

[0299] Step 3:

[0300] The terminal sends the analyzed data to the server. Security protocols are used for communication, ensuring data confidentiality during transmission.

[0301] Step 4:

[0302] The server receives data sent from the terminal and stores it in the database. The received data is immediately ready for processing.

[0303] Step 5:

[0304] The server analyzes the transmitted data using machine learning algorithms to determine the person's movements and location in detail. It also detects movements that deviate from normal patterns as anomalies.

[0305] Step 6:

[0306] The server collects and analyzes the user's voice and video through the emotion engine. Thereby, the server confirms the user's emotional state and determines, for example, whether the user is feeling stressed.

[0307] Step 7:

[0308] The server integrates and analyzes the user's motion data and emotion data and generates a notification if necessary. If the user is feeling stressed, it may be considered to prepare a notification to promote relaxation.

[0309] Step 8:

[0310] The user receives the notification sent from the server on the terminal. The notification describes an action suitable for the user and provides audio and visual guidance.

[0311] Step 9:

[0312] The user takes the necessary actions referring to the notification content. If at home, the user can adjust the lighting or play background music, and in the office, it provides a clue for considering how to take a break.

[0313] Through this series of processes, the system can monitor both the user's movement and emotion in real time and provide a safer and more comfortable environment.

[0314] (Example 2)

[0315] Next, Example 2 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0316] In modern living environments, there is a need for systems that utilize wireless communication technology to provide safe and comfortable living and working environments. However, existing systems have limitations in detecting the movement and anomalies of dynamic objects, and may not be able to provide appropriate responses that take into account the user's emotional state. Furthermore, there is a lack of effective means to comprehensively manage user safety and mental comfort.

[0317] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0318] In this invention, the server includes information gathering means, information transmission means, analysis means, anomaly detection means, emotion recognition means, and user notification means. This makes it possible to analyze the behavior and emotional state of dynamic targets using wireless communication devices in real time, evaluate anomalies and the user's emotional state, and provide appropriate notifications.

[0319] "Information gathering means" refers to devices that receive signals from wireless communication equipment and sense dynamic objects or human movements.

[0320] "Information transmission means" refers to a function for transmitting data acquired by information collection means to a computing device (server).

[0321] "Analysis means" refers to processes and algorithms for determining the movement and position of a dynamic object based on transmitted information.

[0322] An "anomaly detection method" is a technology that detects anomalies that deviate from normal operating patterns based on analysis results and generates warnings.

[0323] "Emotion recognition means" refers to technology that analyzes a user's voice and video data to evaluate their emotional state.

[0324] "User notification means" refers to functions or interfaces for providing users with generated warnings and sentiment evaluation results.

[0325] This invention is a safety management system that combines wireless communication technology and emotion recognition technology to make home and office environments safer and more comfortable. The terminal senses dynamic objects and human movements by receiving signals emitted from existing wireless communication devices. Specifically, it uses wireless technologies such as Bluetooth and Wi-Fi to acquire data on the position of objects and the movement of people. This data is transmitted to a server in real time, and the security of the information is ensured using a secure protocol.

[0326] The server analyzes the received data using machine learning algorithms to determine the behavior, location, and anomalies of dynamic objects. The machine learning models used are trained on platforms such as TensorFlow and PyTorch. Furthermore, the server acquires user audio and video data via cameras and microphones and evaluates their emotional state through an emotion engine. This evaluation makes it possible to provide appropriate emotion-based feedback while ensuring user safety.

[0327] As a concrete example, in an office environment, if the system detects that a user is experiencing stress, the server will notify the user via their terminal with music to promote relaxation. Furthermore, if abnormal activity is detected, an immediate warning can be issued, prompting a quick response.

[0328] Examples of prompts for a generative AI model:

[0329] I want to develop an emotion recognition system for use in the office. It would use wireless communication devices to sense movement and analyze the user's emotional state from their voice and facial expressions. This system would send notifications to users who are feeling stressed, encouraging them to relax.

[0330] This allows users to live and work in a safe and comfortable environment, enjoying a high level of security and convenience.

[0331] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0332] Step 1:

[0333] The terminal receives signals emitted from wireless communication devices and senses the movement of moving objects and human actions. Specifically, it uses Bluetooth and Wi-Fi signals to acquire location information of people and objects. This input information is managed within the system as motion data. Sensed motion data is generated as output.

[0334] Step 2:

[0335] The terminal sends the collected operational data to the server. The data is securely transmitted using protocols such as HTTP and MQTT, and received by the server. The input to this transmission is operational data, and the output is the server's reception of the data.

[0336] Step 3:

[0337] The server analyzes the received operational data using machine learning algorithms. Specifically, models trained using TensorFlow or PyTorch analyze the operational data and detect anomalies. The input is operational data, and the output is the anomaly detection result.

[0338] Step 4:

[0339] The server acquires user audio and video data using a camera and microphone, and analyzes it with an emotion recognition engine. The input is audio and video data, and the emotional state is output. Specifically, it evaluates the user's stress level and relaxation state.

[0340] Step 5:

[0341] The server generates notifications and feedback based on anomaly detection results and emotional states. The inputs are anomaly detection results and emotional states, and the output is a notification to be provided to the user. Specifically, it sends a message encouraging relaxation to users who are feeling stressed.

[0342] Step 6:

[0343] Users receive notifications from the server via their devices. This allows them to take actions such as playing music to reduce stress or adjusting their environment. The input is the notifications from the server, and the output is the user's actions and changes in their environment.

[0344] (Application Example 2)

[0345] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0346] While motion detection technologies utilizing existing wireless communication devices exist, they have not yet achieved simultaneous improvements in environmental safety and occupant comfort. In particular, security management that takes into account the user's emotional state is insufficient, potentially leading to delays in responding to emotional changes, stress, and dangerous situations. A new system that balances safety management and comfort is needed.

[0347] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0348] In this invention, the server includes: information gathering means for sensing the movement of objects and people by utilizing signal reflection from existing wireless communication devices; information transmission means for transmitting information obtained from the information gathering means to the server; analysis means for analyzing the transmitted information and determining the movement and location of people; anomaly detection means for detecting anomalies based on the analysis results and generating notifications; user notification means for providing the generated notifications to the user; and emotion recognition means for analyzing the user's voice and facial information to determine their emotional state and generating notifications to promote relaxation. This makes it possible to adjust the environment to be comfortable according to the user's emotional state while maintaining safety.

[0349] "Information gathering means" refers to a function that uses signal reflection from existing wireless communication devices to acquire information about the movements of objects or people.

[0350] "Information transmission means" refers to a function for transmitting information acquired by information collection means to a server.

[0351] "Analysis means" refers to a function that analyzes information transmitted to the server to determine a person's movements and location.

[0352] An "anomaly detection means" is a function that detects anomalies based on the results of the analysis means and generates notifications as necessary.

[0353] "User notification means" refers to a function that provides generated notifications to users and prompts them to take appropriate action.

[0354] The "emotion recognition means" is a function that analyzes the user's voice and facial expression information to determine their emotional state and, if necessary, generates notifications to promote relaxation.

[0355] This system consists primarily of a wireless communication device, a server, and a smartphone or similar terminal. The wireless communication device constantly monitors the surrounding environment in a home or office setting, collecting information on the movements of objects and people using signal reflection. This information is acquired through the information collection means and transmitted to the server via the terminal.

[0356] The server uses software such as Python and TensorFlow to analyze the received information. From the analyzed data, it determines a person's movements, location, and emotional state. This analysis activates anomaly detection mechanisms, making it possible to detect deviations from normal behavioral patterns and high-stress states. If an anomaly is detected, the server immediately generates a notification, which is presented to the user through the notification mechanism.

[0357] The emotion recognition system uses the smartphone's camera and microphone to collect emotional data from the user's voice and facial expressions. This allows it to determine the user's emotional state and provide information to promote relaxation. For example, if the user is feeling stressed, a selected relaxation method (e.g., playing relaxation music) is suggested.

[0358] For example, if the system determines that a user working in an office is experiencing high stress, a notification will be sent to their smartphone saying, "Play some relaxation music to change your mood." An example of a prompt message for the generating AI model would be, "The emotion engine has detected a high-stress state. Please suggest that the user play some relaxation music."

[0359] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0360] Step 1:

[0361] Wireless communication devices collect motion information of objects and people from the surrounding environment using signal reflection. Signal data is input and output as motion information. This motion information is received by information collection means within the terminal.

[0362] Step 2:

[0363] The terminal transmits operational information obtained from the information gathering device to the server. Operational information is input and output directly to the server as transmitted information. This process utilizes wireless communication technologies such as Bluetooth and Wi-Fi.

[0364] Step 3:

[0365] The server analyzes the received motion information. Here, the input motion information is analyzed using Python or TensorFlow to determine the person's movements and position. Through this analysis, abnormal patterns are generated as output.

[0366] Step 4:

[0367] The server determines an anomaly based on the detection of anomaly patterns and generates a notification as necessary. An anomaly pattern is input, and a notification is output to attract the user's attention. In this step, the anomaly detection means uses a learning algorithm to determine deviations from normal behavior.

[0368] Step 5:

[0369] The server utilizes the camera and microphone of the smartphone or device to collect the user's voice and facial expression data. This data is input, and information is gathered to determine the user's emotional state.

[0370] Step 6:

[0371] The server's emotion recognition system analyzes collected voice and facial expression data to evaluate the user's emotional state. The user's current emotional state is output based on the input emotion data. A generative AI model is used in this process.

[0372] Step 7:

[0373] The server generates and provides notifications to the user to promote relaxation, based on the user's emotional state. The user's emotional state is input, and relaxation suggestions are output. Specifically, it generates a message such as "Play relaxation music" and sends a notification to the terminal. This notification is provided through a user notification system.

[0374] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0375] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0376] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0377] [Third Embodiment]

[0378] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0379] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0380] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0381] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0382] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0383] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0384] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0385] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0386] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0387] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0388] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0389] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0390] This invention provides a system that utilizes signals generated by wireless communication devices to improve safety and convenience in home and office environments.

[0391] The terminal senses signals emitted from wireless communication devices installed in homes and offices, captures the characteristics of those signals when reflected by objects or people, and obtains motion and location information. This data is transmitted to the server in real time.

[0392] The server uses machine learning algorithms to analyze the received data and perform pattern recognition. This allows it to identify, for example, whether a resident is in the living room or bedroom, and detect anomalies based on their normal activity patterns.

[0393] If an anomaly is detected, the server immediately generates an alert and notifies the user through a user notification system. The user receives the notification on a device such as a smartphone or PC and takes action as needed. Because this process is automated, a quick and effective response is possible.

[0394] For example, if an elderly person exhibits unusual behavior at night, the user can receive an immediate notification and take appropriate action to ensure their safety. Furthermore, in a workplace environment, it's possible to monitor employee movements and optimize energy consumption.

[0395] This system is a cost-effective solution because it utilizes existing wireless communication devices, eliminating the need for additional hardware installation. Furthermore, AI-powered analysis enables real-time situation assessment and rapid response, significantly improving user convenience and safety.

[0396] The following describes the processing flow.

[0397] Step 1:

[0398] The terminal receives signals emitted from wireless communication devices within a designated area. This collects basic data for detecting changes in signals caused by the movement of objects or people.

[0399] Step 2:

[0400] The terminal analyzes the characteristics of the received signal and records the amplitude and phase changes of the reflected signal as numerical data. This data contains information about the movement and position of objects and people.

[0401] Step 3:

[0402] The terminal sends the analysis results described above to the server in packet format. The transmitted data includes timestamps and location information.

[0403] Step 4:

[0404] The server receives data sent from the terminal in real time. The received data is immediately stored in the database.

[0405] Step 5:

[0406] The server analyzes the received data using machine learning algorithms. Here, it compares the patterns of signal changes with a model to estimate human movement and location.

[0407] Step 6:

[0408] The server detects abnormal behavior or patterns based on the analysis results. If an anomaly is detected, it generates an alert based on pre-configured conditions.

[0409] Step 7:

[0410] Users receive alerts generated by the server on their devices. These alerts include the type of anomaly, the time of occurrence, and recommended actions.

[0411] Step 8:

[0412] Users take necessary actions based on the alert information displayed on their devices. This includes, for example, checking their home environment and taking security measures at work.

[0413] This process allows the system to operate efficiently, providing users with real-time situational awareness and immediate responses.

[0414] (Example 1)

[0415] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0416] There is a need to provide a means of accurately detecting the behavior of entities or people in the environment in real time, without requiring additional hardware, by utilizing existing wireless communication equipment, and to quickly notify users of any anomalies. Conventional systems require additional sensors or equipment to respond to changes in the environment, which presents challenges in terms of cost and installation. Furthermore, there is room for improvement in the speed and accuracy of anomaly detection and notification.

[0417] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0418] In this invention, the server includes information gathering means for detecting the behavior of entities or people in the environment by utilizing signal reflection from wireless communication equipment; information transfer means for transferring the information obtained from the information gathering means to an information processing device; and analysis means for analyzing the transferred information to determine the behavior and location of entities or people. This makes it possible to detect the movement of objects and people in the environment without additional hardware, and to accurately grasp and quickly notify of anomalies in real time.

[0419] "Wireless communication equipment" refers to devices that use radio waves to send and receive voice, data, and other signals, and is a device that exchanges signals within an environment.

[0420] "Signal reflection" refers to the phenomenon where radio waves transmitted from wireless communication devices hit an object or person and return, allowing information about the object's location and movement to be obtained.

[0421] "Information gathering means" refers to a process or device for detecting entities or human behavior in the environment using signal reflection from wireless communication devices and acquiring data related thereto.

[0422] "Information transfer means" refers to a process or device for transmitting data obtained by information collection means to other devices, particularly information processing devices.

[0423] An "information processing device" is a device for analyzing data transmitted by an information transfer means, and includes devices that perform computational processing, such as servers.

[0424] "Analysis means" refers to a method or device that performs processing to determine the behavior and position of an entity or human being based on data received in an information processing device.

[0425] An "anomaly detection means" refers to a process or device that detects events that deviate from normal activity patterns based on results obtained from analysis means, and generates warnings or notifications.

[0426] "User notification means" refers to a process or device for providing users with generated warnings or notifications and prompting them to take necessary action.

[0427] This invention is a system that uses signals emitted from existing wireless communication devices to sense objects and human movements in the environment and detect anomalies. The server, terminal, and user play three main roles.

[0428] The server provides the computing resources necessary to analyze signals from wireless communication devices. As an information processing device, the server runs a generative AI model to analyze the received data. Here, frameworks such as TensorFlow and PyTorch are utilized to train learning algorithms, thereby building a model that learns normal behavioral patterns and detects anomalies.

[0429] The terminal senses signals from wireless communication devices installed in the environment. For example, it receives reflected signals from Wi-Fi routers and Bluetooth devices and acquires them as data using information gathering means. This data is transmitted from the terminal to the server via information transfer means.

[0430] Users receive notifications from their devices and servers and take necessary actions. For example, if unusual movement is detected in an elderly person's room, the user will receive a prompt message such as, "The system detected unusual activity during the night. Do you want to check on the elderly person's safety?" After receiving this notification, the user can use their smartphone or PC to check the situation and take appropriate action.

[0431] The key features of this invention are its cost-effectiveness, as it utilizes existing devices and requires no additional hardware, enabling rapid and accurate anomaly detection and notification. This can enhance safety in the home and efficiency in the workplace.

[0432] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0433] Step 1:

[0434] The terminal receives signals transmitted from wireless communication devices and collects them as data. Inputs include radio waves from Wi-Fi routers and Bluetooth devices, which are reflected when they strike objects and people. The output is data indicating the characteristics of the reflected signals. This data includes information such as signal strength, arrival time, and phase. Specifically, one scenario involves the terminal detecting the movement of a person in a living room.

[0435] Step 2:

[0436] The terminal sends the collected data to the server for information processing. The input data is signal characteristic data measured by the terminal, which is transferred to the server via the internet connection. The output is the signal characteristic data received by the server. This allows the information processing device to perform data analysis. Specifically, the terminal sends signal data to the server several times per second.

[0437] Step 3:

[0438] The server analyzes the movements and locations of entities or people in the environment based on the received data. The input is signal characteristic data sent to the server, which is then analyzed using a generative AI model. Specifically, TensorFlow and PyTorch are used as the software. The output is information about movements and locations in the environment obtained through the analysis. For example, the server performs an analysis to determine that a person is in a bedroom at a specific time.

[0439] Step 4:

[0440] The server uses the analysis results to detect anomalies in behavioral patterns and generates alerts as needed. Input includes information on movement and location obtained from the analysis system, which is compared to normal patterns to determine anomalies. Output is an alert indicating the anomaly. This alert is sent to the user via a notification system. For example, a message such as "Unusual movement detected in the living room" might be generated.

[0441] Step 5:

[0442] The user receives alerts sent from the server on their device (smartphone or PC). The input is alert information from the server, which is displayed on the receiving device. The output is a notification to the user, providing them with information to take action. Specifically, the user can check the notification and then take appropriate action, such as checking the safety of the elderly person, as needed.

[0443] (Application Example 1)

[0444] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0445] The present invention aims to improve safety and convenience by using wireless communication devices to sense the movements of objects and people in a space in real time, enabling anomaly detection and rapid notification. In particular, it aims to support emergency response and enhance a sense of security in the living environment by quickly identifying abnormal behavior in the elderly or in specific environments.

[0446] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0447] In this invention, the server includes information acquisition means, data transmission means, and analysis means. This enables precise motion detection using a wireless communication device and rapid detection of abnormal behavior.

[0448] An "information acquisition means" is a device that uses signal reflection from existing wireless communication devices to detect the movement of objects or people.

[0449] A "data transmission means" is a device for transmitting information obtained from an information acquisition means to a data processing device.

[0450] An "analysis device" is a device that analyzes transmitted information to determine a person's movements and location.

[0451] An "anomaly detection device" is a device that detects anomalies based on analysis results and generates an alert.

[0452] A "user notification device" is a device for providing generated alerts to the user's terminal.

[0453] "Application means" refers to software installed on a user's portable information terminal that displays warnings in real time when an anomaly is detected.

[0454] The system for carrying out this invention is configured as follows.

[0455] The server receives signals from wireless communication devices and processes the data obtained by the information acquisition means. This data is transmitted to the server using the data transmission means. The server analyzes the transmitted information using the analysis means to determine human movement and location. The anomaly detection means detects anomalies based on the analysis results, and if an anomaly is detected, the server immediately generates an alert. This alert is communicated to the user terminal by the user notification means, and the user receives the warning through the application means installed on their mobile device. This application notifies the user of anomalies in real time.

[0456] Specifically, the server uses machine learning algorithms for data processing and utilizes software such as TensorFlow and scikit-learn. This makes it possible to detect abnormal behavior compared to everyday behavioral patterns. For example, if abnormal movement is detected in an elderly person at night, the application is designed to send a notification to their smartphone stating, "Unusual movement has been detected. Please check your safety."

[0457] An example of a prompt message would be: "Develop a machine learning model for detecting abnormal behavior using wireless signal data within a home. In particular, we need a model that can distinguish between normal and abnormal behavior in the living room and bedroom."

[0458] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0459] Step 1: The server receives signals transmitted from the wireless communication device. The input is wireless signal strength data, and the output is the data collection result from the information acquisition means. In this step, the reception of the signal and the collection of data based on it are specifically carried out.

[0460] Step 2: The server transmits numerical data obtained from the information acquisition means to the server using the data transmission means. The input is the data collection result, and the output is data in a format that can be processed within the server. This step involves data format conversion and network transmission.

[0461] Step 3: The server analyzes the data using analytical tools to determine human movement and location. The input is data in a processable format, and the output is the identification result of movement and location. In this step, machine learning algorithms are used to analyze movement patterns using TensorFlow or scikit-learn.

[0462] Step 4: The server uses anomaly detection means to detect anomalies based on the analysis results and generates an alert. The input is the identification result of movement and location, and the output is alert information indicating an anomaly. In this step, a learning model is used to detect deviations from normal behavior patterns.

[0463] Step 5: The server sends the generated alert to the user terminal via a user notification mechanism. The input is the alert information, and the output is the notification information sent to the user. This step utilizes a network protocol for sending notifications in real time.

[0464] Step 6: The user receives a notification through an application on their mobile device. The input is notification information, and the output is a visually displayed warning message. In this step, the application visually and intuitively presents the notification content to the user.

[0465] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0466] This invention combines a safety management system that utilizes signals from wireless communication devices with a user emotion recognition function, aiming to make home and office environments safer and more comfortable.

[0467] The terminal receives signals emitted from existing wireless communication devices and senses the movement of objects and people. This allows it to collect data on the movement and location of the target. This data is transmitted to a server in real time and analyzed appropriately. Machine learning models are used in this analysis to detect human movement and anomalies.

[0468] Furthermore, the server is equipped with an emotion engine that can evaluate the user's emotional state by analyzing their voice and facial expression data. The emotion engine acquires voice and video data through interfaces such as cameras and microphones and determines whether the user's emotions deviate from the normal range.

[0469] For example, the system can sense whether a resident is relaxed or stressed. Based on this emotional state, the server adjusts notifications and actions to provide the user with more relevant information. For instance, if a user is stressed, the system might provide information to encourage relaxation.

[0470] Users can receive these notifications via their devices and take the most appropriate action based on their environment. This system enables flexible and effective management of living and working environments based on detailed information, including the user's emotional state. Furthermore, if an anomaly is detected based on data analysis, the system quickly alerts the user and prompts them to take the necessary action, thereby ensuring user safety.

[0471] This invention offers high security and convenience while being an inexpensive device, making it suitable for a wide range of applications in homes and offices. Furthermore, its emotion-based recognition capabilities could contribute to health management and mental support. This would allow users to enjoy an environment where they can focus on their lives and work with greater peace of mind.

[0472] The following describes the processing flow.

[0473] Step 1:

[0474] The terminal receives radio waves emitted from wireless communication devices and detects signals reflected by surrounding objects and people. This allows it to collect basic data for recording environmental changes in real time.

[0475] Step 2:

[0476] The terminal analyzes the received signal data and converts the movement and position of objects and people into numerical data. This data includes information such as speed and direction of movement, enabling detailed motion analysis.

[0477] Step 3:

[0478] The terminal sends the analyzed data to the server. Security protocols are used for communication, ensuring data confidentiality during transmission.

[0479] Step 4:

[0480] The server receives data sent from the terminal and stores it in the database. The received data is immediately ready for processing.

[0481] Step 5:

[0482] The server analyzes the transmitted data using machine learning algorithms to determine the person's movements and location in detail. It also detects movements that deviate from normal patterns as anomalies.

[0483] Step 6:

[0484] The server collects and analyzes the user's voice and video through an emotion engine. This allows it to determine the user's emotional state, for example, whether they are feeling stressed.

[0485] Step 7:

[0486] The server integrates and analyzes user behavior and emotional data, and generates notifications as needed. If a user is experiencing stress, it might prepare a notification to encourage relaxation.

[0487] Step 8:

[0488] Users receive notifications sent from the server on their devices. These notifications include appropriate actions for the user and provide audio and visual guidance.

[0489] Step 9:

[0490] Users can take necessary actions based on the notification content. At home, this might involve adjusting lighting or playing background music; in the office, it might provide clues for deciding how to take breaks.

[0491] This entire process allows the system to monitor both user behavior and emotions in real time, providing a safer and more comfortable environment.

[0492] (Example 2)

[0493] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0494] In modern living environments, there is a need for systems that utilize wireless communication technology to provide safe and comfortable living and working environments. However, existing systems have limitations in detecting the movement and anomalies of dynamic objects, and may not be able to provide appropriate responses that take into account the user's emotional state. Furthermore, there is a lack of effective means to comprehensively manage user safety and mental comfort.

[0495] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0496] In this invention, the server includes information gathering means, information transmission means, analysis means, anomaly detection means, emotion recognition means, and user notification means. This makes it possible to analyze the behavior and emotional state of dynamic targets using wireless communication devices in real time, evaluate anomalies and the user's emotional state, and provide appropriate notifications.

[0497] "Information gathering means" refers to devices that receive signals from wireless communication equipment and sense dynamic objects or human movements.

[0498] "Information transmission means" refers to a function for transmitting data acquired by information collection means to a computing device (server).

[0499] "Analysis means" refers to processes and algorithms for determining the movement and position of a dynamic object based on transmitted information.

[0500] An "anomaly detection method" is a technology that detects anomalies that deviate from normal operating patterns based on analysis results and generates warnings.

[0501] "Emotion recognition means" refers to technology that analyzes a user's voice and video data to evaluate their emotional state.

[0502] "User notification means" refers to functions or interfaces for providing users with generated warnings and sentiment evaluation results.

[0503] This invention is a safety management system that combines wireless communication technology and emotion recognition technology to make home and office environments safer and more comfortable. The terminal senses dynamic objects and human movements by receiving signals emitted from existing wireless communication devices. Specifically, it uses wireless technologies such as Bluetooth and Wi-Fi to acquire data on the position of objects and the movement of people. This data is transmitted to a server in real time, and the security of the information is ensured using a secure protocol.

[0504] The server analyzes the received data using machine learning algorithms to determine the behavior, location, and anomalies of dynamic objects. The machine learning models used are trained on platforms such as TensorFlow and PyTorch. Furthermore, the server acquires user audio and video data via cameras and microphones and evaluates their emotional state through an emotion engine. This evaluation makes it possible to provide appropriate emotion-based feedback while ensuring user safety.

[0505] As a concrete example, in an office environment, if the system detects that a user is experiencing stress, the server will notify the user via their terminal with music to promote relaxation. Furthermore, if abnormal activity is detected, an immediate warning can be issued, prompting a quick response.

[0506] Examples of prompts for a generative AI model:

[0507] I want to develop an emotion recognition system for use in the office. It would use wireless communication devices to sense movement and analyze the user's emotional state from their voice and facial expressions. This system would send notifications to users who are feeling stressed, encouraging them to relax.

[0508] This allows users to live and work in a safe and comfortable environment, enjoying a high level of security and convenience.

[0509] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0510] Step 1:

[0511] The terminal receives signals emitted from wireless communication devices and senses the movement of moving objects and human actions. Specifically, it uses Bluetooth and Wi-Fi signals to acquire location information of people and objects. This input information is managed within the system as motion data. Sensed motion data is generated as output.

[0512] Step 2:

[0513] The terminal sends the collected operational data to the server. The data is securely transmitted using protocols such as HTTP and MQTT, and received by the server. The input to this transmission is operational data, and the output is the server's reception of the data.

[0514] Step 3:

[0515] The server analyzes the received operational data using machine learning algorithms. Specifically, models trained using TensorFlow or PyTorch analyze the operational data and detect anomalies. The input is operational data, and the output is the anomaly detection result.

[0516] Step 4:

[0517] The server acquires user audio and video data using a camera and microphone, and analyzes it with an emotion recognition engine. The input is audio and video data, and the emotional state is output. Specifically, it evaluates the user's stress level and relaxation state.

[0518] Step 5:

[0519] The server generates notifications and feedback based on anomaly detection results and emotional states. The inputs are anomaly detection results and emotional states, and the output is a notification to be provided to the user. Specifically, it sends a message encouraging relaxation to users who are feeling stressed.

[0520] Step 6:

[0521] Users receive notifications from the server via their devices. This allows them to take actions such as playing music to reduce stress or adjusting their environment. The input is the notifications from the server, and the output is the user's actions and changes in their environment.

[0522] (Application Example 2)

[0523] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0524] While motion detection technologies utilizing existing wireless communication devices exist, they have not yet achieved simultaneous improvements in environmental safety and occupant comfort. In particular, security management that takes into account the user's emotional state is insufficient, potentially leading to delays in responding to emotional changes, stress, and dangerous situations. A new system that balances safety management and comfort is needed.

[0525] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0526] In this invention, the server includes: information gathering means for sensing the movement of objects and people by utilizing signal reflection from existing wireless communication devices; information transmission means for transmitting information obtained from the information gathering means to the server; analysis means for analyzing the transmitted information and determining the movement and location of people; anomaly detection means for detecting anomalies based on the analysis results and generating notifications; user notification means for providing the generated notifications to the user; and emotion recognition means for analyzing the user's voice and facial information to determine their emotional state and generating notifications to promote relaxation. This makes it possible to adjust the environment to be comfortable according to the user's emotional state while maintaining safety.

[0527] "Information gathering means" refers to a function that uses signal reflection from existing wireless communication devices to acquire information about the movements of objects or people.

[0528] "Information transmission means" refers to a function for transmitting information acquired by information collection means to a server.

[0529] "Analysis means" refers to a function that analyzes information transmitted to the server to determine a person's movements and location.

[0530] An "anomaly detection means" is a function that detects anomalies based on the results of the analysis means and generates notifications as necessary.

[0531] "User notification means" refers to a function that provides generated notifications to users and prompts them to take appropriate action.

[0532] The "emotion recognition means" is a function that analyzes the user's voice and facial expression information to determine their emotional state and, if necessary, generates notifications to promote relaxation.

[0533] This system consists primarily of a wireless communication device, a server, and a smartphone or similar terminal. The wireless communication device constantly monitors the surrounding environment in a home or office setting, collecting information on the movements of objects and people using signal reflection. This information is acquired through the information collection means and transmitted to the server via the terminal.

[0534] The server uses software such as Python and TensorFlow to analyze the received information. From the analyzed data, it determines a person's movements, location, and emotional state. This analysis activates anomaly detection mechanisms, making it possible to detect deviations from normal behavioral patterns and high-stress states. If an anomaly is detected, the server immediately generates a notification, which is presented to the user through the notification mechanism.

[0535] The emotion recognition system uses the smartphone's camera and microphone to collect emotional data from the user's voice and facial expressions. This allows it to determine the user's emotional state and provide information to promote relaxation. For example, if the user is feeling stressed, a selected relaxation method (e.g., playing relaxation music) is suggested.

[0536] For example, if the system determines that a user working in an office is experiencing high stress, a notification will be sent to their smartphone saying, "Play some relaxation music to change your mood." An example of a prompt message for the generating AI model would be, "The emotion engine has detected a high-stress state. Please suggest that the user play some relaxation music."

[0537] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0538] Step 1:

[0539] Wireless communication devices collect motion information of objects and people from the surrounding environment using signal reflection. Signal data is input and output as motion information. This motion information is received by information collection means within the terminal.

[0540] Step 2:

[0541] The terminal transmits operational information obtained from the information gathering device to the server. Operational information is input and output directly to the server as transmitted information. This process utilizes wireless communication technologies such as Bluetooth and Wi-Fi.

[0542] Step 3:

[0543] The server analyzes the received motion information. Here, the input motion information is analyzed using Python or TensorFlow to determine the person's movements and position. Through this analysis, abnormal patterns are generated as output.

[0544] Step 4:

[0545] The server determines an anomaly based on the detection of anomaly patterns and generates a notification as necessary. An anomaly pattern is input, and a notification is output to attract the user's attention. In this step, the anomaly detection means uses a learning algorithm to determine deviations from normal behavior.

[0546] Step 5:

[0547] The server utilizes the camera and microphone of the smartphone or device to collect the user's voice and facial expression data. This data is input, and information is gathered to determine the user's emotional state.

[0548] Step 6:

[0549] The server's emotion recognition system analyzes collected voice and facial expression data to evaluate the user's emotional state. The user's current emotional state is output based on the input emotion data. A generative AI model is used in this process.

[0550] Step 7:

[0551] The server generates and provides notifications to the user to promote relaxation, based on the user's emotional state. The user's emotional state is input, and relaxation suggestions are output. Specifically, it generates a message such as "Play relaxation music" and sends a notification to the terminal. This notification is provided through a user notification system.

[0552] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0553] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0554] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0555] [Fourth Embodiment]

[0556] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0557] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0558] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0559] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0560] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0561] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0562] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0563] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0564] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0565] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0566] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0567] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0568] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0569] This invention provides a system that utilizes signals generated by wireless communication devices to improve safety and convenience in home and office environments.

[0570] The terminal senses signals emitted from wireless communication devices installed in homes and offices, captures the characteristics of those signals when reflected by objects or people, and obtains motion and location information. This data is transmitted to the server in real time.

[0571] The server uses machine learning algorithms to analyze the received data and perform pattern recognition. This allows it to identify, for example, whether a resident is in the living room or bedroom, and detect anomalies based on their normal activity patterns.

[0572] If an anomaly is detected, the server immediately generates an alert and notifies the user through a user notification system. The user receives the notification on a device such as a smartphone or PC and takes action as needed. Because this process is automated, a quick and effective response is possible.

[0573] For example, if an elderly person exhibits unusual behavior at night, the user can receive an immediate notification and take appropriate action to ensure their safety. Furthermore, in a workplace environment, it's possible to monitor employee movements and optimize energy consumption.

[0574] This system is a cost-effective solution because it utilizes existing wireless communication devices, eliminating the need for additional hardware installation. Furthermore, AI-powered analysis enables real-time situation assessment and rapid response, significantly improving user convenience and safety.

[0575] The following describes the processing flow.

[0576] Step 1:

[0577] The terminal receives signals emitted from wireless communication devices within a designated area. This collects basic data for detecting changes in signals caused by the movement of objects or people.

[0578] Step 2:

[0579] The terminal analyzes the characteristics of the received signal and records the amplitude and phase changes of the reflected signal as numerical data. This data contains information about the movement and position of objects and people.

[0580] Step 3:

[0581] The terminal sends the analysis results described above to the server in packet format. The transmitted data includes timestamps and location information.

[0582] Step 4:

[0583] The server receives data sent from the terminal in real time. The received data is immediately stored in the database.

[0584] Step 5:

[0585] The server analyzes the received data using machine learning algorithms. Here, it compares the patterns of signal changes with a model to estimate human movement and location.

[0586] Step 6:

[0587] The server detects abnormal behavior or patterns based on the analysis results. If an anomaly is detected, it generates an alert based on pre-configured conditions.

[0588] Step 7:

[0589] Users receive alerts generated by the server on their devices. These alerts include the type of anomaly, the time of occurrence, and recommended actions.

[0590] Step 8:

[0591] Users take necessary actions based on the alert information displayed on their devices. This includes, for example, checking their home environment and taking security measures at work.

[0592] This process allows the system to operate efficiently, providing users with real-time situational awareness and immediate responses.

[0593] (Example 1)

[0594] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0595] There is a need to provide a means of accurately detecting the behavior of entities or people in the environment in real time, without requiring additional hardware, by utilizing existing wireless communication equipment, and to quickly notify users of any anomalies. Conventional systems require additional sensors or equipment to respond to changes in the environment, which presents challenges in terms of cost and installation. Furthermore, there is room for improvement in the speed and accuracy of anomaly detection and notification.

[0596] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0597] In this invention, the server includes information gathering means for detecting the behavior of entities or people in the environment by utilizing signal reflection from wireless communication equipment; information transfer means for transferring the information obtained from the information gathering means to an information processing device; and analysis means for analyzing the transferred information to determine the behavior and location of entities or people. This makes it possible to detect the movement of objects and people in the environment without additional hardware, and to accurately grasp and quickly notify of anomalies in real time.

[0598] "Wireless communication equipment" refers to devices that use radio waves to send and receive voice, data, and other signals, and is a device that exchanges signals within an environment.

[0599] "Signal reflection" refers to the phenomenon where radio waves transmitted from wireless communication devices hit an object or person and return, allowing information about the object's location and movement to be obtained.

[0600] "Information gathering means" refers to a process or device for detecting entities or human behavior in the environment using signal reflection from wireless communication devices and acquiring data related thereto.

[0601] "Information transfer means" refers to a process or device for transmitting data obtained by information collection means to other devices, particularly information processing devices.

[0602] An "information processing device" is a device for analyzing data transmitted by an information transfer means, and includes devices that perform computational processing, such as servers.

[0603] "Analysis means" refers to a method or device that performs processing to determine the behavior and position of an entity or human being based on data received in an information processing device.

[0604] An "anomaly detection means" refers to a process or device that detects events that deviate from normal activity patterns based on results obtained from analysis means, and generates warnings or notifications.

[0605] "User notification means" refers to a process or device for providing users with generated warnings or notifications and prompting them to take necessary action.

[0606] This invention is a system that uses signals emitted from existing wireless communication devices to sense objects and human movements in the environment and detect anomalies. The server, terminal, and user play three main roles.

[0607] The server provides the computing resources necessary to analyze signals from wireless communication devices. As an information processing device, the server runs a generative AI model to analyze the received data. Here, frameworks such as TensorFlow and PyTorch are utilized to train learning algorithms, thereby building a model that learns normal behavioral patterns and detects anomalies.

[0608] The terminal senses signals from wireless communication devices installed in the environment. For example, it receives reflected signals from Wi-Fi routers and Bluetooth devices and acquires them as data using information gathering means. This data is transmitted from the terminal to the server via information transfer means.

[0609] Users receive notifications from their devices and servers and take necessary actions. For example, if unusual movement is detected in an elderly person's room, the user will receive a prompt message such as, "The system detected unusual activity during the night. Do you want to check on the elderly person's safety?" After receiving this notification, the user can use their smartphone or PC to check the situation and take appropriate action.

[0610] The key features of this invention are its cost-effectiveness, as it utilizes existing devices and requires no additional hardware, enabling rapid and accurate anomaly detection and notification. This can enhance safety in the home and efficiency in the workplace.

[0611] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0612] Step 1:

[0613] The terminal receives signals transmitted from wireless communication devices and collects them as data. Inputs include radio waves from Wi-Fi routers and Bluetooth devices, which are reflected when they strike objects and people. The output is data indicating the characteristics of the reflected signals. This data includes information such as signal strength, arrival time, and phase. Specifically, one scenario involves the terminal detecting the movement of a person in a living room.

[0614] Step 2:

[0615] The terminal sends the collected data to the server for information processing. The input data is signal characteristic data measured by the terminal, which is transferred to the server via the internet connection. The output is the signal characteristic data received by the server. This allows the information processing device to perform data analysis. Specifically, the terminal sends signal data to the server several times per second.

[0616] Step 3:

[0617] The server analyzes the movements and locations of entities or people in the environment based on the received data. The input is signal characteristic data sent to the server, which is then analyzed using a generative AI model. Specifically, TensorFlow and PyTorch are used as the software. The output is information about movements and locations in the environment obtained through the analysis. For example, the server performs an analysis to determine that a person is in a bedroom at a specific time.

[0618] Step 4:

[0619] The server uses the analysis results to detect anomalies in behavioral patterns and generates alerts as needed. Input includes information on movement and location obtained from the analysis system, which is compared to normal patterns to determine anomalies. Output is an alert indicating the anomaly. This alert is sent to the user via a notification system. For example, a message such as "Unusual movement detected in the living room" might be generated.

[0620] Step 5:

[0621] The user receives alerts sent from the server on their device (smartphone or PC). The input is alert information from the server, which is displayed on the receiving device. The output is a notification to the user, providing them with information to take action. Specifically, the user can check the notification and then take appropriate action, such as checking the safety of the elderly person, as needed.

[0622] (Application Example 1)

[0623] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0624] The present invention aims to improve safety and convenience by using wireless communication devices to sense the movements of objects and people in a space in real time, enabling anomaly detection and rapid notification. In particular, it aims to support emergency response and enhance a sense of security in the living environment by quickly identifying abnormal behavior in the elderly or in specific environments.

[0625] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0626] In this invention, the server includes information acquisition means, data transmission means, and analysis means. This enables precise motion detection using a wireless communication device and rapid detection of abnormal behavior.

[0627] An "information acquisition means" is a device that uses signal reflection from existing wireless communication devices to detect the movement of objects or people.

[0628] A "data transmission means" is a device for transmitting information obtained from an information acquisition means to a data processing device.

[0629] An "analysis device" is a device that analyzes transmitted information to determine a person's movements and location.

[0630] An "anomaly detection device" is a device that detects anomalies based on analysis results and generates an alert.

[0631] A "user notification device" is a device for providing generated alerts to the user's terminal.

[0632] "Application means" refers to software installed on a user's portable information terminal that displays warnings in real time when an anomaly is detected.

[0633] The system for carrying out this invention is configured as follows.

[0634] The server receives signals from wireless communication devices and processes the data obtained by the information acquisition means. This data is transmitted to the server using the data transmission means. The server analyzes the transmitted information using the analysis means to determine human movement and location. The anomaly detection means detects anomalies based on the analysis results, and if an anomaly is detected, the server immediately generates an alert. This alert is communicated to the user terminal by the user notification means, and the user receives the warning through the application means installed on their mobile device. This application notifies the user of anomalies in real time.

[0635] Specifically, the server uses machine learning algorithms for data processing and utilizes software such as TensorFlow and scikit-learn. This makes it possible to detect abnormal behavior compared to everyday behavioral patterns. For example, if abnormal movement is detected in an elderly person at night, the application is designed to send a notification to their smartphone stating, "Unusual movement has been detected. Please check your safety."

[0636] An example of a prompt message would be: "Develop a machine learning model for detecting abnormal behavior using wireless signal data within a home. In particular, we need a model that can distinguish between normal and abnormal behavior in the living room and bedroom."

[0637] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0638] Step 1: The server receives signals transmitted from the wireless communication device. The input is wireless signal strength data, and the output is the data collection result from the information acquisition means. In this step, the reception of the signal and the collection of data based on it are specifically carried out.

[0639] Step 2: The server transmits numerical data obtained from the information acquisition means to the server using the data transmission means. The input is the data collection result, and the output is data in a format that can be processed within the server. This step involves data format conversion and network transmission.

[0640] Step 3: The server analyzes the data using analytical tools to determine human movement and location. The input is data in a processable format, and the output is the identification result of movement and location. In this step, machine learning algorithms are used to analyze movement patterns using TensorFlow or scikit-learn.

[0641] Step 4: The server uses anomaly detection means to detect anomalies based on the analysis results and generates an alert. The input is the identification result of movement and location, and the output is alert information indicating an anomaly. In this step, a learning model is used to detect deviations from normal behavior patterns.

[0642] Step 5: The server sends the generated alert to the user terminal via a user notification mechanism. The input is the alert information, and the output is the notification information sent to the user. This step utilizes a network protocol for sending notifications in real time.

[0643] Step 6: The user receives a notification through an application on their mobile device. The input is notification information, and the output is a visually displayed warning message. In this step, the application visually and intuitively presents the notification content to the user.

[0644] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0645] This invention combines a safety management system that utilizes signals from wireless communication devices with a user emotion recognition function, aiming to make home and office environments safer and more comfortable.

[0646] The terminal receives signals emitted from existing wireless communication devices and senses the movement of objects and people. This allows it to collect data on the movement and location of the target. This data is transmitted to a server in real time and analyzed appropriately. Machine learning models are used in this analysis to detect human movement and anomalies.

[0647] Furthermore, the server is equipped with an emotion engine that can evaluate the user's emotional state by analyzing their voice and facial expression data. The emotion engine acquires voice and video data through interfaces such as cameras and microphones and determines whether the user's emotions deviate from the normal range.

[0648] For example, the system can sense whether a resident is relaxed or stressed. Based on this emotional state, the server adjusts notifications and actions to provide the user with more relevant information. For instance, if a user is stressed, the system might provide information to encourage relaxation.

[0649] Users can receive these notifications via their devices and take the most appropriate action based on their environment. This system enables flexible and effective management of living and working environments based on detailed information, including the user's emotional state. Furthermore, if an anomaly is detected based on data analysis, the system quickly alerts the user and prompts them to take the necessary action, thereby ensuring user safety.

[0650] This invention offers high security and convenience while being an inexpensive device, making it suitable for a wide range of applications in homes and offices. Furthermore, its emotion-based recognition capabilities could contribute to health management and mental support. This would allow users to enjoy an environment where they can focus on their lives and work with greater peace of mind.

[0651] The following describes the processing flow.

[0652] Step 1:

[0653] The terminal receives radio waves emitted from wireless communication devices and detects signals reflected by surrounding objects and people. This allows it to collect basic data for recording environmental changes in real time.

[0654] Step 2:

[0655] The terminal analyzes the received signal data and converts the movement and position of objects and people into numerical data. This data includes information such as speed and direction of movement, enabling detailed motion analysis.

[0656] Step 3:

[0657] The terminal sends the analyzed data to the server. Security protocols are used for communication, ensuring data confidentiality during transmission.

[0658] Step 4:

[0659] The server receives data sent from the terminal and stores it in the database. The received data is immediately ready for processing.

[0660] Step 5:

[0661] The server analyzes the transmitted data using machine learning algorithms to determine the person's movements and location in detail. It also detects movements that deviate from normal patterns as anomalies.

[0662] Step 6:

[0663] The server collects and analyzes the user's voice and video through an emotion engine. This allows it to determine the user's emotional state, for example, whether they are feeling stressed.

[0664] Step 7:

[0665] The server integrates and analyzes user behavior and emotional data, and generates notifications as needed. If a user is experiencing stress, it might prepare a notification to encourage relaxation.

[0666] Step 8:

[0667] Users receive notifications sent from the server on their devices. These notifications include appropriate actions for the user and provide audio and visual guidance.

[0668] Step 9:

[0669] Users can take necessary actions based on the notification content. At home, this might involve adjusting lighting or playing background music; in the office, it might provide clues for deciding how to take breaks.

[0670] This entire process allows the system to monitor both user behavior and emotions in real time, providing a safer and more comfortable environment.

[0671] (Example 2)

[0672] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0673] In modern living environments, there is a need for systems that utilize wireless communication technology to provide safe and comfortable living and working environments. However, existing systems have limitations in detecting the movement and anomalies of dynamic objects, and may not be able to provide appropriate responses that take into account the user's emotional state. Furthermore, there is a lack of effective means to comprehensively manage user safety and mental comfort.

[0674] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0675] In this invention, the server includes information gathering means, information transmission means, analysis means, anomaly detection means, emotion recognition means, and user notification means. This makes it possible to analyze the behavior and emotional state of dynamic targets using wireless communication devices in real time, evaluate anomalies and the user's emotional state, and provide appropriate notifications.

[0676] "Information gathering means" refers to devices that receive signals from wireless communication equipment and sense dynamic objects or human movements.

[0677] "Information transmission means" refers to a function for transmitting data acquired by information collection means to a computing device (server).

[0678] "Analysis means" refers to processes and algorithms for determining the movement and position of a dynamic object based on transmitted information.

[0679] An "anomaly detection method" is a technology that detects anomalies that deviate from normal operating patterns based on analysis results and generates warnings.

[0680] "Emotion recognition means" refers to technology that analyzes a user's voice and video data to evaluate their emotional state.

[0681] "User notification means" refers to functions or interfaces for providing users with generated warnings and sentiment evaluation results.

[0682] This invention is a safety management system that combines wireless communication technology and emotion recognition technology to make home and office environments safer and more comfortable. The terminal senses dynamic objects and human movements by receiving signals emitted from existing wireless communication devices. Specifically, it uses wireless technologies such as Bluetooth and Wi-Fi to acquire data on the position of objects and the movement of people. This data is transmitted to a server in real time, and the security of the information is ensured using a secure protocol.

[0683] The server analyzes the received data using machine learning algorithms to determine the behavior, location, and anomalies of dynamic objects. The machine learning models used are trained on platforms such as TensorFlow and PyTorch. Furthermore, the server acquires user audio and video data via cameras and microphones and evaluates their emotional state through an emotion engine. This evaluation makes it possible to provide appropriate emotion-based feedback while ensuring user safety.

[0684] As a concrete example, in an office environment, if the system detects that a user is experiencing stress, the server will notify the user via their terminal with music to promote relaxation. Furthermore, if abnormal activity is detected, an immediate warning can be issued, prompting a quick response.

[0685] Examples of prompts for a generative AI model:

[0686] I want to develop an emotion recognition system for use in the office. It would use wireless communication devices to sense movement and analyze the user's emotional state from their voice and facial expressions. This system would send notifications to users who are feeling stressed, encouraging them to relax.

[0687] This allows users to live and work in a safe and comfortable environment, enjoying a high level of security and convenience.

[0688] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0689] Step 1:

[0690] The terminal receives signals emitted from wireless communication devices and senses the movement of moving objects and human actions. Specifically, it uses Bluetooth and Wi-Fi signals to acquire location information of people and objects. This input information is managed within the system as motion data. Sensed motion data is generated as output.

[0691] Step 2:

[0692] The terminal sends the collected operational data to the server. The data is securely transmitted using protocols such as HTTP and MQTT, and received by the server. The input to this transmission is operational data, and the output is the server's reception of the data.

[0693] Step 3:

[0694] The server analyzes the received operational data using machine learning algorithms. Specifically, models trained using TensorFlow or PyTorch analyze the operational data and detect anomalies. The input is operational data, and the output is the anomaly detection result.

[0695] Step 4:

[0696] The server acquires user audio and video data using a camera and microphone, and analyzes it with an emotion recognition engine. The input is audio and video data, and the emotional state is output. Specifically, it evaluates the user's stress level and relaxation state.

[0697] Step 5:

[0698] The server generates notifications and feedback based on anomaly detection results and emotional states. The inputs are anomaly detection results and emotional states, and the output is a notification to be provided to the user. Specifically, it sends a message encouraging relaxation to users who are feeling stressed.

[0699] Step 6:

[0700] Users receive notifications from the server via their devices. This allows them to take actions such as playing music to reduce stress or adjusting their environment. The input is the notifications from the server, and the output is the user's actions and changes in their environment.

[0701] (Application Example 2)

[0702] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0703] While motion detection technologies utilizing existing wireless communication devices exist, they have not yet achieved simultaneous improvements in environmental safety and occupant comfort. In particular, security management that takes into account the user's emotional state is insufficient, potentially leading to delays in responding to emotional changes, stress, and dangerous situations. A new system that balances safety management and comfort is needed.

[0704] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0705] In this invention, the server includes: information gathering means for sensing the movement of objects and people by utilizing signal reflection from existing wireless communication devices; information transmission means for transmitting information obtained from the information gathering means to the server; analysis means for analyzing the transmitted information and determining the movement and location of people; anomaly detection means for detecting anomalies based on the analysis results and generating notifications; user notification means for providing the generated notifications to the user; and emotion recognition means for analyzing the user's voice and facial information to determine their emotional state and generating notifications to promote relaxation. This makes it possible to adjust the environment to be comfortable according to the user's emotional state while maintaining safety.

[0706] "Information gathering means" refers to a function that uses signal reflection from existing wireless communication devices to acquire information about the movements of objects or people.

[0707] "Information transmission means" refers to a function for transmitting information acquired by information collection means to a server.

[0708] "Analysis means" refers to a function that analyzes information transmitted to the server to determine a person's movements and location.

[0709] An "anomaly detection means" is a function that detects anomalies based on the results of the analysis means and generates notifications as necessary.

[0710] "User notification means" refers to a function that provides generated notifications to users and prompts them to take appropriate action.

[0711] The "emotion recognition means" is a function that analyzes the user's voice and facial expression information to determine their emotional state and, if necessary, generates notifications to promote relaxation.

[0712] This system consists primarily of a wireless communication device, a server, and a smartphone or similar terminal. The wireless communication device constantly monitors the surrounding environment in a home or office setting, collecting information on the movements of objects and people using signal reflection. This information is acquired through the information collection means and transmitted to the server via the terminal.

[0713] The server uses software such as Python and TensorFlow to analyze the received information. From the analyzed data, it determines a person's movements, location, and emotional state. This analysis activates anomaly detection mechanisms, making it possible to detect deviations from normal behavioral patterns and high-stress states. If an anomaly is detected, the server immediately generates a notification, which is presented to the user through the notification mechanism.

[0714] The emotion recognition system uses the smartphone's camera and microphone to collect emotional data from the user's voice and facial expressions. This allows it to determine the user's emotional state and provide information to promote relaxation. For example, if the user is feeling stressed, a selected relaxation method (e.g., playing relaxation music) is suggested.

[0715] For example, if the system determines that a user working in an office is experiencing high stress, a notification will be sent to their smartphone saying, "Play some relaxation music to change your mood." An example of a prompt message for the generating AI model would be, "The emotion engine has detected a high-stress state. Please suggest that the user play some relaxation music."

[0716] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0717] Step 1:

[0718] Wireless communication devices collect motion information of objects and people from the surrounding environment using signal reflection. Signal data is input and output as motion information. This motion information is received by information collection means within the terminal.

[0719] Step 2:

[0720] The terminal transmits operational information obtained from the information gathering device to the server. Operational information is input and output directly to the server as transmitted information. This process utilizes wireless communication technologies such as Bluetooth and Wi-Fi.

[0721] Step 3:

[0722] The server analyzes the received motion information. Here, the input motion information is analyzed using Python or TensorFlow to determine the person's movements and position. Through this analysis, abnormal patterns are generated as output.

[0723] Step 4:

[0724] The server determines an anomaly based on the detection of anomaly patterns and generates a notification as necessary. An anomaly pattern is input, and a notification is output to attract the user's attention. In this step, the anomaly detection means uses a learning algorithm to determine deviations from normal behavior.

[0725] Step 5:

[0726] The server utilizes the camera and microphone of the smartphone or device to collect the user's voice and facial expression data. This data is input, and information is gathered to determine the user's emotional state.

[0727] Step 6:

[0728] The server's emotion recognition system analyzes collected voice and facial expression data to evaluate the user's emotional state. The user's current emotional state is output based on the input emotion data. A generative AI model is used in this process.

[0729] Step 7:

[0730] The server generates and provides notifications to the user to promote relaxation, based on the user's emotional state. The user's emotional state is input, and relaxation suggestions are output. Specifically, it generates a message such as "Play relaxation music" and sends a notification to the terminal. This notification is provided through a user notification system.

[0731] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0732] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0733] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0734] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0735] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0736] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0737] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0738] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0739] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0740] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0741] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0742] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0743] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0744] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0745] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0746] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0747] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0748] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0749] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0750] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0751] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0752] The following is further disclosed regarding the embodiments described above.

[0753] (Claim 1)

[0754] A data collection method for sensing the movement of objects or people by utilizing signal reflection from existing wireless communication devices,

[0755] A data transmission means that transmits data obtained from a data collection means to a server,

[0756] An analysis means that analyzes the transmitted data and determines the movements and location of a person,

[0757] An anomaly detection means that detects anomalies based on analysis results and generates notifications,

[0758] A user notification means that provides the generated notification to the user,

[0759] A system that includes this.

[0760] (Claim 2)

[0761] The system according to claim 1, wherein the anomaly detection means uses a learning algorithm for detecting deviations from normal behavioral patterns.

[0762] (Claim 3)

[0763] The system according to claim 1, wherein the analysis means analyzes data using a machine learning model.

[0764] "Example 1"

[0765] (Claim 1)

[0766] Information gathering means for detecting the behavior of entities or humans in the environment by utilizing signal reflection from wireless communication devices,

[0767] Information transfer means for transferring information obtained from information gathering means to information processing device,

[0768] An analytical means for analyzing the transmitted information to determine the behavior and position of an entity or human,

[0769] An anomaly detection means that detects anomalies based on analysis results and generates a notification,

[0770] A user notification means that supplies the generated notification to the user,

[0771] A system that includes this.

[0772] (Claim 2)

[0773] The system according to claim 1, wherein the anomaly detection means utilizes a learning algorithm for detecting deviations from standard behavioral patterns.

[0774] (Claim 3)

[0775] The system according to claim 1, wherein the analysis means analyzes information using an information processing model.

[0776] "Application Example 1"

[0777] (Claim 1)

[0778] Information acquisition means for sensing the movement of objects or people by utilizing signal reflection from existing wireless communication devices,

[0779] A data transmission means that transmits information obtained from an information acquisition means to a data processing device,

[0780] An analysis means that analyzes the transmitted information and determines the movements and location of the person,

[0781] An anomaly detection means that detects anomalies and generates alerts based on analysis results,

[0782] A user notification means that provides the generated alert to the user's terminal,

[0783] An application means that is installed on a portable information terminal carried by the user and displays a warning in real time when an anomaly is detected,

[0784] A system that includes this.

[0785] (Claim 2)

[0786] The system according to claim 1, wherein the anomaly detection means uses a machine learning algorithm to detect deviations from normal behavioral patterns.

[0787] (Claim 3)

[0788] The system according to claim 1, wherein the analysis means analyzes information using a learning model and detects anomalies.

[0789] "Example 2 of combining an emotion engine"

[0790] (Claim 1)

[0791] Information gathering means for sensing dynamic objects or human movements using signal reflections from existing wireless communication devices,

[0792] Information transmission means for transmitting information obtained from information collection means to a computing device,

[0793] An analysis means that analyzes the transmitted information and determines the movement and position of the dynamic object,

[0794] An anomaly detection means that detects anomalies and generates warnings based on the analysis results,

[0795] An emotion recognition means that analyzes information about actions and emotional states and evaluates the user's emotional state,

[0796] A user notification mechanism that provides users with generated warnings and evaluation results,

[0797] A system that includes this.

[0798] (Claim 2)

[0799] The system according to claim 1, wherein the anomaly detection means uses a learning computation to detect deviations from general behavioral patterns.

[0800] (Claim 3)

[0801] The system according to claim 1, wherein the analysis means analyzes information using a machine learning algorithm, and the emotion recognition means evaluates the emotional state through audio and video data.

[0802] "Application example 2 of combining emotional engines"

[0803] (Claim 1)

[0804] Information gathering means for sensing the movement of objects and people by utilizing signal reflection from existing wireless communication devices,

[0805] Information transmission means that transmits information obtained from information collection means to a server,

[0806] An analysis means that analyzes the transmitted information and determines the movements and location of the person,

[0807] An anomaly detection means that detects anomalies based on analysis results and generates notifications,

[0808] A user notification means that provides the generated notification to the user,

[0809] An emotion recognition means that analyzes the user's voice and facial expression information to determine their emotional state and generates a notification to promote relaxation,

[0810] A system that includes this.

[0811] (Claim 2)

[0812] The system according to claim 1, wherein the anomaly detection means uses a learning algorithm for detecting deviations from normal behavioral patterns.

[0813] (Claim 3)

[0814] The system according to claim 1, wherein the analysis means analyzes information using a machine learning model. [Explanation of Symbols]

[0815] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A data collection method for sensing the movement of objects or people by utilizing signal reflection from existing wireless communication devices, A data transmission means that transmits data obtained from a data collection means to a server, An analysis means that analyzes the transmitted data and determines the movements and location of a person, An anomaly detection means that detects anomalies based on analysis results and generates notifications, A user notification means that provides the generated notification to the user, A system that includes this.

2. The system according to claim 1, wherein the anomaly detection means uses a learning algorithm for detecting deviations from normal behavioral patterns.

3. The system according to claim 1, wherein the analysis means analyzes data using a machine learning model.

Images