system

Abstract

We provide the system. [Solution] A processing device equipped with an artificial intelligence model for detecting suspicious behavior or appearance, comprising: an analysis means for inputting video information and determining a suspicious person from said video information; An audio warning means that generates a command to emit a warning sound based on the results of the analysis means's determination of a suspicious person, An information notification means that notifies a notification device of the results of the analysis means in determining the presence of a suspicious person, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, criminal acts, especially theft and robbery, have been increasing in private homes, and residents are in a situation where their safety is threatened. In current crime prevention measures, there is a lack of a system that can detect suspicious persons early and enable immediate response, so it is difficult to prevent crimes before they occur, and it is required to shorten the time until residents face danger. Therefore, there is an urgent need to develop a system that can identify suspicious persons in real time and give early warnings, which has not been achievable with conventional security cameras.

Means for Solving the Problems

[0005] The present invention solves the above problems by providing a processing device equipped with an artificial intelligence model for detecting suspicious behavior or appearance. To determine the presence of a suspicious person, first, video information is input, and a suspicious person is identified from the video information by an analysis means. This analysis means uses an AI model optimized based on multiple training data. Furthermore, if a suspicious person is identified by the analysis means, an audio warning means generates a command to emit a warning sound, providing an immediate warning to the suspicious person. This warning can enhance the crime deterrent effect. In addition, the determination result is transmitted to a notification device using an information notification means, and the user is promptly notified, enabling a rapid response. This aims to prevent crime damage through security cameras and ensure the safety of residents.

[0006] "Suspicious behavior or appearance" refers to actions, postures, clothing, or possessions that deviate from normal behavioral patterns or typical appearances, and that have characteristics that suggest the possibility of a crime.

[0007] An "artificial intelligence model" is a program that uses machine learning and deep learning techniques to perform pattern recognition and prediction from data, and is designed to mimic human intelligence in specific tasks.

[0008] A "processing device" is an electronic device that analyzes received data and generates or transmits necessary information, and includes a central processing unit and memory.

[0009] "Analytical means" refers to techniques or methods for analyzing data and deriving specific results, particularly those used to identify suspicious behavior or appearance.

[0010] "Audio warning means" refers to a device or system that uses sound to issue a warning for the purpose of deterring the actions of a suspicious person.

[0011] "Information notification means" refers to a technology or system for transmitting information generated by a processing device to an external device or user.

[0012] A "notification device" is a device or application used to display or notify information to a user, and includes mobile terminals and computers. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0017] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] The security system according to the present invention enables the detection of suspicious behavior or appearance in real time and the immediate implementation of countermeasures. This system consists of three main elements: a terminal, a server, and a user.

[0035] The terminal used is a camera equipped with video capture capabilities. This camera continuously acquires video data and transmits it to the server. The video is high resolution and equipped with an infrared sensor to operate even at night.

[0036] The server receives video data transmitted from the terminal. Internally, it incorporates an artificial intelligence model for analyzing the video, which allows for detailed analysis of a person's behavior, posture, clothing, and belongings. If the analysis detects suspicious behavior or appearance, the server sends a command to an audio warning system. This audio warning system sends a signal to the relevant camera, which then emits a warning sound at the location.

[0037] Furthermore, the server communicates the analysis results and warnings to the user through information notification systems. This process utilizes smartphone applications and email notification systems, allowing users to respond immediately. The notification includes detailed information about the detected suspicious person, video snapshots, the time of detection, and the location.

[0038] A concrete example is when a suspicious person is loitering in a yard at night. In this case, the device captures the person's image and sends the data to a server. The server uses an AI model to analyze the person's behavior, and if any suspicious movements or locations are detected, it immediately issues a warning. At the same time, the user is notified of the information, allowing them to quickly take the next step, such as contacting the police. As a result, criminal activity can be prevented, and the safety of residents can be ensured.

[0039] Thus, this system enables real-time detection and immediate response to suspicious individuals, significantly improving the security of private homes.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The device captures video in real time via the security camera and sends the video data to the server at regular frame intervals. The camera operates in high-resolution mode and is configured to acquire clear images day and night.

[0043] Step 2:

[0044] The server receives video data sent from the terminal in real time. As a preprocessing step for the video, it performs noise reduction and resolution optimization to generate a dataset suitable for analysis.

[0045] Step 3:

[0046] The server inputs pre-processed video data into an artificial intelligence model. This AI model is built on a deep learning algorithm and automatically analyzes people's movements and appearances. The model learns suspicious behaviors and appearances based on past data, and performs real-time pattern recognition accordingly.

[0047] Step 4:

[0048] The server identifies suspicious individuals based on the analysis results of the AI ​​model. If the analysis detects suspicious behavior, the server generates the necessary commands to take immediate action. These commands include instructions to generate an alarm sound.

[0049] Step 5:

[0050] Upon receiving instructions from the server, the terminal emits a warning sound through a speaker at the site. The warning sound is generated based on pre-configured parameters, and its volume and duration are adjusted accordingly. This warning sound prompts the suspicious person to leave the area.

[0051] Step 6:

[0052] Simultaneously, the server notifies the user of the detection results. Details of the suspicious person's movements, as well as information about the time and location of the alert, are sent to the user's smartphone or other mobile device. The user can then receive the notification and quickly consider how to respond.

[0053] Step 7:

[0054] Based on the notification, users can take countermeasures such as contacting the police or further utilizing the system. If necessary, user feedback is sent to the server and used as data to improve the accuracy of the AI ​​model.

[0055] (Example 1)

[0056] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0057] In recent years, the importance of crime prevention has been increasing in society. However, conventional surveillance systems have difficulty detecting suspicious behavior in real time and taking appropriate action immediately. Therefore, there is a need for a crime prevention system that can efficiently and quickly identify suspicious individuals and issue warnings.

[0058] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0059] In this invention, the server includes an analysis device equipped with an artificial intelligence model for receiving video information acquired from an image acquisition device and analyzing a person's behavior, posture, clothing, and belongings based on the video information; an audio warning device that recognizes suspicious behavior from the analysis results of a person's behavior or appearance by the analysis device and generates a command to send a warning signal to an audio output device to emit a warning sound; and an information notification device for notifying the user of the analysis results of suspicious behavior or appearance by the analysis device via a notification device. This enables real-time detection of suspicious persons, rapid warning issuance, and information notification.

[0060] An "image acquisition device" is a device for acquiring video information of the surroundings and has the function of monitoring a certain range.

[0061] An "analysis device" is a device that analyzes the movement, appearance, and shape of people and objects based on acquired video information, and has the ability to recognize a variety of patterns.

[0062] An "artificial intelligence model" is a program used to analyze video information and identify specific behaviors or characteristics, and it applies machine learning and deep learning technologies.

[0063] An "audio warning device" is a device that emits a warning sound when it receives a specific signal, and has the function of alerting suspicious individuals.

[0064] A "notification device" is a device used to communicate analysis results to the user and has the function of transmitting information via a communication network.

[0065] A "user terminal" is the device on the user's side that ultimately receives the information, and can take various forms, such as a smartphone or a computer.

[0066] This invention provides a security system that can detect suspicious behavior or appearance in real time and respond immediately. This system mainly consists of three elements: a terminal, a server, and a user.

[0067] The terminal used is a camera, which is a video acquisition device. This camera can capture clear images day and night and is equipped with high resolution and an infrared sensor. The terminal continuously collects video data and sends it to the server. This transmission is performed in real time over the network.

[0068] The server receives video information transmitted from the terminal. An artificial intelligence model is used to analyze the video. Specifically, a generative AI model is used to analyze the actions, postures, clothing, and belongings of people in the video data. This AI model uses advanced image recognition technology and is excellent at recognizing patterns and detecting abnormal movements. If suspicious behavior is detected as a result of the analysis, the server sends a warning signal to an audio warning device. This signal immediately emits a warning sound at the site, alerting the suspicious person to be vigilant.

[0069] Furthermore, the server notifies the user of the analysis results and the fact of the warning. The notification is sent to the user's terminal via a notification device. User terminals such as smartphones and computers can receive detailed information of the analysis results in real time. The notification includes the characteristics of the suspicious person, video snapshots, and information on the time and location of detection.

[0070] As a concrete example, imagine a situation where a suspicious person is active in a garden at night. In this case, the device captures the person's image and sends the data to a server. The server uses an AI model to analyze the person's actions, and if suspicious activity is detected, it immediately issues a warning. Simultaneously, the user receives a prompt message such as, "A suspicious person has been detected in your garden. Please check their actions." This allows the user to understand the situation and take immediate action.

[0071] This system significantly improves home security, enabling real-time detection of intruders and rapid response.

[0072] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0073] Step 1:

[0074] The device uses a camera, an image acquisition device, to acquire surrounding video data in real time. In this step, the camera uses high resolution and an infrared sensor to capture clear video data 24 hours a day. The input is the surrounding visual information, and the output is the captured video data. This video data is transmitted directly to the next step.

[0075] Step 2:

[0076] The server receives video data transmitted from the terminal. Using this data as input, an AI model within the server performs video analysis. The data processing involves recognizing the movement, posture, clothing, and possessions of a person in each frame of the video and analyzing their characteristics. The output is the analysis results for each frame. Specifically, the AI ​​model tracks the person's movement and detects abnormal patterns.

[0077] Step 3:

[0078] Based on the analysis results obtained in step 2, the server evaluates the behavior and appearance to determine whether or not suspicious behavior is present. The input is the analyzed behavioral data, and the data calculation involves comparing whether the person's movements and behavioral patterns fall within the normal range. The output is a determination of whether or not suspicious behavior was detected.

[0079] Step 4:

[0080] If the server detects suspicious activity, it sends a command to the audio warning device. This command instructs the terminal's warning system to emit a warning sound. The input is the result of the suspicious activity detection, and the output is the actual sound emitted. Specifically, the audio warning device emits a loud warning sound to draw attention to the area.

[0081] Step 5:

[0082] The server transmits the analysis results and warning information to the user via an information notification device. The input is the judgment result and the generated warning information, and the output is a notification sent to the user's terminal. Specifically, this includes the display of information such as video snapshots, detection time, and location on a smartphone or computer.

[0083] Step 6:

[0084] After the user receives a notification, they take the necessary action based on its content. The input is notification information from the server, and the output is a specific action taken by the user (e.g., contacting the police). Specifically, the user checks the details through the application and considers a course of action.

[0085] (Application Example 1)

[0086] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0087] In modern society, criminal activity and illegal activities by suspicious individuals have become a serious problem. However, conventional security systems have limited ability to detect suspicious movements and individuals in real time, making rapid response difficult. Identifying suspicious individuals and responding immediately, especially in homes and businesses, is a crucial issue directly linked to improving safety.

[0088] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0089] In this invention, the server includes an analysis means for receiving video data and identifying suspicious individuals, an audio alarm means for issuing commands to generate audio alarms, an information notification means for transmitting information to an information device, and a data communication means for providing video snapshots to user devices. This enables real-time detection of suspicious behavior or appearances, rapid alarm activation and immediate notification to users, and provision of video snapshots, thereby facilitating a quick response and improving security.

[0090] "Suspicious behavior or appearance" refers to any state that deviates from normal behavior or appearance and is judged to potentially threaten safety.

[0091] An "intelligent model" is an algorithm that uses technologies such as machine learning and deep learning to learn patterns from data and identify suspicious individuals.

[0092] A "processing unit" is a computing device used to process data and perform specified tasks.

[0093] "Video data" refers to the digital format of visual information captured by a camera or other recording device.

[0094] "Analysis methods" refer to mechanisms that process data received to identify suspicious individuals.

[0095] An "audio alarm system" is a device or system that generates and emits an audible warning when a suspicious person is detected.

[0096] An "information device" is a machine equipped with communication and display functions to enable notifications and data transfer.

[0097] "Data communication means" refers to communication technology used to transmit video and analysis results to user devices.

[0098] The present invention is specifically implemented as a security system that enables the detection of suspicious individuals and immediate response. The operation of this system, the hardware and software used, and specific embodiments are described below.

[0099] The server is equipped with an analysis system for image processing and receives video data transmitted from surveillance cameras in real time. This data is in a high-resolution format that allows for night vision. The server uses software libraries such as Python and TENSORFLOW® as intelligent models to perform data analysis to detect suspicious behavior or appearances.

[0100] Image analysis is performed, and if suspicious behavior or appearance is detected, the server will issue an alarm using audio alarm means. This uses hardware devices such as digital speakers and alarm devices. Furthermore, an information notification means will send a notification containing information about the suspicious person to the user's individual device, such as a smartphone or tablet. This notification will include real-time video snapshots and detailed information about the detected suspicious person.

[0101] When users receive these notifications, they can view video snapshots and past data within the application. The application also includes additional features for directly contacting the police or security companies if necessary.

[0102] A concrete example is when a user detects suspicious activity in their yard while doing their daily shopping. The user checks the notification on their smartphone, views the security camera footage, and confirms the anomaly. This allows them to quickly contact their neighbors and take steps to ensure their safety.

[0103] An example of a prompt might be: "Use the video feed from the camera installed in the garden to monitor for suspicious activity or individuals in real time and immediately notify the user's smartphone. If necessary, allow the user to access police contact information." This type of prompt appropriately provides the necessary instructions to the generating AI model.

[0104] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0105] Step 1:

[0106] The terminal continuously acquires video data from the installed camera. The input is real-time high-resolution video, which is captured and transmitted to the server. In operation, the terminal's video capture function works, and data is collected even at night using an infrared sensor.

[0107] Step 2:

[0108] The server receives video data transmitted from the terminal. The input is real-time video data, which is processed by an intelligent model (using Python and TensorFlow). Here, data analysis is performed, and movement and appearance within the video frames are evaluated to detect suspicious behavior or appearance.

[0109] Step 3:

[0110] The server activates an audio alarm system if suspicious activity is detected. The input is the result of the intelligent model's determination of suspicious activity; based on this, it creates an alarm and outputs a command. Specifically, it uses a digital speaker to emit an alarm sound at the location.

[0111] Step 4:

[0112] The server uses an information notification system to send information about suspicious individuals and video snapshots to the user's device. The input is the result of the suspicious activity detection and the snapshot data, and the output is a notification containing this information. This operation involves rapidly distributing the data over a communication network.

[0113] Step 5:

[0114] The user checks the received notification on their smartphone or tablet and views the video snapshot. The input is the notification data from the server, and the output is the displayed snapshot and details of the suspicious person. The user reviews the video to determine whether they need to contact the police or security services.

[0115] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0116] The security system according to the present invention not only detects suspicious individuals but also analyzes the user's emotions, enabling a more flexible and human-like response. The system consists of a terminal, a server, and a user, and its functionality is enhanced by incorporating an emotion engine.

[0117] In addition to conventional security camera functions, the device is equipped with audio input devices such as a microphone. This allows the device to collect ambient audio data and provide information for use in analyzing the user's emotions. Audio and video data are transmitted to the server in real time.

[0118] The server receives video and audio data transmitted from the terminal. Based on this, an AI model detects suspicious individuals, while an emotion engine analyzes the user's emotional state. The emotion engine analyzes the tone of voice, speaking style, and facial expressions that can be read from the video, and evaluates the user's mental state based on the results. The server combines the emotion analysis results with the AI ​​model's results to optimize the content of the audio warning for detected suspicious individuals and the subsequent notification method.

[0119] A concrete example would be a situation where a suspicious person is loitering around the front door at night. The device captures video and audio and sends the data to a server. The server detects the suspicious person from the video and analyzes the user's emotions from the audio they speak to the system. For example, if the server detects anxiety from the user's voice, it can use that feedback to adjust the tone of the audio warning to be more intimidating, or add a message to the user's notification suggesting they contact the police.

[0120] On the other hand, after receiving a notification, users can choose a course of action based on the emotion-based advice provided by the system. Providing further information as needed helps improve the system's response accuracy.

[0121] Thus, by introducing the emotion engine in this invention, it becomes possible to respond flexibly to the user's emotions, thereby achieving a higher level of safety assurance.

[0122] The following describes the processing flow.

[0123] Step 1:

[0124] The device captures video using a security camera while simultaneously collecting ambient audio data using a microphone. This data is transmitted to a server in real time. The camera operates 24 hours a day in high definition, and the microphone is set to record natural-sounding audio.

[0125] Step 2:

[0126] The server receives video and audio data transmitted from the terminal. The received data is preprocessed, including noise reduction and resolution optimization, to prepare it for analysis by AI models and emotion engines.

[0127] Step 3:

[0128] The server uses an AI model to analyze video data and detect suspicious individuals. Specifically, it analyzes a person's movements, clothing, and belongings, and compares them with past data to determine if there is anything unusual. Simultaneously, it inputs audio data into an emotion engine to analyze the user's emotional state.

[0129] Step 4:

[0130] The server integrates the analysis results and, if a suspicious person is detected, adjusts the output of the voice warning system, taking into account the evaluation of the emotion engine. For example, if the user's emotions indicate anxiety, the warning sound is set to a more alarming tone.

[0131] Step 5:

[0132] The terminal receives instructions from the server and emits an appropriate warning sound from a speaker installed at the site. This warning sound is intended to alert the suspicious person to leave the area, and the volume and tone are dynamically adjusted according to the situation.

[0133] Step 6:

[0134] The server generates and sends notifications to the user based on the analysis results of the suspicious person and the user's sentiment evaluation. These notifications include information about the detected suspicious person, recommended actions based on the user's sentiment, and suggestions to report to the police.

[0135] Step 7:

[0136] Based on the notifications received from the server, users can consider the suggested countermeasures and take action, such as contacting the police, if necessary. They can also send feedback on the notifications to the server, contributing to system improvements.

[0137] (Example 2)

[0138] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0139] Traditional security systems focus on identifying suspicious individuals, but they lack flexibility in considering the user's feelings, resulting in warnings and notifications not being provided in a way that is optimal for the user's situation. As a result, the system may give users an insufficient sense of security.

[0140] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0141] In this invention, the server includes an analysis means for inputting video material and determining a suspicious person, an audio output means for generating a command to emit an audio signal based on the suspicious person determination result, and an emotion analysis means for inputting audio material and evaluating the user's emotional state. This makes it possible to combine the suspicious person detection result and the user's emotion analysis result to provide the user with the most appropriate response, thereby improving safety and security.

[0142] "Suspicious behavior" refers to actions or movements that deviate from normal behavioral patterns and may pose a potential danger.

[0143] An "artificial intelligence model" is a model with a program structure that learns from large amounts of data and can identify specific patterns or anomalies.

[0144] A "computer" is an electronic device used for inputting, processing, and outputting data.

[0145] "Visual materials" refers to visual information data collected by cameras or other video acquisition devices.

[0146] "Analysis means" refers to the process or device that analyzes input data and derives meaningful information.

[0147] "Audio signals" refer to acoustic information used for warnings and commands, generated based on audio data.

[0148] "Audio output means" refers to a device or mechanism for generating an audio signal based on the analysis results and transmitting it externally.

[0149] A "notification mechanism" is a device or system that conveys specific information to users through sound or visual signals.

[0150] "Information transmission means" refers to means for transmitting judgment results and related information to other devices or users.

[0151] "Emotional analysis means" refers to a process or mechanism that analyzes voice tone, manner of speaking, or facial expressions to evaluate the user's emotional state.

[0152] A "control means" is a process or device that combines results obtained from different data to perform optimal processing or output.

[0153] The security system of this invention consists of terminals, servers, and users, and provides a high level of security through the coordinated functioning of each element.

[0154] The terminal functions as a security camera and collects video information. Furthermore, the terminal is equipped with a microphone and has the capability to collect audio information. The terminal transmits the collected video and audio information to a server in real time. This communication is typically conducted via a wireless or wired network, and the collected data is encrypted to ensure security.

[0155] The server plays a central role in receiving and analyzing data transmitted from terminals. The server utilizes generative AI models to analyze video information and identify individuals exhibiting suspicious behavior. It also uses emotion analysis to evaluate the user's emotional state from audio information. This emotion analysis is based on factors such as voice tone, speed, and pauses to determine the user's psychological state. The analysis results are integrated by the server and used to generate warning signals against suspicious individuals and optimize notification methods.

[0156] As a concrete example, consider a scenario where a suspicious person appears at the front door at night. The device records this situation with video and audio and sends it to the server. The server identifies the suspicious person using a generative AI model and simultaneously analyzes the user's emotions from the audio data. For example, if the analysis indicates that the user is feeling anxious, the server may intensify the warning sound or generate a notification message suggesting that the police be notified.

[0157] The user receives a notification from the server and selects an appropriate response based on the information and suggestions provided. An example of a specific prompt message is: "Analyze the suspicious person at the front door and the user's emotions at that time, and suggest an appropriate voice warning and notification method."

[0158] In this way, by having each element work in conjunction, it is possible to realize a comprehensive security system that not only detects suspicious individuals but also takes into account the user's emotions.

[0159] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0160] Step 1:

[0161] The device collects surrounding video and audio information. Inputs include visual data from the camera and audio data from the microphone. The device records this data in real time and prepares it as data packets for subsequent processing. Simultaneous acquisition of video and audio is crucial in this process.

[0162] Step 2:

[0163] The terminal sends the data collected in Step 1 to the server. This input includes encrypted video and audio data. The output is sent to the server via this secure data transfer. The terminal transfers the data over the network, ensuring that it reaches the server in real time.

[0164] Step 3:

[0165] The server receives data transmitted from the terminal. The server's input is the previously transmitted video and audio data. After confirming its reception, it passes it on to the analysis process. Checksums and other methods are used to ensure that the data is received accurately and completely.

[0166] Step 4:

[0167] The server uses a generative AI model to analyze video data and detect suspicious behavior and unusual patterns. The input data consists of video frames, which are analyzed by the AI ​​model and output results identifying suspicious individuals. These results assess potential threats based on specific movements and appearances.

[0168] Step 5:

[0169] The server analyzes voice data using emotion analysis tools to determine the user's emotional state. Input includes voice tone, speed, and pauses between words. From this, the system obtains an emotional evaluation of the user as output. The system uses this to identify emotions such as anxiety or reassurance.

[0170] Step 6:

[0171] The server integrates the results of identifying suspicious individuals with the user's emotional assessment to generate optimal warning signals and notification content. The input includes the aforementioned analysis results. The output generates adjusted warning sounds and notification messages, which are modified according to the situation. This step takes the user's psychological state into account to ensure a more effective response.

[0172] Step 7:

[0173] The user receives notifications from the server and takes action as needed. The input is a notification message, which the user uses to select the appropriate action. The output could be, for example, reporting to the police or providing additional information to the system. The user's choices help improve the system's future response accuracy.

[0174] (Application Example 2)

[0175] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0176] Conventional security systems are solely designed to detect intruders and lack the flexibility to respond flexibly to the user's mental state and emotional condition. In particular, emotion analysis capabilities were needed to provide appropriate warnings and notifications to users who are feeling anxious.

[0177] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0178] In this invention, the server includes emotion analysis means for analyzing the user's emotions from their voice, optimization means for optimizing the content of the warning or notification method based on the suspicious person determination result by the analysis means, and information notification means for notifying the suspicious person determination result by the analysis means to the notification device. This enables appropriate crime prevention measures according to the user's emotional state.

[0179] An "artificial intelligence model" is a system that uses machine learning techniques to analyze information and detect specific patterns or anomalies.

[0180] A "processing device" is a computer system used for data input, analysis, and output.

[0181] "Analysis means" refers to an algorithm or process for deriving specific information based on input data.

[0182] "Audio warning means" refers to a device or system that has the function of generating and transmitting audio or sound signals to alert of the presence of a suspicious person.

[0183] "Information notification means" refers to a means of communication for transmitting analysis results to the user or other devices.

[0184] "Emotional analysis means" refers to a technology that uses audio or video data to determine an individual's emotional state.

[0185] An "optimization method" is a technique for adjusting the operation and output of a system according to a specific purpose in order to obtain the most effective results.

[0186] The system that realizes this invention consists of a server, a terminal, and a user. The server functions as a processing unit and uses an artificial intelligence model and emotion analysis means to detect suspicious persons and analyze the user's emotional state. Data for suspicious person detection is acquired in real time from the camera and microphone installed in the terminal and transmitted to the server as audio and video information.

[0187] The server uses an artificial intelligence model to detect suspicious individuals based on acquired video information, and simultaneously analyzes the user's emotional state using emotion analysis tools based on audio data. Based on the results of the emotion analysis, the server uses optimization tools to optimize the tone of warning sounds and the content of notifications sent to the user. Optimized warnings and suggestions, based on both the detection of suspicious individuals and the user's emotional state, are then delivered to the user.

[0188] As a concrete example, if a user speaks in an anxious voice at a location where a suspicious person is present at night, the server can analyze their emotions, issue an intimidating voice warning, and quickly suggest contacting a security company. This system enables detailed security responses tailored to the user's emotions, providing a higher level of safety.

[0189] An example of a prompt is, "Explain how this security system provides reassuring notifications to users who are concerned about nighttime security." This prompt prompts the AI ​​model to generate appropriate warnings and suggestions. In this way, flexible responses that take user emotions into consideration are achieved.

[0190] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0191] Step 1:

[0192] The device acquires surrounding video and audio information in real time. This information is obtained using the camera and microphone and sent to the server as input data.

[0193] Step 2:

[0194] The server receives video and audio data transmitted from the terminal. The received video data is analyzed using an artificial intelligence model to determine the presence of a suspicious person. The result of this determination is used as output information regarding the presence or absence of a suspicious person.

[0195] Step 3:

[0196] The server simultaneously analyzes the audio data using emotion analysis tools. It analyzes the tone and speed of the voice from the audio data to determine the user's emotional state. The determined emotional state is output and used in the next optimization step.

[0197] Step 4:

[0198] Based on the results of the suspicious person detection and emotion analysis, the server uses optimization techniques to determine the content of the warning and the notification method. For example, if it is determined that the user is feeling anxious, the warning sound is adjusted to be more intimidating as the output.

[0199] Step 5:

[0200] The server transmits optimized warning content to the terminal via an audio warning system and issues a command to issue an audio warning. At this time, the warning sound and notification content are output to the user.

[0201] Step 6:

[0202] Users receive notifications on their devices. These notifications include emotion-based suggestions and action plans, allowing users to choose how they respond. This feedback is reused within the system to help optimize future features.

[0203] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0204] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0205] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0206] [Second Embodiment]

[0207] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0208] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0209] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0210] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0211] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0212] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0213] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0214] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0215] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0216] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0217] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0218] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0219] The security system according to the present invention enables the detection of suspicious behavior or appearance in real time and the immediate implementation of countermeasures. This system consists of three main elements: a terminal, a server, and a user.

[0220] The terminal used is a camera equipped with video capture capabilities. This camera continuously acquires video data and transmits it to the server. The video is high resolution and equipped with an infrared sensor to operate even at night.

[0221] The server receives video data transmitted from the terminal. Internally, it incorporates an artificial intelligence model for analyzing the video, which allows for detailed analysis of a person's behavior, posture, clothing, and belongings. If the analysis detects suspicious behavior or appearance, the server sends a command to an audio warning system. This audio warning system sends a signal to the relevant camera, which then emits a warning sound at the location.

[0222] Furthermore, the server communicates the analysis results and warnings to the user through information notification systems. This process utilizes smartphone applications and email notification systems, allowing users to respond immediately. The notification includes detailed information about the detected suspicious person, video snapshots, the time of detection, and the location.

[0223] A concrete example is when a suspicious person is loitering in a yard at night. In this case, the device captures the person's image and sends the data to a server. The server uses an AI model to analyze the person's behavior, and if any suspicious movements or locations are detected, it immediately issues a warning. At the same time, the user is notified of the information, allowing them to quickly take the next step, such as contacting the police. As a result, criminal activity can be prevented, and the safety of residents can be ensured.

[0224] Thus, this system enables real-time detection and immediate response to suspicious individuals, significantly improving the security of private homes.

[0225] The following describes the processing flow.

[0226] Step 1:

[0227] The device captures video in real time via the security camera and sends the video data to the server at regular frame intervals. The camera operates in high-resolution mode and is configured to acquire clear images day and night.

[0228] Step 2:

[0229] The server receives video data sent from the terminal in real time. As a preprocessing step for the video, it performs noise reduction and resolution optimization to generate a dataset suitable for analysis.

[0230] Step 3:

[0231] The server inputs pre-processed video data into an artificial intelligence model. This AI model is built on a deep learning algorithm and automatically analyzes people's movements and appearances. The model learns suspicious behaviors and appearances based on past data, and performs real-time pattern recognition accordingly.

[0232] Step 4:

[0233] The server identifies suspicious individuals based on the analysis results of the AI ​​model. If the analysis detects suspicious behavior, the server generates the necessary commands to take immediate action. These commands include instructions to generate an alarm sound.

[0234] Step 5:

[0235] Upon receiving instructions from the server, the terminal emits a warning sound through a speaker at the site. The warning sound is generated based on pre-configured parameters, and its volume and duration are adjusted accordingly. This warning sound prompts the suspicious person to leave the area.

[0236] Step 6:

[0237] Simultaneously, the server notifies the user of the detection results. Details of the suspicious person's movements, as well as information about the time and location of the alert, are sent to the user's smartphone or other mobile device. The user can then receive the notification and quickly consider how to respond.

[0238] Step 7:

[0239] Based on the notification, users can take countermeasures such as contacting the police or further utilizing the system. If necessary, user feedback is sent to the server and used as data to improve the accuracy of the AI ​​model.

[0240] (Example 1)

[0241] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0242] In recent years, the importance of crime prevention has been increasing in society. However, conventional surveillance systems have difficulty detecting suspicious behavior in real time and taking appropriate action immediately. Therefore, there is a need for a crime prevention system that can efficiently and quickly identify suspicious individuals and issue warnings.

[0243] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0244] In this invention, the server includes an analysis device equipped with an artificial intelligence model for receiving video information acquired from an image acquisition device and analyzing a person's behavior, posture, clothing, and belongings based on the video information; an audio warning device that recognizes suspicious behavior from the analysis results of a person's behavior or appearance by the analysis device and generates a command to send a warning signal to an audio output device to emit a warning sound; and an information notification device for notifying the user of the analysis results of suspicious behavior or appearance by the analysis device via a notification device. This enables real-time detection of suspicious persons, rapid warning issuance, and information notification.

[0245] An "image acquisition device" is a device for acquiring video information of the surroundings and has the function of monitoring a certain range.

[0246] An "analysis device" is a device that analyzes the movement, appearance, and shape of people and objects based on acquired video information, and has the ability to recognize a variety of patterns.

[0247] An "artificial intelligence model" is a program used to analyze video information and identify specific behaviors or characteristics, and it applies machine learning and deep learning technologies.

[0248] An "audio warning device" is a device that emits a warning sound when it receives a specific signal, and has the function of alerting suspicious individuals.

[0249] A "notification device" is a device used to communicate analysis results to the user and has the function of transmitting information via a communication network.

[0250] A "user terminal" is the device on the user's side that ultimately receives the information, and can take various forms, such as a smartphone or a computer.

[0251] This invention provides a security system that can detect suspicious behavior or appearance in real time and respond immediately. This system mainly consists of three elements: a terminal, a server, and a user.

[0252] The terminal used is a camera, which is a video acquisition device. This camera can capture clear images day and night and is equipped with high resolution and an infrared sensor. The terminal continuously collects video data and sends it to the server. This transmission is performed in real time over the network.

[0253] The server receives video information transmitted from the terminal. An artificial intelligence model is used to analyze the video. Specifically, a generative AI model is used to analyze the actions, postures, clothing, and belongings of people in the video data. This AI model uses advanced image recognition technology and is excellent at recognizing patterns and detecting abnormal movements. If suspicious behavior is detected as a result of the analysis, the server sends a warning signal to an audio warning device. This signal immediately emits a warning sound at the site, alerting the suspicious person to be vigilant.

[0254] Furthermore, the server notifies the user of the analysis results and the fact of the warning. The notification is sent to the user's terminal via a notification device. User terminals such as smartphones and computers can receive detailed information of the analysis results in real time. The notification includes the characteristics of the suspicious person, video snapshots, and information on the time and location of detection.

[0255] As a concrete example, imagine a situation where a suspicious person is active in a garden at night. In this case, the device captures the person's image and sends the data to a server. The server uses an AI model to analyze the person's actions, and if suspicious activity is detected, it immediately issues a warning. Simultaneously, the user receives a prompt message such as, "A suspicious person has been detected in your garden. Please check their actions." This allows the user to understand the situation and take immediate action.

[0256] This system significantly improves home security, enabling real-time detection of intruders and rapid response.

[0257] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0258] Step 1:

[0259] The device uses a camera, an image acquisition device, to acquire surrounding video data in real time. In this step, the camera uses high resolution and an infrared sensor to capture clear video data 24 hours a day. The input is the surrounding visual information, and the output is the captured video data. This video data is transmitted directly to the next step.

[0260] Step 2:

[0261] The server receives video data transmitted from the terminal. Using this data as input, an AI model within the server performs video analysis. The data processing involves recognizing the movement, posture, clothing, and possessions of a person in each frame of the video and analyzing their characteristics. The output is the analysis results for each frame. Specifically, the AI ​​model tracks the person's movement and detects abnormal patterns.

[0262] Step 3:

[0263] Based on the analysis results obtained in step 2, the server evaluates the behavior and appearance to determine whether or not suspicious behavior is present. The input is the analyzed behavioral data, and the data calculation involves comparing whether the person's movements and behavioral patterns fall within the normal range. The output is a determination of whether or not suspicious behavior was detected.

[0264] Step 4:

[0265] If the server detects suspicious activity, it sends a command to the audio warning device. This command instructs the terminal's warning system to emit a warning sound. The input is the result of the suspicious activity detection, and the output is the actual sound emitted. Specifically, the audio warning device emits a loud warning sound to draw attention to the area.

[0266] Step 5:

[0267] The server transmits the analysis results and warning information to the user via an information notification device. The input is the judgment result and the generated warning information, and the output is a notification sent to the user's terminal. Specifically, this includes the display of information such as video snapshots, detection time, and location on a smartphone or computer.

[0268] Step 6:

[0269] After the user receives a notification, they take the necessary action based on its content. The input is notification information from the server, and the output is a specific action taken by the user (e.g., contacting the police). Specifically, the user checks the details through the application and considers a course of action.

[0270] (Application Example 1)

[0271] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0272] In modern society, criminal activity and illegal activities by suspicious individuals have become a serious problem. However, conventional security systems have limited ability to detect suspicious movements and individuals in real time, making rapid response difficult. Identifying suspicious individuals and responding immediately, especially in homes and businesses, is a crucial issue directly linked to improving safety.

[0273] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0274] In this invention, the server includes an analysis means for receiving video data and identifying suspicious individuals, an audio alarm means for issuing commands to generate audio alarms, an information notification means for transmitting information to an information device, and a data communication means for providing video snapshots to user devices. This enables real-time detection of suspicious behavior or appearances, rapid alarm activation and immediate notification to users, and provision of video snapshots, thereby facilitating a quick response and improving security.

[0275] "Suspicious behavior or appearance" refers to any state that deviates from normal behavior or appearance and is judged to potentially threaten safety.

[0276] An "intelligent model" is an algorithm that uses technologies such as machine learning and deep learning to learn patterns from data and identify suspicious individuals.

[0277] A "processing unit" is a computing device used to process data and perform specified tasks.

[0278] "Video data" refers to the digital format of visual information captured by a camera or other recording device.

[0279] "Analysis methods" refer to mechanisms that process data received to identify suspicious individuals.

[0280] An "audio alarm system" is a device or system that generates and emits an audible warning when a suspicious person is detected.

[0281] An "information device" is a machine equipped with communication and display functions to enable notifications and data transfer.

[0282] "Data communication means" refers to communication technology used to transmit video and analysis results to user devices.

[0283] The present invention is specifically implemented as a security system that enables the detection of suspicious persons and immediate response. Below, the operation of this system, the hardware and software used, as well as specific examples will be described.

[0284] The server is provided with analysis means for image processing and receives video data transmitted from surveillance cameras in real time. This data is in a format that enables high-resolution night-time shooting. The server uses software libraries such as Python and TensorFlow as intelligent models to perform data analysis for detecting suspicious actions or appearances.

[0285] When image analysis is performed and a suspicious action or appearance is detected, the server issues an alarm by utilizing voice alarm means. For this purpose, hardware devices such as digital speakers and alarm devices are used. Furthermore, a notification containing information about the suspicious person is transmitted to the user's individual device, such as a smartphone or tablet, via the information notification means. This notification includes a real-time video snapshot and detailed information about the detected suspicious person.

[0286] When the user receives these notifications, they can view the video snapshots and past data on the application. Additionally, it also has an additional function for direct contact with the police or security company if necessary.

[0287] As a specific example, there is a case where a user detects suspicious movement in their home garden during daily shopping. The user checks the smartphone notification, views the video snapshot of the security camera, and confirms the abnormality. Thereby, it is possible to quickly contact the neighboring residents and take measures to ensure safety.

[0288] An example of a prompt might be: "Use the video feed from the camera installed in the garden to monitor for suspicious activity or individuals in real time and immediately notify the user's smartphone. If necessary, allow the user to access police contact information." This type of prompt appropriately provides the necessary instructions to the generating AI model.

[0289] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0290] Step 1:

[0291] The terminal continuously acquires video data from the installed camera. The input is real-time high-resolution video, which is captured and transmitted to the server. In operation, the terminal's video capture function works, and data is collected even at night using an infrared sensor.

[0292] Step 2:

[0293] The server receives video data transmitted from the terminal. The input is real-time video data, which is processed by an intelligent model (using Python and TensorFlow). Here, data analysis is performed, and movement and appearance within the video frames are evaluated to detect suspicious behavior or appearance.

[0294] Step 3:

[0295] The server activates an audio alarm system if suspicious activity is detected. The input is the result of the intelligent model's determination of suspicious activity; based on this, it creates an alarm and outputs a command. Specifically, it uses a digital speaker to emit an alarm sound at the location.

[0296] Step 4:

[0297] The server uses an information notification system to send information about suspicious individuals and video snapshots to the user's device. The input is the result of the suspicious activity detection and the snapshot data, and the output is a notification containing this information. This operation involves rapidly distributing the data over a communication network.

[0298] Step 5:

[0299] The user checks the received notification on their smartphone or tablet and views the video snapshot. The input is the notification data from the server, and the output is the displayed snapshot and details of the suspicious person. The user reviews the video to determine whether they need to contact the police or security services.

[0300] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0301] The security system according to the present invention not only detects suspicious individuals but also analyzes the user's emotions, enabling a more flexible and human-like response. The system consists of a terminal, a server, and a user, and its functionality is enhanced by incorporating an emotion engine.

[0302] In addition to conventional security camera functions, the device is equipped with audio input devices such as a microphone. This allows the device to collect ambient audio data and provide information for use in analyzing the user's emotions. Audio and video data are transmitted to the server in real time.

[0303] The server receives the video and audio data transmitted from the terminal. Based on this, while the AI model detects suspicious persons, the emotion engine analyzes the user's emotional state. The emotion engine analyzes the tone and way of speaking of the voice, as well as the expressions that can be read from the video, and evaluates the user's mental state based on the results. The server combines the emotion analysis results with the results of the AI model to optimize the content of the voice warning for the detected suspicious person and the subsequent notification method.

[0304] As a specific case, consider the situation where a suspicious person is loitering in front of the entrance at night. The terminal captures the video and audio and transmits the data to the server. The server detects the suspicious person from the video and analyzes the emotion from the voice uttered by the user towards the system. For example, if an anxious emotion is recognized from the user's voice, the server can, through this feedback, adjust the tone of the voice warning to be more intimidating, or add a message suggesting reporting to the police to the notification to the user.

[0305] On the other hand, after receiving the notification, the user can choose a countermeasure according to the advice based on the emotion presented by the system. If necessary, by providing further information as feedback to the system, it can help improve the response accuracy of the system.

[0306] In this way, by introducing the emotion engine in the present invention, flexible responses reflecting the user's emotions become possible, and a higher level of security can be achieved.

[0307] The following describes the processing flow.

[0308] Step 1:

[0309] The terminal captures video using a security camera and at the same time collects ambient audio data using a microphone. These data are transmitted to the server in real time. The camera operates 24 hours a day with high image quality, and the microphone is set to record natural voices.

[0310] Step 2:

[0311] The server receives video and audio data transmitted from the terminal. The received data is preprocessed, including noise reduction and resolution optimization, to prepare it for analysis by AI models and emotion engines.

[0312] Step 3:

[0313] The server uses an AI model to analyze video data and detect suspicious individuals. Specifically, it analyzes a person's movements, clothing, and belongings, and compares them with past data to determine if there is anything unusual. Simultaneously, it inputs audio data into an emotion engine to analyze the user's emotional state.

[0314] Step 4:

[0315] The server integrates the analysis results and, if a suspicious person is detected, adjusts the output of the voice warning system, taking into account the evaluation of the emotion engine. For example, if the user's emotions indicate anxiety, the warning sound is set to a more alarming tone.

[0316] Step 5:

[0317] The terminal receives instructions from the server and emits an appropriate warning sound from a speaker installed at the site. This warning sound is intended to alert the suspicious person to leave the area, and the volume and tone are dynamically adjusted according to the situation.

[0318] Step 6:

[0319] The server generates and sends notifications to the user based on the analysis results of the suspicious person and the user's sentiment evaluation. These notifications include information about the detected suspicious person, recommended actions based on the user's sentiment, and suggestions to report to the police.

[0320] Step 7:

[0321] Based on the notifications received from the server, users can consider the suggested countermeasures and take action, such as contacting the police, if necessary. They can also send feedback on the notifications to the server, contributing to system improvements.

[0322] (Example 2)

[0323] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0324] Traditional security systems focus on identifying suspicious individuals, but they lack flexibility in considering the user's feelings, resulting in warnings and notifications not being provided in a way that is optimal for the user's situation. As a result, the system may give users an insufficient sense of security.

[0325] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0326] In this invention, the server includes an analysis means for inputting video material and determining a suspicious person, an audio output means for generating a command to emit an audio signal based on the suspicious person determination result, and an emotion analysis means for inputting audio material and evaluating the user's emotional state. This makes it possible to combine the suspicious person detection result and the user's emotion analysis result to provide the user with the most appropriate response, thereby improving safety and security.

[0327] "Suspicious behavior" refers to actions or movements that deviate from normal behavioral patterns and may pose a potential danger.

[0328] An "artificial intelligence model" is a model with a program structure that learns from large amounts of data and can identify specific patterns or anomalies.

[0329] A "computer" is an electronic device used for inputting, processing, and outputting data.

[0330] "Visual materials" refers to visual information data collected by cameras or other video acquisition devices.

[0331] "Analysis means" refers to the process or device that analyzes input data and derives meaningful information.

[0332] "Audio signals" refer to acoustic information used for warnings and commands, generated based on audio data.

[0333] "Audio output means" refers to a device or mechanism for generating an audio signal based on the analysis results and transmitting it externally.

[0334] A "notification mechanism" is a device or system that conveys specific information to users through sound or visual signals.

[0335] "Information transmission means" refers to means for transmitting judgment results and related information to other devices or users.

[0336] "Emotional analysis means" refers to a process or mechanism that analyzes voice tone, manner of speaking, or facial expressions to evaluate the user's emotional state.

[0337] A "control means" is a process or device that combines results obtained from different data to perform optimal processing or output.

[0338] The security system of this invention consists of terminals, servers, and users, and provides a high level of security through the coordinated functioning of each element.

[0339] The terminal functions as a security camera and collects video information. Furthermore, the terminal is equipped with a microphone and has the capability to collect audio information. The terminal transmits the collected video and audio information to a server in real time. This communication is typically conducted via a wireless or wired network, and the collected data is encrypted to ensure security.

[0340] The server plays a central role in receiving and analyzing data transmitted from terminals. The server utilizes generative AI models to analyze video information and identify individuals exhibiting suspicious behavior. It also uses emotion analysis to evaluate the user's emotional state from audio information. This emotion analysis is based on factors such as voice tone, speed, and pauses to determine the user's psychological state. The analysis results are integrated by the server and used to generate warning signals against suspicious individuals and optimize notification methods.

[0341] As a concrete example, consider a scenario where a suspicious person appears at the front door at night. The device records this situation with video and audio and sends it to the server. The server identifies the suspicious person using a generative AI model and simultaneously analyzes the user's emotions from the audio data. For example, if the analysis indicates that the user is feeling anxious, the server may intensify the warning sound or generate a notification message suggesting that the police be notified.

[0342] The user receives a notification from the server and selects an appropriate response based on the information and suggestions provided. An example of a specific prompt message is: "Analyze the suspicious person at the front door and the user's emotions at that time, and suggest an appropriate voice warning and notification method."

[0343] In this way, by having each element work in conjunction, it is possible to realize a comprehensive security system that not only detects suspicious individuals but also takes into account the user's emotions.

[0344] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0345] Step 1:

[0346] The device collects surrounding video and audio information. Inputs include visual data from the camera and audio data from the microphone. The device records this data in real time and prepares it as data packets for subsequent processing. Simultaneous acquisition of video and audio is crucial in this process.

[0347] Step 2:

[0348] The terminal sends the data collected in Step 1 to the server. This input includes encrypted video and audio data. The output is sent to the server via this secure data transfer. The terminal transfers the data over the network, ensuring that it reaches the server in real time.

[0349] Step 3:

[0350] The server receives data transmitted from the terminal. The server's input is the previously transmitted video and audio data. After confirming its reception, it passes it on to the analysis process. Checksums and other methods are used to ensure that the data is received accurately and completely.

[0351] Step 4:

[0352] The server uses a generative AI model to analyze video data and detect suspicious behavior and unusual patterns. The input data consists of video frames, which are analyzed by the AI ​​model and output results identifying suspicious individuals. These results assess potential threats based on specific movements and appearances.

[0353] Step 5:

[0354] The server analyzes voice data using emotion analysis tools to determine the user's emotional state. Input includes voice tone, speed, and pauses between words. From this, the system obtains an emotional evaluation of the user as output. The system uses this to identify emotions such as anxiety or reassurance.

[0355] Step 6:

[0356] The server integrates the results of identifying suspicious individuals with the user's emotional assessment to generate optimal warning signals and notification content. The input includes the aforementioned analysis results. The output generates adjusted warning sounds and notification messages, which are modified according to the situation. This step takes the user's psychological state into account to ensure a more effective response.

[0357] Step 7:

[0358] The user receives notifications from the server and takes action as needed. The input is a notification message, which the user uses to select the appropriate action. The output could be, for example, reporting to the police or providing additional information to the system. The user's choices help improve the system's future response accuracy.

[0359] (Application Example 2)

[0360] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0361] Conventional security systems are solely designed to detect intruders and lack the flexibility to respond flexibly to the user's mental state and emotional condition. In particular, emotion analysis capabilities were needed to provide appropriate warnings and notifications to users who are feeling anxious.

[0362] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0363] In this invention, the server includes emotion analysis means for analyzing the user's emotions from their voice, optimization means for optimizing the content of the warning or notification method based on the suspicious person determination result by the analysis means, and information notification means for notifying the suspicious person determination result by the analysis means to the notification device. This enables appropriate crime prevention measures according to the user's emotional state.

[0364] An "artificial intelligence model" is a system that uses machine learning techniques to analyze information and detect specific patterns or anomalies.

[0365] A "processing device" is a computer system used for data input, analysis, and output.

[0366] "Analysis means" refers to an algorithm or process for deriving specific information based on input data.

[0367] "Audio warning means" refers to a device or system that has the function of generating and transmitting audio or sound signals to alert of the presence of a suspicious person.

[0368] "Information notification means" refers to a means of communication for transmitting analysis results to the user or other devices.

[0369] "Emotional analysis means" refers to a technology that uses audio or video data to determine an individual's emotional state.

[0370] An "optimization method" is a technique for adjusting the operation and output of a system according to a specific purpose in order to obtain the most effective results.

[0371] The system that realizes this invention consists of a server, a terminal, and a user. The server functions as a processing unit and uses an artificial intelligence model and emotion analysis means to detect suspicious persons and analyze the user's emotional state. Data for suspicious person detection is acquired in real time from the camera and microphone installed in the terminal and transmitted to the server as audio and video information.

[0372] The server uses an artificial intelligence model to detect suspicious individuals based on acquired video information, and simultaneously analyzes the user's emotional state using emotion analysis tools based on audio data. Based on the results of the emotion analysis, the server uses optimization tools to optimize the tone of warning sounds and the content of notifications sent to the user. Optimized warnings and suggestions, based on both the detection of suspicious individuals and the user's emotional state, are then delivered to the user.

[0373] As a concrete example, if a user speaks in an anxious voice at a location where a suspicious person is present at night, the server can analyze their emotions, issue an intimidating voice warning, and quickly suggest contacting a security company. This system enables detailed security responses tailored to the user's emotions, providing a higher level of safety.

[0374] An example of a prompt is, "Explain how this security system provides reassuring notifications to users who are concerned about nighttime security." This prompt prompts the AI ​​model to generate appropriate warnings and suggestions. In this way, flexible responses that take user emotions into consideration are achieved.

[0375] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0376] Step 1:

[0377] The device acquires surrounding video and audio information in real time. This information is obtained using the camera and microphone and sent to the server as input data.

[0378] Step 2:

[0379] The server receives video and audio data transmitted from the terminal. The received video data is analyzed using an artificial intelligence model to determine the presence of a suspicious person. The result of this determination is used as output information regarding the presence or absence of a suspicious person.

[0380] Step 3:

[0381] The server simultaneously analyzes the audio data using emotion analysis tools. It analyzes the tone and speed of the voice from the audio data to determine the user's emotional state. The determined emotional state is output and used in the next optimization step.

[0382] Step 4:

[0383] Based on the results of the suspicious person detection and emotion analysis, the server uses optimization techniques to determine the content of the warning and the notification method. For example, if it is determined that the user is feeling anxious, the warning sound is adjusted to be more intimidating as the output.

[0384] Step 5:

[0385] The server transmits optimized warning content to the terminal via an audio warning system and issues a command to issue an audio warning. At this time, the warning sound and notification content are output to the user.

[0386] Step 6:

[0387] Users receive notifications on their devices. These notifications include emotion-based suggestions and action plans, allowing users to choose how they respond. This feedback is reused within the system to help optimize future features.

[0388] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0389] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0390] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0391] [Third Embodiment]

[0392] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0393] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0394] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0395] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0396] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0397] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0398] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0399] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0400] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0401] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0402] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0403] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0404] The security system according to the present invention enables the detection of suspicious behavior or appearance in real time and the immediate implementation of countermeasures. This system consists of three main elements: a terminal, a server, and a user.

[0405] The terminal used is a camera equipped with video capture capabilities. This camera continuously acquires video data and transmits it to the server. The video is high resolution and equipped with an infrared sensor to operate even at night.

[0406] The server receives video data transmitted from the terminal. Internally, it incorporates an artificial intelligence model for analyzing the video, which allows for detailed analysis of a person's behavior, posture, clothing, and belongings. If the analysis detects suspicious behavior or appearance, the server sends a command to an audio warning system. This audio warning system sends a signal to the relevant camera, which then emits a warning sound at the location.

[0407] Furthermore, the server communicates the analysis results and warnings to the user through information notification systems. This process utilizes smartphone applications and email notification systems, allowing users to respond immediately. The notification includes detailed information about the detected suspicious person, video snapshots, the time of detection, and the location.

[0408] A concrete example is when a suspicious person is loitering in a yard at night. In this case, the device captures the person's image and sends the data to a server. The server uses an AI model to analyze the person's behavior, and if any suspicious movements or locations are detected, it immediately issues a warning. At the same time, the user is notified of the information, allowing them to quickly take the next step, such as contacting the police. As a result, criminal activity can be prevented, and the safety of residents can be ensured.

[0409] Thus, this system enables real-time detection and immediate response to suspicious individuals, significantly improving the security of private homes.

[0410] The following describes the processing flow.

[0411] Step 1:

[0412] The device captures video in real time via the security camera and sends the video data to the server at regular frame intervals. The camera operates in high-resolution mode and is configured to acquire clear images day and night.

[0413] Step 2:

[0414] The server receives video data sent from the terminal in real time. As a preprocessing step for the video, it performs noise reduction and resolution optimization to generate a dataset suitable for analysis.

[0415] Step 3:

[0416] The server inputs pre-processed video data into an artificial intelligence model. This AI model is built on a deep learning algorithm and automatically analyzes people's movements and appearances. The model learns suspicious behaviors and appearances based on past data, and performs real-time pattern recognition accordingly.

[0417] Step 4:

[0418] The server identifies suspicious individuals based on the analysis results of the AI ​​model. If the analysis detects suspicious behavior, the server generates the necessary commands to take immediate action. These commands include instructions to generate an alarm sound.

[0419] Step 5:

[0420] Upon receiving instructions from the server, the terminal emits a warning sound through a speaker at the site. The warning sound is generated based on pre-configured parameters, and its volume and duration are adjusted accordingly. This warning sound prompts the suspicious person to leave the area.

[0421] Step 6:

[0422] Simultaneously, the server notifies the user of the detection results. Details of the suspicious person's movements, as well as information about the time and location of the alert, are sent to the user's smartphone or other mobile device. The user can then receive the notification and quickly consider how to respond.

[0423] Step 7:

[0424] Based on the notification, users can take countermeasures such as contacting the police or further utilizing the system. If necessary, user feedback is sent to the server and used as data to improve the accuracy of the AI ​​model.

[0425] (Example 1)

[0426] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0427] In recent years, the importance of crime prevention has been increasing in society. However, conventional surveillance systems have difficulty detecting suspicious behavior in real time and taking appropriate action immediately. Therefore, there is a need for a crime prevention system that can efficiently and quickly identify suspicious individuals and issue warnings.

[0428] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0429] In this invention, the server includes an analysis device equipped with an artificial intelligence model for receiving video information acquired from an image acquisition device and analyzing a person's behavior, posture, clothing, and belongings based on the video information; an audio warning device that recognizes suspicious behavior from the analysis results of a person's behavior or appearance by the analysis device and generates a command to send a warning signal to an audio output device to emit a warning sound; and an information notification device for notifying the user of the analysis results of suspicious behavior or appearance by the analysis device via a notification device. This enables real-time detection of suspicious persons, rapid warning issuance, and information notification.

[0430] An "image acquisition device" is a device for acquiring video information of the surroundings and has the function of monitoring a certain range.

[0431] An "analysis device" is a device that analyzes the movement, appearance, and shape of people and objects based on acquired video information, and has the ability to recognize a variety of patterns.

[0432] An "artificial intelligence model" is a program used to analyze video information and identify specific behaviors or characteristics, and it applies machine learning and deep learning technologies.

[0433] An "audio warning device" is a device that emits a warning sound when it receives a specific signal, and has the function of alerting suspicious individuals.

[0434] A "notification device" is a device used to communicate analysis results to the user and has the function of transmitting information via a communication network.

[0435] A "user terminal" is the device on the user's side that ultimately receives the information, and can take various forms, such as a smartphone or a computer.

[0436] This invention provides a security system that can detect suspicious behavior or appearance in real time and respond immediately. This system mainly consists of three elements: a terminal, a server, and a user.

[0437] The terminal used is a camera, which is a video acquisition device. This camera can capture clear images day and night and is equipped with high resolution and an infrared sensor. The terminal continuously collects video data and sends it to the server. This transmission is performed in real time over the network.

[0438] The server receives video information transmitted from the terminal. An artificial intelligence model is used to analyze the video. Specifically, a generative AI model is used to analyze the actions, postures, clothing, and belongings of people in the video data. This AI model uses advanced image recognition technology and is excellent at recognizing patterns and detecting abnormal movements. If suspicious behavior is detected as a result of the analysis, the server sends a warning signal to an audio warning device. This signal immediately emits a warning sound at the site, alerting the suspicious person to be vigilant.

[0439] Furthermore, the server notifies the user of the analysis results and the fact of the warning. The notification is sent to the user's terminal via a notification device. User terminals such as smartphones and computers can receive detailed information of the analysis results in real time. The notification includes the characteristics of the suspicious person, video snapshots, and information on the time and location of detection.

[0440] As a concrete example, imagine a situation where a suspicious person is active in a garden at night. In this case, the device captures the person's image and sends the data to a server. The server uses an AI model to analyze the person's actions, and if suspicious activity is detected, it immediately issues a warning. Simultaneously, the user receives a prompt message such as, "A suspicious person has been detected in your garden. Please check their actions." This allows the user to understand the situation and take immediate action.

[0441] This system significantly improves home security, enabling real-time detection of intruders and rapid response.

[0442] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0443] Step 1:

[0444] The device uses a camera, an image acquisition device, to acquire surrounding video data in real time. In this step, the camera uses high resolution and an infrared sensor to capture clear video data 24 hours a day. The input is the surrounding visual information, and the output is the captured video data. This video data is transmitted directly to the next step.

[0445] Step 2:

[0446] The server receives video data transmitted from the terminal. Using this data as input, an AI model within the server performs video analysis. The data processing involves recognizing the movement, posture, clothing, and possessions of a person in each frame of the video and analyzing their characteristics. The output is the analysis results for each frame. Specifically, the AI ​​model tracks the person's movement and detects abnormal patterns.

[0447] Step 3:

[0448] Based on the analysis results obtained in step 2, the server evaluates the behavior and appearance to determine whether or not suspicious behavior is present. The input is the analyzed behavioral data, and the data calculation involves comparing whether the person's movements and behavioral patterns fall within the normal range. The output is a determination of whether or not suspicious behavior was detected.

[0449] Step 4:

[0450] If the server detects suspicious activity, it sends a command to the audio warning device. This command instructs the terminal's warning system to emit a warning sound. The input is the result of the suspicious activity detection, and the output is the actual sound emitted. Specifically, the audio warning device emits a loud warning sound to draw attention to the area.

[0451] Step 5:

[0452] The server transmits the analysis results and warning information to the user via an information notification device. The input is the judgment result and the generated warning information, and the output is a notification sent to the user's terminal. Specifically, this includes the display of information such as video snapshots, detection time, and location on a smartphone or computer.

[0453] Step 6:

[0454] After the user receives a notification, they take the necessary action based on its content. The input is notification information from the server, and the output is a specific action taken by the user (e.g., contacting the police). Specifically, the user checks the details through the application and considers a course of action.

[0455] (Application Example 1)

[0456] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0457] In modern society, criminal activity and illegal activities by suspicious individuals have become a serious problem. However, conventional security systems have limited ability to detect suspicious movements and individuals in real time, making rapid response difficult. Identifying suspicious individuals and responding immediately, especially in homes and businesses, is a crucial issue directly linked to improving safety.

[0458] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0459] In this invention, the server includes an analysis means for receiving video data and identifying suspicious individuals, an audio alarm means for issuing commands to generate audio alarms, an information notification means for transmitting information to an information device, and a data communication means for providing video snapshots to user devices. This enables real-time detection of suspicious behavior or appearances, rapid alarm activation and immediate notification to users, and provision of video snapshots, thereby facilitating a quick response and improving security.

[0460] "Suspicious behavior or appearance" refers to any state that deviates from normal behavior or appearance and is judged to potentially threaten safety.

[0461] An "intelligent model" is an algorithm that uses technologies such as machine learning and deep learning to learn patterns from data and identify suspicious individuals.

[0462] A "processing unit" is a computing device used to process data and perform specified tasks.

[0463] "Video data" refers to the digital format of visual information captured by a camera or other recording device.

[0464] "Analysis methods" refer to mechanisms that process data received to identify suspicious individuals.

[0465] An "audio alarm system" is a device or system that generates and emits an audible warning when a suspicious person is detected.

[0466] An "information device" is a machine equipped with communication and display functions to enable notifications and data transfer.

[0467] "Data communication means" refers to communication technology used to transmit video and analysis results to user devices.

[0468] The present invention is specifically implemented as a security system that enables the detection of suspicious individuals and immediate response. The operation of this system, the hardware and software used, and specific embodiments are described below.

[0469] The server is equipped with an analysis system for image processing, which receives video data transmitted from surveillance cameras in real time. This data is in a high-resolution format that allows for nighttime recording. The server uses software libraries such as Python and TensorFlow as intelligent models to perform data analysis to detect suspicious behavior or appearances.

[0470] Image analysis is performed, and if suspicious behavior or appearance is detected, the server will issue an alarm using audio alarm means. This uses hardware devices such as digital speakers and alarm devices. Furthermore, an information notification means will send a notification containing information about the suspicious person to the user's individual device, such as a smartphone or tablet. This notification will include real-time video snapshots and detailed information about the detected suspicious person.

[0471] When users receive these notifications, they can view video snapshots and past data within the application. The application also includes additional features for directly contacting the police or security companies if necessary.

[0472] A concrete example is when a user detects suspicious activity in their yard while doing their daily shopping. The user checks the notification on their smartphone, views the security camera footage, and confirms the anomaly. This allows them to quickly contact their neighbors and take steps to ensure their safety.

[0473] An example of a prompt might be: "Use the video feed from the camera installed in the garden to monitor for suspicious activity or individuals in real time and immediately notify the user's smartphone. If necessary, allow the user to access police contact information." This type of prompt appropriately provides the necessary instructions to the generating AI model.

[0474] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0475] Step 1:

[0476] The terminal continuously acquires video data from the installed camera. The input is real-time high-resolution video, which is captured and transmitted to the server. In operation, the terminal's video capture function works, and data is collected even at night using an infrared sensor.

[0477] Step 2:

[0478] The server receives video data transmitted from the terminal. The input is real-time video data, which is processed by an intelligent model (using Python and TensorFlow). Here, data analysis is performed, and movement and appearance within the video frames are evaluated to detect suspicious behavior or appearance.

[0479] Step 3:

[0480] The server activates an audio alarm system if suspicious activity is detected. The input is the result of the intelligent model's determination of suspicious activity; based on this, it creates an alarm and outputs a command. Specifically, it uses a digital speaker to emit an alarm sound at the location.

[0481] Step 4:

[0482] The server uses an information notification system to send information about suspicious individuals and video snapshots to the user's device. The input is the result of the suspicious activity detection and the snapshot data, and the output is a notification containing this information. This operation involves rapidly distributing the data over a communication network.

[0483] Step 5:

[0484] The user checks the received notification on their smartphone or tablet and views the video snapshot. The input is the notification data from the server, and the output is the displayed snapshot and details of the suspicious person. The user reviews the video to determine whether they need to contact the police or security services.

[0485] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0486] The security system according to the present invention not only detects suspicious individuals but also analyzes the user's emotions, enabling a more flexible and human-like response. The system consists of a terminal, a server, and a user, and its functionality is enhanced by incorporating an emotion engine.

[0487] In addition to conventional security camera functions, the device is equipped with audio input devices such as a microphone. This allows the device to collect ambient audio data and provide information for use in analyzing the user's emotions. Audio and video data are transmitted to the server in real time.

[0488] The server receives video and audio data transmitted from the terminal. Based on this, an AI model detects suspicious individuals, while an emotion engine analyzes the user's emotional state. The emotion engine analyzes the tone of voice, speaking style, and facial expressions that can be read from the video, and evaluates the user's mental state based on the results. The server combines the emotion analysis results with the AI ​​model's results to optimize the content of the audio warning for detected suspicious individuals and the subsequent notification method.

[0489] A concrete example would be a situation where a suspicious person is loitering around the front door at night. The device captures video and audio and sends the data to a server. The server detects the suspicious person from the video and analyzes the user's emotions from the audio they speak to the system. For example, if the server detects anxiety from the user's voice, it can use that feedback to adjust the tone of the audio warning to be more intimidating, or add a message to the user's notification suggesting they contact the police.

[0490] On the other hand, after receiving a notification, users can choose a course of action based on the emotion-based advice provided by the system. Providing further information as needed helps improve the system's response accuracy.

[0491] Thus, by introducing the emotion engine in this invention, it becomes possible to respond flexibly to the user's emotions, thereby achieving a higher level of safety assurance.

[0492] The following describes the processing flow.

[0493] Step 1:

[0494] The device captures video using a security camera while simultaneously collecting ambient audio data using a microphone. This data is transmitted to a server in real time. The camera operates 24 hours a day in high definition, and the microphone is set to record natural-sounding audio.

[0495] Step 2:

[0496] The server receives video and audio data transmitted from the terminal. The received data is preprocessed, including noise reduction and resolution optimization, to prepare it for analysis by AI models and emotion engines.

[0497] Step 3:

[0498] The server uses an AI model to analyze video data and detect suspicious individuals. Specifically, it analyzes a person's movements, clothing, and belongings, and compares them with past data to determine if there is anything unusual. Simultaneously, it inputs audio data into an emotion engine to analyze the user's emotional state.

[0499] Step 4:

[0500] The server integrates the analysis results and, if a suspicious person is detected, adjusts the output of the voice warning system, taking into account the evaluation of the emotion engine. For example, if the user's emotions indicate anxiety, the warning sound is set to a more alarming tone.

[0501] Step 5:

[0502] The terminal receives instructions from the server and emits an appropriate warning sound from a speaker installed at the site. This warning sound is intended to alert the suspicious person to leave the area, and the volume and tone are dynamically adjusted according to the situation.

[0503] Step 6:

[0504] The server generates and sends notifications to the user based on the analysis results of the suspicious person and the user's sentiment evaluation. These notifications include information about the detected suspicious person, recommended actions based on the user's sentiment, and suggestions to report to the police.

[0505] Step 7:

[0506] Based on the notifications received from the server, users can consider the suggested countermeasures and take action, such as contacting the police, if necessary. They can also send feedback on the notifications to the server, contributing to system improvements.

[0507] (Example 2)

[0508] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0509] Traditional security systems focus on identifying suspicious individuals, but they lack flexibility in considering the user's feelings, resulting in warnings and notifications not being provided in a way that is optimal for the user's situation. As a result, the system may give users an insufficient sense of security.

[0510] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0511] In this invention, the server includes an analysis means for inputting video material and determining a suspicious person, an audio output means for generating a command to emit an audio signal based on the suspicious person determination result, and an emotion analysis means for inputting audio material and evaluating the user's emotional state. This makes it possible to combine the suspicious person detection result and the user's emotion analysis result to provide the user with the most appropriate response, thereby improving safety and security.

[0512] "Suspicious behavior" refers to actions or movements that deviate from normal behavioral patterns and may pose a potential danger.

[0513] An "artificial intelligence model" is a model with a program structure that learns from large amounts of data and can identify specific patterns or anomalies.

[0514] A "computer" is an electronic device used for inputting, processing, and outputting data.

[0515] "Visual materials" refers to visual information data collected by cameras or other video acquisition devices.

[0516] "Analysis means" refers to the process or device that analyzes input data and derives meaningful information.

[0517] "Audio signals" refer to acoustic information used for warnings and commands, generated based on audio data.

[0518] "Audio output means" refers to a device or mechanism for generating an audio signal based on the analysis results and transmitting it externally.

[0519] A "notification mechanism" is a device or system that conveys specific information to users through sound or visual signals.

[0520] "Information transmission means" refers to means for transmitting judgment results and related information to other devices or users.

[0521] "Emotional analysis means" refers to a process or mechanism that analyzes voice tone, manner of speaking, or facial expressions to evaluate the user's emotional state.

[0522] A "control means" is a process or device that combines results obtained from different data to perform optimal processing or output.

[0523] The security system of this invention consists of terminals, servers, and users, and provides a high level of security through the coordinated functioning of each element.

[0524] The terminal functions as a security camera and collects video information. Furthermore, the terminal is equipped with a microphone and has the capability to collect audio information. The terminal transmits the collected video and audio information to a server in real time. This communication is typically conducted via a wireless or wired network, and the collected data is encrypted to ensure security.

[0525] The server plays a central role in receiving and analyzing data transmitted from terminals. The server utilizes generative AI models to analyze video information and identify individuals exhibiting suspicious behavior. It also uses emotion analysis to evaluate the user's emotional state from audio information. This emotion analysis is based on factors such as voice tone, speed, and pauses to determine the user's psychological state. The analysis results are integrated by the server and used to generate warning signals against suspicious individuals and optimize notification methods.

[0526] As a concrete example, consider a scenario where a suspicious person appears at the front door at night. The device records this situation with video and audio and sends it to the server. The server identifies the suspicious person using a generative AI model and simultaneously analyzes the user's emotions from the audio data. For example, if the analysis indicates that the user is feeling anxious, the server may intensify the warning sound or generate a notification message suggesting that the police be notified.

[0527] The user receives a notification from the server and selects an appropriate response based on the information and suggestions provided. An example of a specific prompt message is: "Analyze the suspicious person at the front door and the user's emotions at that time, and suggest an appropriate voice warning and notification method."

[0528] In this way, by having each element work in conjunction, it is possible to realize a comprehensive security system that not only detects suspicious individuals but also takes into account the user's emotions.

[0529] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0530] Step 1:

[0531] The device collects surrounding video and audio information. Inputs include visual data from the camera and audio data from the microphone. The device records this data in real time and prepares it as data packets for subsequent processing. Simultaneous acquisition of video and audio is crucial in this process.

[0532] Step 2:

[0533] The terminal sends the data collected in Step 1 to the server. This input includes encrypted video and audio data. The output is sent to the server via this secure data transfer. The terminal transfers the data over the network, ensuring that it reaches the server in real time.

[0534] Step 3:

[0535] The server receives data transmitted from the terminal. The server's input is the previously transmitted video and audio data. After confirming its reception, it passes it on to the analysis process. Checksums and other methods are used to ensure that the data is received accurately and completely.

[0536] Step 4:

[0537] The server uses a generative AI model to analyze video data and detect suspicious behavior and unusual patterns. The input data consists of video frames, which are analyzed by the AI ​​model and output results identifying suspicious individuals. These results assess potential threats based on specific movements and appearances.

[0538] Step 5:

[0539] The server analyzes voice data using emotion analysis tools to determine the user's emotional state. Input includes voice tone, speed, and pauses between words. From this, the system obtains an emotional evaluation of the user as output. The system uses this to identify emotions such as anxiety or reassurance.

[0540] Step 6:

[0541] The server integrates the results of identifying suspicious individuals with the user's emotional assessment to generate optimal warning signals and notification content. The input includes the aforementioned analysis results. The output generates adjusted warning sounds and notification messages, which are modified according to the situation. This step takes the user's psychological state into account to ensure a more effective response.

[0542] Step 7:

[0543] The user receives notifications from the server and takes action as needed. The input is a notification message, which the user uses to select the appropriate action. The output could be, for example, reporting to the police or providing additional information to the system. The user's choices help improve the system's future response accuracy.

[0544] (Application Example 2)

[0545] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0546] Conventional security systems are solely designed to detect intruders and lack the flexibility to respond flexibly to the user's mental state and emotional condition. In particular, emotion analysis capabilities were needed to provide appropriate warnings and notifications to users who are feeling anxious.

[0547] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0548] In this invention, the server includes emotion analysis means for analyzing the user's emotions from their voice, optimization means for optimizing the content of the warning or notification method based on the suspicious person determination result by the analysis means, and information notification means for notifying the suspicious person determination result by the analysis means to the notification device. This enables appropriate crime prevention measures according to the user's emotional state.

[0549] An "artificial intelligence model" is a system that uses machine learning techniques to analyze information and detect specific patterns or anomalies.

[0550] A "processing device" is a computer system used for data input, analysis, and output.

[0551] "Analysis means" refers to an algorithm or process for deriving specific information based on input data.

[0552] "Audio warning means" refers to a device or system that has the function of generating and transmitting audio or sound signals to alert of the presence of a suspicious person.

[0553] "Information notification means" refers to a means of communication for transmitting analysis results to the user or other devices.

[0554] "Emotional analysis means" refers to a technology that uses audio or video data to determine an individual's emotional state.

[0555] An "optimization method" is a technique for adjusting the operation and output of a system according to a specific purpose in order to obtain the most effective results.

[0556] The system that realizes this invention consists of a server, a terminal, and a user. The server functions as a processing unit and uses an artificial intelligence model and emotion analysis means to detect suspicious persons and analyze the user's emotional state. Data for suspicious person detection is acquired in real time from the camera and microphone installed in the terminal and transmitted to the server as audio and video information.

[0557] The server uses an artificial intelligence model to detect suspicious individuals based on acquired video information, and simultaneously analyzes the user's emotional state using emotion analysis tools based on audio data. Based on the results of the emotion analysis, the server uses optimization tools to optimize the tone of warning sounds and the content of notifications sent to the user. Optimized warnings and suggestions, based on both the detection of suspicious individuals and the user's emotional state, are then delivered to the user.

[0558] As a concrete example, if a user speaks in an anxious voice at a location where a suspicious person is present at night, the server can analyze their emotions, issue an intimidating voice warning, and quickly suggest contacting a security company. This system enables detailed security responses tailored to the user's emotions, providing a higher level of safety.

[0559] An example of a prompt is, "Explain how this security system provides reassuring notifications to users who are concerned about nighttime security." This prompt prompts the AI ​​model to generate appropriate warnings and suggestions. In this way, flexible responses that take user emotions into consideration are achieved.

[0560] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0561] Step 1:

[0562] The device acquires surrounding video and audio information in real time. This information is obtained using the camera and microphone and sent to the server as input data.

[0563] Step 2:

[0564] The server receives video and audio data transmitted from the terminal. The received video data is analyzed using an artificial intelligence model to determine the presence of a suspicious person. The result of this determination is used as output information regarding the presence or absence of a suspicious person.

[0565] Step 3:

[0566] The server simultaneously analyzes the audio data using emotion analysis tools. It analyzes the tone and speed of the voice from the audio data to determine the user's emotional state. The determined emotional state is output and used in the next optimization step.

[0567] Step 4:

[0568] Based on the results of the suspicious person detection and emotion analysis, the server uses optimization techniques to determine the content of the warning and the notification method. For example, if it is determined that the user is feeling anxious, the warning sound is adjusted to be more intimidating as the output.

[0569] Step 5:

[0570] The server transmits optimized warning content to the terminal via an audio warning system and issues a command to issue an audio warning. At this time, the warning sound and notification content are output to the user.

[0571] Step 6:

[0572] Users receive notifications on their devices. These notifications include emotion-based suggestions and action plans, allowing users to choose how they respond. This feedback is reused within the system to help optimize future features.

[0573] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0574] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0575] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0576] [Fourth Embodiment]

[0577] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0578] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0579] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0580] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0581] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0582] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0583] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0584] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0585] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0586] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0587] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0588] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0589] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0590] The security system according to the present invention enables the detection of suspicious behavior or appearance in real time and the immediate implementation of countermeasures. This system consists of three main elements: a terminal, a server, and a user.

[0591] The terminal used is a camera equipped with video capture capabilities. This camera continuously acquires video data and transmits it to the server. The video is high resolution and equipped with an infrared sensor to operate even at night.

[0592] The server receives video data transmitted from the terminal. Internally, it incorporates an artificial intelligence model for analyzing the video, which allows for detailed analysis of a person's behavior, posture, clothing, and belongings. If the analysis detects suspicious behavior or appearance, the server sends a command to an audio warning system. This audio warning system sends a signal to the relevant camera, which then emits a warning sound at the location.

[0593] Furthermore, the server communicates the analysis results and warnings to the user through information notification systems. This process utilizes smartphone applications and email notification systems, allowing users to respond immediately. The notification includes detailed information about the detected suspicious person, video snapshots, the time of detection, and the location.

[0594] A concrete example is when a suspicious person is loitering in a yard at night. In this case, the device captures the person's image and sends the data to a server. The server uses an AI model to analyze the person's behavior, and if any suspicious movements or locations are detected, it immediately issues a warning. At the same time, the user is notified of the information, allowing them to quickly take the next step, such as contacting the police. As a result, criminal activity can be prevented, and the safety of residents can be ensured.

[0595] Thus, this system enables real-time detection and immediate response to suspicious individuals, significantly improving the security of private homes.

[0596] The following describes the processing flow.

[0597] Step 1:

[0598] The device captures video in real time via the security camera and sends the video data to the server at regular frame intervals. The camera operates in high-resolution mode and is configured to acquire clear images day and night.

[0599] Step 2:

[0600] The server receives video data sent from the terminal in real time. As a preprocessing step for the video, it performs noise reduction and resolution optimization to generate a dataset suitable for analysis.

[0601] Step 3:

[0602] The server inputs pre-processed video data into an artificial intelligence model. This AI model is built on a deep learning algorithm and automatically analyzes people's movements and appearances. The model learns suspicious behaviors and appearances based on past data, and performs real-time pattern recognition accordingly.

[0603] Step 4:

[0604] The server identifies suspicious individuals based on the analysis results of the AI ​​model. If the analysis detects suspicious behavior, the server generates the necessary commands to take immediate action. These commands include instructions to generate an alarm sound.

[0605] Step 5:

[0606] Upon receiving instructions from the server, the terminal emits a warning sound through a speaker at the site. The warning sound is generated based on pre-configured parameters, and its volume and duration are adjusted accordingly. This warning sound prompts the suspicious person to leave the area.

[0607] Step 6:

[0608] Simultaneously, the server notifies the user of the detection results. Details of the suspicious person's movements, as well as information about the time and location of the alert, are sent to the user's smartphone or other mobile device. The user can then receive the notification and quickly consider how to respond.

[0609] Step 7:

[0610] Based on the notification, users can take countermeasures such as contacting the police or further utilizing the system. If necessary, user feedback is sent to the server and used as data to improve the accuracy of the AI ​​model.

[0611] (Example 1)

[0612] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0613] In recent years, the importance of crime prevention has been increasing in society. However, conventional surveillance systems have difficulty detecting suspicious behavior in real time and taking appropriate action immediately. Therefore, there is a need for a crime prevention system that can efficiently and quickly identify suspicious individuals and issue warnings.

[0614] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0615] In this invention, the server includes an analysis device equipped with an artificial intelligence model for receiving video information acquired from an image acquisition device and analyzing a person's behavior, posture, clothing, and belongings based on the video information; an audio warning device that recognizes suspicious behavior from the analysis results of a person's behavior or appearance by the analysis device and generates a command to send a warning signal to an audio output device to emit a warning sound; and an information notification device for notifying the user of the analysis results of suspicious behavior or appearance by the analysis device via a notification device. This enables real-time detection of suspicious persons, rapid warning issuance, and information notification.

[0616] An "image acquisition device" is a device for acquiring video information of the surroundings and has the function of monitoring a certain range.

[0617] An "analysis device" is a device that analyzes the movement, appearance, and shape of people and objects based on acquired video information, and has the ability to recognize a variety of patterns.

[0618] An "artificial intelligence model" is a program used to analyze video information and identify specific behaviors or characteristics, and it applies machine learning and deep learning technologies.

[0619] An "audio warning device" is a device that emits a warning sound when it receives a specific signal, and has the function of alerting suspicious individuals.

[0620] A "notification device" is a device used to communicate analysis results to the user and has the function of transmitting information via a communication network.

[0621] A "user terminal" is the device on the user's side that ultimately receives the information, and can take various forms, such as a smartphone or a computer.

[0622] This invention provides a security system that can detect suspicious behavior or appearance in real time and respond immediately. This system mainly consists of three elements: a terminal, a server, and a user.

[0623] The terminal used is a camera, which is a video acquisition device. This camera can capture clear images day and night and is equipped with high resolution and an infrared sensor. The terminal continuously collects video data and sends it to the server. This transmission is performed in real time over the network.

[0624] The server receives video information transmitted from the terminal. An artificial intelligence model is used to analyze the video. Specifically, a generative AI model is used to analyze the actions, postures, clothing, and belongings of people in the video data. This AI model uses advanced image recognition technology and is excellent at recognizing patterns and detecting abnormal movements. If suspicious behavior is detected as a result of the analysis, the server sends a warning signal to an audio warning device. This signal immediately emits a warning sound at the site, alerting the suspicious person to be vigilant.

[0625] Furthermore, the server notifies the user of the analysis results and the fact of the warning. The notification is sent to the user's terminal via a notification device. User terminals such as smartphones and computers can receive detailed information of the analysis results in real time. The notification includes the characteristics of the suspicious person, video snapshots, and information on the time and location of detection.

[0626] As a concrete example, imagine a situation where a suspicious person is active in a garden at night. In this case, the device captures the person's image and sends the data to a server. The server uses an AI model to analyze the person's actions, and if suspicious activity is detected, it immediately issues a warning. Simultaneously, the user receives a prompt message such as, "A suspicious person has been detected in your garden. Please check their actions." This allows the user to understand the situation and take immediate action.

[0627] This system significantly improves home security, enabling real-time detection of intruders and rapid response.

[0628] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0629] Step 1:

[0630] The device uses a camera, an image acquisition device, to acquire surrounding video data in real time. In this step, the camera uses high resolution and an infrared sensor to capture clear video data 24 hours a day. The input is the surrounding visual information, and the output is the captured video data. This video data is transmitted directly to the next step.

[0631] Step 2:

[0632] The server receives video data transmitted from the terminal. Using this data as input, an AI model within the server performs video analysis. The data processing involves recognizing the movement, posture, clothing, and possessions of a person in each frame of the video and analyzing their characteristics. The output is the analysis results for each frame. Specifically, the AI ​​model tracks the person's movement and detects abnormal patterns.

[0633] Step 3:

[0634] Based on the analysis results obtained in step 2, the server evaluates the behavior and appearance to determine whether or not suspicious behavior is present. The input is the analyzed behavioral data, and the data calculation involves comparing whether the person's movements and behavioral patterns fall within the normal range. The output is a determination of whether or not suspicious behavior was detected.

[0635] Step 4:

[0636] If the server detects suspicious activity, it sends a command to the audio warning device. This command instructs the terminal's warning system to emit a warning sound. The input is the result of the suspicious activity detection, and the output is the actual sound emitted. Specifically, the audio warning device emits a loud warning sound to draw attention to the area.

[0637] Step 5:

[0638] The server transmits the analysis results and warning information to the user via an information notification device. The input is the judgment result and the generated warning information, and the output is a notification sent to the user's terminal. Specifically, this includes the display of information such as video snapshots, detection time, and location on a smartphone or computer.

[0639] Step 6:

[0640] After the user receives a notification, they take the necessary action based on its content. The input is notification information from the server, and the output is a specific action taken by the user (e.g., contacting the police). Specifically, the user checks the details through the application and considers a course of action.

[0641] (Application Example 1)

[0642] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0643] In modern society, criminal activity and illegal activities by suspicious individuals have become a serious problem. However, conventional security systems have limited ability to detect suspicious movements and individuals in real time, making rapid response difficult. Identifying suspicious individuals and responding immediately, especially in homes and businesses, is a crucial issue directly linked to improving safety.

[0644] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0645] In this invention, the server includes an analysis means for receiving video data and identifying suspicious individuals, an audio alarm means for issuing commands to generate audio alarms, an information notification means for transmitting information to an information device, and a data communication means for providing video snapshots to user devices. This enables real-time detection of suspicious behavior or appearances, rapid alarm activation and immediate notification to users, and provision of video snapshots, thereby facilitating a quick response and improving security.

[0646] "Suspicious behavior or appearance" refers to any state that deviates from normal behavior or appearance and is judged to potentially threaten safety.

[0647] An "intelligent model" is an algorithm that uses technologies such as machine learning and deep learning to learn patterns from data and identify suspicious individuals.

[0648] A "processing unit" is a computing device used to process data and perform specified tasks.

[0649] "Video data" refers to the digital format of visual information captured by a camera or other recording device.

[0650] "Analysis methods" refer to mechanisms that process data received to identify suspicious individuals.

[0651] An "audio alarm system" is a device or system that generates and emits an audible warning when a suspicious person is detected.

[0652] An "information device" is a machine equipped with communication and display functions to enable notifications and data transfer.

[0653] "Data communication means" refers to communication technology used to transmit video and analysis results to user devices.

[0654] The present invention is specifically implemented as a security system that enables the detection of suspicious individuals and immediate response. The operation of this system, the hardware and software used, and specific embodiments are described below.

[0655] The server is equipped with an analysis system for image processing, which receives video data transmitted from surveillance cameras in real time. This data is in a high-resolution format that allows for nighttime recording. The server uses software libraries such as Python and TensorFlow as intelligent models to perform data analysis to detect suspicious behavior or appearances.

[0656] Image analysis is performed, and if suspicious behavior or appearance is detected, the server will issue an alarm using audio alarm means. This uses hardware devices such as digital speakers and alarm devices. Furthermore, an information notification means will send a notification containing information about the suspicious person to the user's individual device, such as a smartphone or tablet. This notification will include real-time video snapshots and detailed information about the detected suspicious person.

[0657] When users receive these notifications, they can view video snapshots and past data within the application. The application also includes additional features for directly contacting the police or security companies if necessary.

[0658] A concrete example is when a user detects suspicious activity in their yard while doing their daily shopping. The user checks the notification on their smartphone, views the security camera footage, and confirms the anomaly. This allows them to quickly contact their neighbors and take steps to ensure their safety.

[0659] An example of a prompt might be: "Use the video feed from the camera installed in the garden to monitor for suspicious activity or individuals in real time and immediately notify the user's smartphone. If necessary, allow the user to access police contact information." This type of prompt appropriately provides the necessary instructions to the generating AI model.

[0660] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0661] Step 1:

[0662] The terminal continuously acquires video data from the installed camera. The input is real-time high-resolution video, which is captured and transmitted to the server. In operation, the terminal's video capture function works, and data is collected even at night using an infrared sensor.

[0663] Step 2:

[0664] The server receives video data transmitted from the terminal. The input is real-time video data, which is processed by an intelligent model (using Python and TensorFlow). Here, data analysis is performed, and movement and appearance within the video frames are evaluated to detect suspicious behavior or appearance.

[0665] Step 3:

[0666] The server activates an audio alarm system if suspicious activity is detected. The input is the result of the intelligent model's determination of suspicious activity; based on this, it creates an alarm and outputs a command. Specifically, it uses a digital speaker to emit an alarm sound at the location.

[0667] Step 4:

[0668] The server uses an information notification system to send information about suspicious individuals and video snapshots to the user's device. The input is the result of the suspicious activity detection and the snapshot data, and the output is a notification containing this information. This operation involves rapidly distributing the data over a communication network.

[0669] Step 5:

[0670] The user checks the received notification on their smartphone or tablet and views the video snapshot. The input is the notification data from the server, and the output is the displayed snapshot and details of the suspicious person. The user reviews the video to determine whether they need to contact the police or security services.

[0671] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0672] The security system according to the present invention not only detects suspicious individuals but also analyzes the user's emotions, enabling a more flexible and human-like response. The system consists of a terminal, a server, and a user, and its functionality is enhanced by incorporating an emotion engine.

[0673] In addition to conventional security camera functions, the device is equipped with audio input devices such as a microphone. This allows the device to collect ambient audio data and provide information for use in analyzing the user's emotions. Audio and video data are transmitted to the server in real time.

[0674] The server receives video and audio data transmitted from the terminal. Based on this, an AI model detects suspicious individuals, while an emotion engine analyzes the user's emotional state. The emotion engine analyzes the tone of voice, speaking style, and facial expressions that can be read from the video, and evaluates the user's mental state based on the results. The server combines the emotion analysis results with the AI ​​model's results to optimize the content of the audio warning for detected suspicious individuals and the subsequent notification method.

[0675] A concrete example would be a situation where a suspicious person is loitering around the front door at night. The device captures video and audio and sends the data to a server. The server detects the suspicious person from the video and analyzes the user's emotions from the audio they speak to the system. For example, if the server detects anxiety from the user's voice, it can use that feedback to adjust the tone of the audio warning to be more intimidating, or add a message to the user's notification suggesting they contact the police.

[0676] On the other hand, after receiving a notification, users can choose a course of action based on the emotion-based advice provided by the system. Providing further information as needed helps improve the system's response accuracy.

[0677] Thus, by introducing the emotion engine in this invention, it becomes possible to respond flexibly to the user's emotions, thereby achieving a higher level of safety assurance.

[0678] The following describes the processing flow.

[0679] Step 1:

[0680] The device captures video using a security camera while simultaneously collecting ambient audio data using a microphone. This data is transmitted to a server in real time. The camera operates 24 hours a day in high definition, and the microphone is set to record natural-sounding audio.

[0681] Step 2:

[0682] The server receives video and audio data transmitted from the terminal. The received data is preprocessed, including noise reduction and resolution optimization, to prepare it for analysis by AI models and emotion engines.

[0683] Step 3:

[0684] The server uses an AI model to analyze video data and detect suspicious individuals. Specifically, it analyzes a person's movements, clothing, and belongings, and compares them with past data to determine if there is anything unusual. Simultaneously, it inputs audio data into an emotion engine to analyze the user's emotional state.

[0685] Step 4:

[0686] The server integrates the analysis results and, if a suspicious person is detected, adjusts the output of the voice warning system, taking into account the evaluation of the emotion engine. For example, if the user's emotions indicate anxiety, the warning sound is set to a more alarming tone.

[0687] Step 5:

[0688] The terminal receives instructions from the server and emits an appropriate warning sound from a speaker installed at the site. This warning sound is intended to alert the suspicious person to leave the area, and the volume and tone are dynamically adjusted according to the situation.

[0689] Step 6:

[0690] The server generates and sends notifications to the user based on the analysis results of the suspicious person and the user's sentiment evaluation. These notifications include information about the detected suspicious person, recommended actions based on the user's sentiment, and suggestions to report to the police.

[0691] Step 7:

[0692] Based on the notifications received from the server, users can consider the suggested countermeasures and take action, such as contacting the police, if necessary. They can also send feedback on the notifications to the server, contributing to system improvements.

[0693] (Example 2)

[0694] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0695] Traditional security systems focus on identifying suspicious individuals, but they lack flexibility in considering the user's feelings, resulting in warnings and notifications not being provided in a way that is optimal for the user's situation. As a result, the system may give users an insufficient sense of security.

[0696] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0697] In this invention, the server includes an analysis means for inputting video material and determining a suspicious person, an audio output means for generating a command to emit an audio signal based on the suspicious person determination result, and an emotion analysis means for inputting audio material and evaluating the user's emotional state. This makes it possible to combine the suspicious person detection result and the user's emotion analysis result to provide the user with the most appropriate response, thereby improving safety and security.

[0698] "Suspicious behavior" refers to actions or movements that deviate from normal behavioral patterns and may pose a potential danger.

[0699] An "artificial intelligence model" is a model with a program structure that learns from large amounts of data and can identify specific patterns or anomalies.

[0700] A "computer" is an electronic device used for inputting, processing, and outputting data.

[0701] "Visual materials" refers to visual information data collected by cameras or other video acquisition devices.

[0702] "Analysis means" refers to the process or device that analyzes input data and derives meaningful information.

[0703] "Audio signals" refer to acoustic information used for warnings and commands, generated based on audio data.

[0704] "Audio output means" refers to a device or mechanism for generating an audio signal based on the analysis results and transmitting it externally.

[0705] A "notification mechanism" is a device or system that conveys specific information to users through sound or visual signals.

[0706] "Information transmission means" refers to means for transmitting judgment results and related information to other devices or users.

[0707] "Emotional analysis means" refers to a process or mechanism that analyzes voice tone, manner of speaking, or facial expressions to evaluate the user's emotional state.

[0708] A "control means" is a process or device that combines results obtained from different data to perform optimal processing or output.

[0709] The security system of this invention consists of terminals, servers, and users, and provides a high level of security through the coordinated functioning of each element.

[0710] The terminal functions as a security camera and collects video information. Furthermore, the terminal is equipped with a microphone and has the capability to collect audio information. The terminal transmits the collected video and audio information to a server in real time. This communication is typically conducted via a wireless or wired network, and the collected data is encrypted to ensure security.

[0711] The server plays a central role in receiving and analyzing data transmitted from terminals. The server utilizes generative AI models to analyze video information and identify individuals exhibiting suspicious behavior. It also uses emotion analysis to evaluate the user's emotional state from audio information. This emotion analysis is based on factors such as voice tone, speed, and pauses to determine the user's psychological state. The analysis results are integrated by the server and used to generate warning signals against suspicious individuals and optimize notification methods.

[0712] As a concrete example, consider a scenario where a suspicious person appears at the front door at night. The device records this situation with video and audio and sends it to the server. The server identifies the suspicious person using a generative AI model and simultaneously analyzes the user's emotions from the audio data. For example, if the analysis indicates that the user is feeling anxious, the server may intensify the warning sound or generate a notification message suggesting that the police be notified.

[0713] The user receives a notification from the server and selects an appropriate response based on the information and suggestions provided. An example of a specific prompt message is: "Analyze the suspicious person at the front door and the user's emotions at that time, and suggest an appropriate voice warning and notification method."

[0714] In this way, by having each element work in conjunction, it is possible to realize a comprehensive security system that not only detects suspicious individuals but also takes into account the user's emotions.

[0715] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0716] Step 1:

[0717] The device collects surrounding video and audio information. Inputs include visual data from the camera and audio data from the microphone. The device records this data in real time and prepares it as data packets for subsequent processing. Simultaneous acquisition of video and audio is crucial in this process.

[0718] Step 2:

[0719] The terminal sends the data collected in Step 1 to the server. This input includes encrypted video and audio data. The output is sent to the server via this secure data transfer. The terminal transfers the data over the network, ensuring that it reaches the server in real time.

[0720] Step 3:

[0721] The server receives data transmitted from the terminal. The server's input is the previously transmitted video and audio data. After confirming its reception, it passes it on to the analysis process. Checksums and other methods are used to ensure that the data is received accurately and completely.

[0722] Step 4:

[0723] The server uses a generative AI model to analyze video data and detect suspicious behavior and unusual patterns. The input data consists of video frames, which are analyzed by the AI ​​model and output results identifying suspicious individuals. These results assess potential threats based on specific movements and appearances.

[0724] Step 5:

[0725] The server analyzes voice data using emotion analysis tools to determine the user's emotional state. Input includes voice tone, speed, and pauses between words. From this, the system obtains an emotional evaluation of the user as output. The system uses this to identify emotions such as anxiety or reassurance.

[0726] Step 6:

[0727] The server integrates the results of identifying suspicious individuals with the user's emotional assessment to generate optimal warning signals and notification content. The input includes the aforementioned analysis results. The output generates adjusted warning sounds and notification messages, which are modified according to the situation. This step takes the user's psychological state into account to ensure a more effective response.

[0728] Step 7:

[0729] The user receives notifications from the server and takes action as needed. The input is a notification message, which the user uses to select the appropriate action. The output could be, for example, reporting to the police or providing additional information to the system. The user's choices help improve the system's future response accuracy.

[0730] (Application Example 2)

[0731] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0732] Conventional security systems are solely designed to detect intruders and lack the flexibility to respond flexibly to the user's mental state and emotional condition. In particular, emotion analysis capabilities were needed to provide appropriate warnings and notifications to users who are feeling anxious.

[0733] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0734] In this invention, the server includes emotion analysis means for analyzing the user's emotions from their voice, optimization means for optimizing the content of the warning or notification method based on the suspicious person determination result by the analysis means, and information notification means for notifying the suspicious person determination result by the analysis means to the notification device. This enables appropriate crime prevention measures according to the user's emotional state.

[0735] An "artificial intelligence model" is a system that uses machine learning techniques to analyze information and detect specific patterns or anomalies.

[0736] A "processing device" is a computer system used for data input, analysis, and output.

[0737] "Analysis means" refers to an algorithm or process for deriving specific information based on input data.

[0738] "Audio warning means" refers to a device or system that has the function of generating and transmitting audio or sound signals to alert of the presence of a suspicious person.

[0739] "Information notification means" refers to a means of communication for transmitting analysis results to the user or other devices.

[0740] "Emotional analysis means" refers to a technology that uses audio or video data to determine an individual's emotional state.

[0741] An "optimization method" is a technique for adjusting the operation and output of a system according to a specific purpose in order to obtain the most effective results.

[0742] The system that realizes this invention consists of a server, a terminal, and a user. The server functions as a processing unit and uses an artificial intelligence model and emotion analysis means to detect suspicious persons and analyze the user's emotional state. Data for suspicious person detection is acquired in real time from the camera and microphone installed in the terminal and transmitted to the server as audio and video information.

[0743] The server uses an artificial intelligence model to detect suspicious individuals based on acquired video information, and simultaneously analyzes the user's emotional state using emotion analysis tools based on audio data. Based on the results of the emotion analysis, the server uses optimization tools to optimize the tone of warning sounds and the content of notifications sent to the user. Optimized warnings and suggestions, based on both the detection of suspicious individuals and the user's emotional state, are then delivered to the user.

[0744] As a concrete example, if a user speaks in an anxious voice at a location where a suspicious person is present at night, the server can analyze their emotions, issue an intimidating voice warning, and quickly suggest contacting a security company. This system enables detailed security responses tailored to the user's emotions, providing a higher level of safety.

[0745] An example of a prompt is, "Explain how this security system provides reassuring notifications to users who are concerned about nighttime security." This prompt prompts the AI ​​model to generate appropriate warnings and suggestions. In this way, flexible responses that take user emotions into consideration are achieved.

[0746] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0747] Step 1:

[0748] The device acquires surrounding video and audio information in real time. This information is obtained using the camera and microphone and sent to the server as input data.

[0749] Step 2:

[0750] The server receives video and audio data transmitted from the terminal. The received video data is analyzed using an artificial intelligence model to determine the presence of a suspicious person. The result of this determination is used as output information regarding the presence or absence of a suspicious person.

[0751] Step 3:

[0752] The server simultaneously analyzes the audio data using emotion analysis tools. It analyzes the tone and speed of the voice from the audio data to determine the user's emotional state. The determined emotional state is output and used in the next optimization step.

[0753] Step 4:

[0754] Based on the results of the suspicious person detection and emotion analysis, the server uses optimization techniques to determine the content of the warning and the notification method. For example, if it is determined that the user is feeling anxious, the warning sound is adjusted to be more intimidating as the output.

[0755] Step 5:

[0756] The server transmits optimized warning content to the terminal via an audio warning system and issues a command to issue an audio warning. At this time, the warning sound and notification content are output to the user.

[0757] Step 6:

[0758] Users receive notifications on their devices. These notifications include emotion-based suggestions and action plans, allowing users to choose how they respond. This feedback is reused within the system to help optimize future features.

[0759] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0760] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0761] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0762] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0763] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0764] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0765] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0766] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0767] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0768] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0769] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0770] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0771] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0772] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0773] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0774] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0775] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0776] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0777] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0778] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0779] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0780] The following is further disclosed regarding the embodiments described above.

[0781] (Claim 1)

[0782] A processing device equipped with an artificial intelligence model for detecting suspicious behavior or appearance, comprising: an analysis means for inputting video information and determining a suspicious person from said video information;

[0783] An audio warning means that generates a command to emit a warning sound based on the results of the analysis means's determination of a suspicious person,

[0784] An information notification means that notifies a notification device of the results of the analysis means in determining the presence of a suspicious person,

[0785] A system that includes this.

[0786] (Claim 2)

[0787] The system according to claim 1, wherein the voice warning means has a function of emitting the warning sound.

[0788] (Claim 3)

[0789] The system according to claim 1, wherein the information notification means has a function to transmit the result of the determination of a suspicious person to a notification device for notifying the user.

[0790] "Example 1"

[0791] (Claim 1)

[0792] An analysis device equipped with an artificial intelligence model for receiving video information acquired from an image acquisition device and analyzing a person's actions, posture, clothing, and possessions based on said video information,

[0793] An audio warning device that recognizes suspicious behavior based on the analysis results of a person's actions or appearance by an analysis device, and generates a command to send a warning signal to an audio output device to emit a warning sound,

[0794] An information notification device for notifying the user via a notification device of the results of an analysis of suspicious behavior or appearance by an analysis device,

[0795] A system that includes this.

[0796] (Claim 2)

[0797] The system according to claim 1, wherein the audio warning device is placed on-site and has the function of emitting a warning sound when it detects suspicious behavior.

[0798] (Claim 3)

[0799] The system according to claim 1, wherein the information notification device has a function to transmit the results of suspicious behavior analysis to a user terminal using a communication device.

[0800] "Application Example 1"

[0801] (Claim 1)

[0802] A computing device equipped with an intelligent model for detecting suspicious behavior or appearance, comprising: an analytical means for receiving video data and determining a suspicious person from said video data;

[0803] An audio alarm means that issues a command to generate an audio alarm based on the results of the analysis means's determination of a suspicious person,

[0804] An information notification means that transmits the results of the analysis means's determination of a suspicious person to an information device,

[0805] A data communication means for providing video snapshots to user devices,

[0806] A system that includes this.

[0807] (Claim 2)

[0808] The system according to claim 1, wherein the voice alarm means has the function of issuing the voice alarm.

[0809] (Claim 3)

[0810] The system according to claim 1, wherein the information notification means has a function to transmit the result of the determination of a suspicious person to a communication device for informing the user.

[0811] "Example 2 of combining an emotion engine"

[0812] (Claim 1)

[0813] A computing device equipped with an artificial intelligence model for detecting suspicious behavior or appearance, comprising: an analysis means for inputting video material and determining a suspicious person from the video material; an audio output means for generating a command to emit an audio signal based on the determination result of the suspicious person; an information transmission means for notifying a notification mechanism of the determination result of the suspicious person by the analysis means; an emotion analysis means for inputting audio material and evaluating the emotional state of the user; and a control means for optimizing the content of a warning signal and notification method by combining the determination result of the suspicious person and the emotion analysis result.

[0814] (Claim 2)

[0815] The system according to claim 1, wherein the audio output means has the function of emitting the audio signal, and the audio signal is adjusted based on the user's emotional state.

[0816] (Claim 3)

[0817] The system according to claim 1, wherein the information transmission means has a function to transmit the results of the determination of a suspicious person and the results of the emotion analysis to a notification mechanism for notifying the user.

[0818] "Application example 2 when combining with an emotional engine"

[0819] (Claim 1)

[0820] A processing device equipped with an artificial intelligence model for detecting suspicious behavior or appearance, comprising: an analysis means for inputting video information and determining a suspicious person from said video information;

[0821] An audio warning means that generates a command to emit a warning sound based on the results of the analysis means's determination of a suspicious person,

[0822] An information notification means that notifies a notification device of the results of the analysis means in determining the presence of a suspicious person,

[0823] An emotion analysis method that analyzes emotions from the user's voice,

[0824] An optimization means that optimizes the content or notification method of a warning based on the analysis results of an emotion analysis means,

[0825] A system that includes this.

[0826] (Claim 2)

[0827] The system according to claim 1, wherein the voice warning means has a function to emit the warning sound, and further has a function to adjust the tone of the warning sound based on the user's emotional state.

[0828] (Claim 3)

[0829] The system according to claim 1, wherein the information notification means has a function to transmit the results of the determination of a suspicious person and the results of the emotion analysis to a notification device for notifying the user. [Explanation of symbols]

[0830] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A processing device equipped with an artificial intelligence model for detecting suspicious behavior or appearance, comprising an analysis means for inputting video information and determining a suspicious person from said video information, An audio warning means that generates a command to emit a warning sound based on the results of the analysis means's determination of a suspicious person, An information notification means that notifies a notification device of the results of the analysis means in determining the presence of a suspicious person, A system that includes this.

2. The system according to claim 1, wherein the voice warning means has a function of emitting the warning sound.

3. The system according to claim 1, wherein the information notification means has a function to transmit the result of the determination of a suspicious person to a notification device for notifying the user.

Images