system

Abstract

We provide the system. [Solution] The system includes a computing device that analyzes video data acquired from an image acquisition device to detect suspicious behavior, and means for issuing an alert when such suspicious behavior is detected. A means of identifying a suspicious person by extracting human facial information from the aforementioned video data and comparing it with pre-registered authorized person information, A means of learning from environmental data and dynamically adjusting the security level based on the learning results, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional security system, since it is specialized in video recording, it is difficult to detect suspicious persons or abnormal behaviors in real time. In addition, unnecessary alerts frequently occur due to false detection, which burdens the administrator. Furthermore, false alarms may occur due to complex environments or the movements of animals and plants, and the security measures are insufficient. It is necessary to solve these problems and realize efficient and effective security measures.

Means for Solving the Problems

[0005] In this invention, a computing device analyzes video data acquired from an image acquisition device to detect suspicious behavior. It also includes a function to issue an alert when suspicious behavior is detected. Furthermore, it extracts facial information from the video data and identifies suspicious individuals by comparing it with pre-registered authorized person information. In addition, it reduces false alarms by learning environmental data and dynamically adjusting the security level based on the results. This resolves conventional problems and provides a more secure security system.

[0006] An "image acquisition device" is a device used to acquire video data in real time.

[0007] "Video data" refers to dynamic or static visual information captured by an image acquisition device.

[0008] A "computing device" is a system consisting of hardware and software that analyzes video data and performs functions such as detecting suspicious behavior and identifying individuals.

[0009] "Suspicious behavior" refers to movements or actions that deviate from normal patterns and are deemed to pose a security risk.

[0010] An "alert" is a message or warning signal that notifies a user or administrator when suspicious activity is detected.

[0011] "Facial information" refers to digital data extracted from video data for recognizing and identifying human faces.

[0012] "Authorized person information" refers to facial information of individuals who have been registered in the system in advance and are recognized as not being suspicious.

[0013] A "suspicious person" refers to an individual whose information does not match the authorized person information or who exhibits suspicious behavior.

[0014] "Environmental data" refers to information about surrounding conditions (e.g., weather, time of day, light intensity, etc.), and is data collected to improve the accuracy of the security system.

[0015] "Security level" refers to the sensitivity and response degree of the security functions of the system, and is adjusted according to environmental conditions and risks.

Brief Description of the Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] The system of the present invention is implemented by a computer program composed of multiple modules. Each module is responsible for a specific function, and together they function as a whole system. A specific embodiment is shown below.

[0038] The server first acquires video data in real time from the image acquisition device. This data is transmitted using a dedicated protocol to ensure security. The AI ​​module on the server analyzes the acquired video data frame by frame and detects suspicious behavior. For example, if a person moves erratically between vehicles in a parking lot, it is compared to a predefined dangerous behavior and recognized as suspicious behavior if it exceeds a threshold.

[0039] The device alerts the user to detected suspicious activity. This notification is sent, for example, via push notification through an application or email, allowing the user to immediately check the situation. Receiving alerts from the device enables the user to respond quickly.

[0040] Furthermore, the server extracts facial information from the video data and compares it with pre-registered authorized person information. If an unregistered person is identified through this process, they are flagged as a suspicious person. For example, in an office building, if a visitor whose face is not registered enters the premises, this system will identify that person as a suspicious person and notify the administrator.

[0041] Furthermore, the server analyzes environmental data and dynamically adjusts security levels according to weather and time of day. For example, under conditions where the risk is higher, such as at night or on weekends, the security level is increased and the detection sensitivity is enhanced. This reduces false alarms while enabling appropriate security responses.

[0042] Finally, the server collects event data from suspicious person detections and automatically generates a report. This report includes detailed information such as the time and location of the incident and the detected behavior. This report is provided to the user via their terminal, allowing administrators to review past events and use the information to improve future security measures.

[0043] Thus, the system of the present invention provides efficient and effective crime prevention measures by enabling real-time detection of suspicious behavior, identification of individuals through facial recognition, and dynamic adjustment of security levels according to the environment.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] The server receives video data from the image acquisition device in real time. The video data is transferred to the server in stream format and communicated over the network using a secure and highly efficient protocol.

[0047] Step 2:

[0048] The server analyzes the received video data frame by frame using an AI module. The AI ​​module evaluates the speed and direction of movement based on predefined suspicious behavior patterns and identifies suspicious behavior in real time.

[0049] Step 3:

[0050] The server immediately generates an alert if suspicious activity is detected. The alert includes the time of detection, a detailed description of the activity, and the scope of its impact. This information is recorded in the log.

[0051] Step 4:

[0052] The device receives alerts from the server and notifies the user. Notification methods include push notifications through the device's application and email notifications. Upon receiving a notification, the user can quickly check the situation.

[0053] Step 5:

[0054] The server extracts facial information from video data and compares it with pre-registered information on authorized individuals. If an unregistered person is detected during this matching process, they are marked as a suspicious person.

[0055] Step 6:

[0056] The server collects environmental data (e.g., weather, date and time, ambient light levels) and uses machine learning models to dynamically adjust security levels. Under certain conditions, it increases alarm sensitivity and optimizes the system to minimize false alarms.

[0057] Step 7:

[0058] The server collects detailed data on suspicious person detection and other alarm events. Based on this data, it automatically generates reports that administrators can use to analyze past events. These reports are then provided to users via their terminals.

[0059] (Example 1)

[0060] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0061] In security systems, it is necessary to quickly and accurately detect suspicious behavior and individuals, minimizing false alarms and enabling efficient responses. Conventional systems suffer from problems such as false alarms and excessive alerts, especially under complex environmental conditions, and these issues need to be resolved.

[0062] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0063] In this invention, the server includes means having an information processing device that analyzes visual data acquired from an image acquisition device to detect suspicious behavior; means extracting facial information from the visual data and comparing it with pre-registered authorized personal information to identify a suspicious person; and means analyzing external environmental information and dynamically adjusting the security level based on the analysis results to detect suspicious behavior with increased sensitivity. This enables the rapid identification of suspicious behavior and suspicious individuals, reduces false alarms through adjustment of security levels according to the environment, and allows for efficient crime prevention response.

[0064] "Image acquisition equipment" refers to devices used to acquire visual data, and includes equipment such as cameras and sensors.

[0065] "Visual data" refers to digital information acquired as images or videos, and is the data that is subject to analysis.

[0066] An "information processing device" refers to computing equipment such as computers and servers, which are devices that perform data analysis and processing.

[0067] "Suspicious behavior" refers to movements or actions that deviate from normal behavioral patterns and are judged to be dangerous or suspicious.

[0068] "Facial information" refers to distinctive digital data about a person's face, and is the information used for personal identification through facial recognition.

[0069] "Authorized personal information" refers to information about individuals whose access and actions have been approved and who have been registered in advance.

[0070] "Verification" refers to the process of comparing acquired data with registered data to check for matches or discrepancies.

[0071] A "suspicious person" refers to a person who is identified as not being registered or authenticated.

[0072] "External environmental information" refers to data related to the system's external conditions, such as weather, time of day, and illuminance.

[0073] "Safety level" refers to an indicator that shows the degree of vigilance and response to suspicious behavior or individuals.

[0074] "Dynamic adjustment" refers to the process of changing system settings and operation in real time according to the situation and conditions.

[0075] A "false alarm" refers to a situation where the system mistakenly issues an alert even though there is no actual suspicious activity or person present.

[0076] The system in this invention efficiently realizes security detection by combining various hardware and software. First, the server acquires visual data in real time from a camera, which is an image acquisition device. This visual data is securely transferred using a dedicated protocol and analyzed by an information processing device.

[0077] The server analyzes this data frame by frame using an AI module to detect suspicious behavior. The AI ​​module uses a generative AI model to evaluate movement and behavioral patterns within the visual data and compare them to predefined suspicious behaviors. For example, in a parking lot, if a person moving irregularly between vehicles is detected, it is recognized as suspicious behavior.

[0078] Furthermore, the server extracts facial information from visual data and compares it with pre-registered, authorized personal information. If an unregistered person is detected, they are flagged as a suspicious individual. In office building security, this allows for immediate notification to administrators of visitors not registered in the facial recognition system.

[0079] The server also incorporates external environmental information and dynamically adjusts the security level according to weather, time of day, and other factors. For example, detection sensitivity is increased at night or during periods of higher risk. This operation minimizes false alarms and enables more appropriate security responses.

[0080] The device generates alerts regarding suspicious behavior or individuals detected by the server and notifies the user. Notifications are sent via smartphone apps or email, allowing users to quickly check the situation and take appropriate action.

[0081] An example of a prompt message is: "Identify suspicious behavior in the parking lot, and prioritize notifying you of individuals who move irregularly between vehicles multiple times." Based on this prompt, the AI ​​operates efficiently, highlighting and detecting anomalies that need to be found.

[0082] In this way, the system of the present invention can provide efficient and effective crime prevention measures by integrating real-time detection of suspicious behavior, identification of individuals through facial recognition, and dynamic adjustment of security levels according to the environment.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] The server acquires visual data in real time from the camera, which is an image acquisition device. The input is the camera's video stream, which is received by the server in a secure, proprietary protocol. The output is digital video data ready for analysis.

[0086] Step 2:

[0087] The server analyzes the acquired visual data frame by frame using an AI module to detect suspicious behavior. The input is the video data acquired in step 1, and the movement and patterns of the data are analyzed using a generating AI model. The output is warning information about the detected abnormal behavior. Specifically, a person moving irregularly between vehicles in a parking lot is identified, and suspicious behavior is recognized by comparing it with predefined dangerous behaviors.

[0088] Step 3:

[0089] The server extracts facial information from visual data and compares it with registered, authorized personal information. The input is frame-by-frame facial data. By matching this with personal information in the database, a list of unregistered individuals is generated as output. Specifically, this is the process by which visitors to an office building are identified as suspicious individuals.

[0090] Step 4:

[0091] The server takes in external environmental information and dynamically adjusts the security level. Environmental data such as time of day and weather are used as input. Based on this data, security settings under risk conditions are re-evaluated, and the adjusted security level is applied as output. Specifically, it reduces false alarms by increasing detection sensitivity at night.

[0092] Step 5:

[0093] The terminal generates alerts for suspicious activity detected by the server and notifies the user. The input is warning information from the server, which is sent to the terminal via push notification or email. The output provides the user with details of the suspicious activity, prompting a quick response. Specifically, it supports administrators in immediately accessing the warning received on their smartphone to check the situation on-site.

[0094] (Application Example 1)

[0095] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0096] Ensuring the safety of homes and their surroundings requires a system that can immediately detect and notify of suspicious behavior and visitors. However, conventional surveillance systems often fail to adapt flexibly to environmental changes, leading to false alarms and missed alerts. Furthermore, adjusting security levels to suit specific situations and accurately identifying unregistered visitors are difficult. This presents a challenge in effectively managing home security.

[0097] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0098] In this invention, the server includes means for analyzing video data collected from an image acquisition device to detect abnormal behavior, means for comparing facial information extracted from the video data with registered person information, and means for learning environmental data to dynamically adjust the security level. This enables rapid detection of abnormalities inside and outside the home and immediate notification to the user. This system achieves more accurate security management by pre-registering visitor information and sending an abnormality notification when an unregistered visitor arrives. Furthermore, by automatically adjusting security sensitivity according to changes in time and weather, it becomes possible to take flexible measures according to each situation while reducing false alarms.

[0099] An "image acquisition device" is a device used to capture video data and plays a role in collecting real-time information about the monitored area.

[0100] An "information processing device" is a device that analyzes collected video data and performs processing to detect abnormalities or suspicious behavior.

[0101] A "warning" refers to an alert message sent to the user when abnormal behavior is detected, serving as a means of immediate alerting.

[0102] "Human facial information" refers to facial images and feature data extracted from video data to identify individuals.

[0103] "Comparison" is the process of matching extracted facial information with pre-registered person information to determine whether or not they match.

[0104] An "abnormal person" refers to an individual whose registered information does not match their behavior and who exhibits unexpected actions.

[0105] "Safety level" refers to a standard value that adjusts the sensitivity of abnormal behavior detection and the intensity of response measures, and it changes depending on the situation.

[0106] "Dynamic adjustment" refers to the process of automatically changing settings in response to changes in environmental data and other factors.

[0107] "Managing safety within and around the home" refers to activities that involve tracking and responding to abnormal behavior in order to protect the inside and surrounding areas of the home.

[0108] "Visitor information" refers to data about people who visit a home, which is registered in the system in advance.

[0109] An "automatically generated report" is a document created based on data collected from detected events, intended to record system operation and abnormal incidents.

[0110] "Reducing false alarms" means reducing incorrect warning messages and irrelevant data to improve accuracy.

[0111] A "prompt message" refers to a sentence that is input to an AI model to give instructions and generate output.

[0112] To realize this invention, software consisting of multiple modules is required. The server plays a central role, acquiring video data in real time from cameras installed in and around the home. Standard home surveillance cameras can be used for this purpose. Suspicious activity is detected by using image analysis software such as TENSORFLOW® or OpenCV for processing on the server.

[0113] The server analyzes the acquired video data frame by frame to determine if the movement and behavior patterns match predefined criteria for "abnormal individuals" or "suspicious behavior." For example, if a suspicious person is loitering around a house late at night, the server can detect it. Upon detection, a push notification is immediately sent to the device to alert the user.

[0114] Furthermore, the server extracts facial information and compares it with visitor information previously registered by the user. This automatically identifies visits from unregistered individuals as anomalies. In addition, by using cloud services such as AWS® Lambda, it is possible to dynamically adjust the "security level" based on environmental data (e.g., time of day and weather). This allows for increased security sensitivity at night, for example, and improves the probability of detecting anomalies.

[0115] As a concrete example, even when a homeowner is away from home, this system can be used to receive notifications if a suspicious person visits, allowing them to confirm the safety of their property. Furthermore, the reports generated using the AI ​​model allow for consideration of future crime prevention measures based on past data.

[0116] Examples of specific prompts for the generated AI model include, "Please suggest effective countermeasures for the abnormal behavior detected around my home." In this way, this system provides concrete means to achieve advanced security management.

[0117] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0118] Step 1:

[0119] The server acquires video data in real time from cameras installed in or around homes. The input is a video stream from the cameras, and the server receives and stores this data, resulting in analyzable video data as output. This video data is stored on the server via a secure protocol.

[0120] Step 2:

[0121] The server divides the acquired video data into frames and performs image analysis using TensorFlow and OpenCV. The input is the framed video data stored on the server. Specifically, it performs motion analysis and behavioral pattern recognition, and sets a flag if suspicious behavior is detected. The output is the result of the determination of whether or not suspicious behavior was detected.

[0122] Step 3:

[0123] If suspicious activity is detected, the server sends a push notification to the device. The input is the result of the detected suspicious activity, and the output is a warning notification to the user's device. This notification is sent immediately, allowing the user to check the situation on their smartphone or computer.

[0124] Step 4:

[0125] The server extracts facial information from video data and compares it with visitor information previously registered by the user. The input is facial feature data extracted for each frame, and the output is the matching result. If an unregistered person visits, the server detects the anomaly and records it as a flag.

[0126] Step 5:

[0127] The server uses cloud services such as AWS Lambda to collect environmental data (e.g., time of day and weather) and dynamically adjust the security level. The input is environmental data, and the output is the adjusted security level. During nighttime or inclement weather, the system switches to high-sensitivity mode. This process reduces false alarms and enables the provision of appropriate countermeasures.

[0128] Step 6:

[0129] The user receives an automatically generated report based on detection results and environmental data. The input is data collected from the server, and the output is a report displayed on the user's terminal. This report includes past anomaly detection events and suggestions for future security measures. A specific example is "the detection time and response when a person moving around in the backyard late at night was detected."

[0130] Step 7:

[0131] The server uses a generative AI model to perform analysis and provide information in response to user prompts. The input is a prompt such as, "Please suggest effective countermeasures for the abnormal behavior detected around my home," and the output is a suggested result generated by the AI. This allows users to take crime prevention measures with high accuracy.

[0132] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0133] This invention relates to a security system comprising an image acquisition device, an AI module, an emotion engine, and related algorithms. Specific embodiments of this invention are described below.

[0134] The server first receives video data acquired from the image acquisition device in real time. This data includes people and their movements within the monitoring area. The server uses an AI module to analyze this data and detect suspicious behavior and the faces of unauthorized individuals.

[0135] Furthermore, the server uses an emotion engine to analyze the user's facial expressions from the video data and recognize their emotional state. This emotion recognition process makes it possible to determine whether the user is showing anxiety, surprise, or other emotional states. For example, if a visitor shows an anxious expression as they pass through a security gate in the lobby of an office building, that emotional state is recognized and considered as one of the factors to adjust the level of alert.

[0136] The terminal, following instructions from the server, notifies the user of alerts that include emotional states assessed by the emotion engine. These alerts are tailored to the situation and, if necessary, help administrators and security personnel respond immediately on-site. Users can refer to the alerts provided by the terminal to assess the urgency and take appropriate action quickly.

[0137] Furthermore, the server utilizes accumulated sentiment data and historical environmental data, analyzing this information using machine learning techniques. This analysis reduces false alarms, predicts potential security risks, and optimizes the overall system operation.

[0138] Furthermore, automated reports generated based on past suspicious person detection events and sentiment data are regularly provided to users. This allows administrators and security personnel to understand past situations and use this information to develop more effective security measures.

[0139] Thus, the present invention is a system that improves overall crime prevention effectiveness by combining an emotion engine with conventional security systems to provide more accurate person recognition and situational response capabilities.

[0140] The following describes the processing flow.

[0141] Step 1:

[0142] The server receives video data in real time from the image acquisition device. The acquired video data is transferred to the server via essential security protocols.

[0143] Step 2:

[0144] The server uses an AI module to analyze video data and execute a motion detection algorithm. It analyzes movement patterns and speeds to evaluate whether there is any suspicious behavior or abnormal movement.

[0145] Step 3:

[0146] The server uses a facial recognition algorithm to extract human faces from video data. The extracted facial information is then compared against already registered authorized person information. If an unregistered person is identified, they are marked as a suspicious person.

[0147] Step 4:

[0148] The server uses an emotion engine to analyze the user's emotional state from facial information. Emotional states include joy, surprise, anger, and anxiety, and the recognition results for each are recorded in the system.

[0149] Step 5:

[0150] The server combines information on recognized emotional states and suspicious behavior to determine the priority of alarms. For example, if a person exhibiting anxious emotions is identified, the alarm priority is increased, and it is determined that immediate action is required.

[0151] Step 6:

[0152] The device receives notifications from the server and provides alerts and related information to the user. The user can receive notifications through the device's application and immediately check the content.

[0153] Step 7:

[0154] Users evaluate the situation on-site based on the alarm information provided via their devices and take necessary security measures. This may include promptly notifying administrators or security staff.

[0155] Step 8:

[0156] The server uses accumulated sentiment data and historical environmental data to perform machine learning. This reduces the frequency of false alarms and predicts future security risks.

[0157] Step 9:

[0158] The server automatically generates reports based on these analysis results and sends them to users and administrators periodically. These reports serve as important resources for improving security measures.

[0159] (Example 2)

[0160] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0161] Traditional security systems rely solely on detecting suspicious behavior and identifying unauthorized individuals, which leads to a high rate of false alarms and an inability to consider emotional shifts. Furthermore, the effectiveness of adjusting security levels and conducting post-incident analyses is limited, making rapid and accurate responses difficult.

[0162] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0163] In this invention, the server includes means having a data processing device that analyzes video data acquired from an image acquisition device to detect suspicious behavior; means analyzing the emotional state of a person based on the video data and dynamically adjusting the level of attention based on an assessment of urgency; and means performing analysis using accumulated emotional data and past situational data to reduce false alarms and predict potential risks. This enables more accurate detection of suspicious individuals and adjustment of the security level appropriately based on emotion recognition.

[0164] An "image acquisition device" is a device used to acquire video data within a surveillance area, and includes cameras and sensors.

[0165] A "data processing device" is a device that analyzes received video data to detect suspicious behavior and extract facial information of individuals.

[0166] "Suspicious behavior" refers to actions or behaviors that deviate from normal behavioral patterns and may indicate a security risk.

[0167] "Warning information" refers to notifications and alerts issued when suspicious behavior or individuals are detected.

[0168] "A person's emotional state" refers to data that represents a person's emotional response, obtained by analyzing facial expressions extracted from video data.

[0169] "Assessing urgency" is the process of determining the immediacy and necessity of a response based on information about detected suspicious behavior and emotional states.

[0170] "Attention level" is an indicator that shows the level of risk recognized by the system, and it is dynamically adjusted.

[0171] "Accumulated emotional data" refers to a collection of data on emotional states collected in the past, which is used for analyzing and predicting security systems.

[0172] "Situational data" refers to historical information about the environment and events, which is used for analysis and machine learning.

[0173] "Reducing false alarms" refers to a series of processes undertaken to minimize the occurrence of incorrect warnings and false detections.

[0174] "Predicting potential risks" is the process of using accumulated data to detect potential dangers that may occur in the future.

[0175] This invention is an advanced system for improving security in a surveillance area, and consists of an image acquisition device, a data processing device, an emotion recognition engine, and a server. The server first receives video data in real time from an image acquisition device such as a surveillance camera. The hardware used includes a network-connected high-resolution camera.

[0176] The server transfers the received video data to a data processing unit, where data analysis is performed using an AI module. This AI module utilizes common machine learning frameworks such as TensorFlow and PyTorch. This enables the detection of suspicious behavior and the identification of unauthorized individuals by matching their faces against a database.

[0177] Furthermore, the server uses an emotion recognition engine to analyze a person's emotional state. This emotion recognition engine utilizes common emotion analysis tools such as Face API or similar technologies. The emotional data derived from facial expressions is used to adjust security levels and set alert levels. For example, if a visitor in an office building lobby displays an anxious expression, the system recognizes their emotional state and issues an alert.

[0178] The device receives warning information from the server and notifies the user of an alert. This alert includes detailed information about the emotional state and suspicious behavior. Based on this information, the user assesses the urgency of the situation and takes appropriate action immediately. For example, a message such as "Anxious facial expression detected: On-site verification recommended" may be displayed on the device screen.

[0179] The server also uses accumulated sentiment data and historical situational data, employing machine learning techniques to perform a comprehensive analysis. This allows for a reduction in false alarms, prediction of potential risks, and optimization of the overall system operation. An example of a prompt using a generative AI model is, "Please provide the sentiment data analysis results based on yesterday's suspicious person detection event." Such prompts allow the server to generate a detailed report, which can be used to develop future security measures based on historical data.

[0180] Thus, the present invention combines data analysis and emotion recognition to provide advanced security measures and enable the construction of an efficient crime prevention system.

[0181] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0182] Step 1:

[0183] The server receives video data in real time from the image acquisition device. The input is video data from the surveillance camera, and the output is a raw video stream. Specifically, it uses a network protocol to receive the video signal and prevents data loss by buffering it.

[0184] Step 2:

[0185] The server passes the received video data to the AI ​​module to detect suspicious behavior. The input for this step is the video data obtained in step 1, and the output is identification information indicating suspicious behavior. Specifically, the AI ​​module uses a built-in machine learning algorithm (e.g., an object detection model using TensorFlow) to analyze the movement in the video and identify suspicious patterns.

[0186] Step 3:

[0187] The server further processes the output data from the AI ​​module, extracting facial information and matching it against the allow list. The input for this step is video data and identification information of suspicious behavior, and the output is facial information of unauthorized individuals. Specifically, it uses a facial recognition framework (e.g., OpenCV) to analyze facial images and compare them with a pre-registered database to identify non-matching faces.

[0188] Step 4:

[0189] The server analyzes facial expressions using an emotion engine and recognizes the emotional state. The input for this step is facial information, and the output is the recognized emotional state (e.g., anxiety, surprise). Specifically, it uses an emotion recognition library to analyze subtle facial movements and facial feature quantities to classify emotions.

[0190] Step 5:

[0191] The terminal notifies the user of an alert based on identification information and emotional state of suspicious behavior sent from the server. The input for this step is the identification information and emotional state from the server, and the output is the alert message displayed to the user. Specifically, the terminal screen displays text such as "Anxious facial expression detected: Site verification recommended."

[0192] Step 6:

[0193] The user assesses the urgency based on the device's alerts and takes the necessary actions. The input for this step is the device's alert message, and the output is the on-site response based on the user's actions. Specifically, the user immediately checks the surveillance camera footage and contacts the security personnel.

[0194] Step 7:

[0195] The server analyzes accumulated data to reduce false alarms and predict potential risks. The inputs for this step are sentiment data and historical environmental data, and the output is a security setting adjusted based on predictions. Specifically, it uses machine learning algorithms to update the model based on past patterns.

[0196] In this way, the entire system can function with greater precision and efficiency.

[0197] (Application Example 2)

[0198] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0199] In recent years, the importance of ensuring safety in facilities and event venues has increased. However, conventional security systems only detect people's movements and actions, and cannot grasp changes in emotions or underlying psychological states, which has made it difficult to respond appropriately and quickly.

[0200] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0201] In this invention, the server includes a computing device that analyzes image data acquired from an image acquisition device to detect suspicious behavior, means for issuing a warning when suspicious behavior is detected, means for analyzing emotional states from video data and adjusting the alert level based on the results, and means for notifying a mobile terminal of information based on the suspicious behavior and emotional state. This makes it possible to comprehensively understand a person's behavior and emotional changes and take appropriate and prompt safety measures.

[0202] An "image acquisition device" is a device used to collect image data from a target area.

[0203] "Image data" refers to visual information that records a person, their movements, and their surrounding environment.

[0204] A "computational device" is a combination of hardware and software that performs processing to analyze image data and detect suspicious behavior.

[0205] "Means of issuing warnings" refers to a function that generates alerts and sends notifications in response to detected suspicious behavior.

[0206] "Facial information of a person" refers to data that shows the characteristics of an individual person, extracted from image data.

[0207] "Authorized person information" refers to a database of authorized individuals that have been pre-registered in the system.

[0208] "Environmental information" refers to data that shows the past and present conditions within the monitoring area.

[0209] "Means for dynamically adjusting the security level" refers to a function that changes the system's security settings according to the analysis results.

[0210] "Means for analyzing emotional states and adjusting alert levels based on the results" refers to a process for estimating emotions from a person's facial expressions and adjusting the system's alert level based on those results.

[0211] "Means of notifying mobile devices" refers to a function for sending information about suspicious behavior or emotional states to the user's mobile device.

[0212] The system of the present invention consists of multiple components and aims to enhance security both inside and outside the facility. The server receives image data in real time from image acquisition devices. The hardware used here includes network cameras and other image sensors. The acquired image data is first sent to a processing unit executed on the server, which uses OpenCV to analyze human movement and facial information.

[0213] The AI ​​module uses deep learning frameworks such as TensorFlow to detect suspicious behavior and analyze emotional states from this image data. This AI module recognizes emotions from facial expressions and extracts key emotional indicators such as surprise and anxiety. Based on this, the emotion engine dynamically adjusts the alert level and issues warnings as needed.

[0214] The device sends notifications to the user based on detected suspicious behavior and emotional states through a mobile application built with React Native. This notification feature allows the user to immediately understand the situation and take necessary actions. The notified data is stored in a database using AWS SageMaker and used for analyzing historical environmental information.

[0215] As a concrete example, let's consider an application in a large commercial facility. Cameras installed in various areas of the facility capture visitors' movements and facial expressions, transmitting the data to a server in real time. For example, if many visitors in a certain area suddenly show signs of anxiety, the system immediately analyzes the data and sends an alert to the security team. This allows potential problems to be detected in advance, enabling swift countermeasures to be taken.

[0216] Using a generative AI model, the following text is used as an example of a prompt:

[0217] "We detected an expression of anxiety in venue area 3. Immediate investigation is required."

[0218] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0219] Step 1:

[0220] The server receives image data in real time from the image acquisition device. This data is video of the monitored area and is raw data for analyzing behavior and emotional states. The received image data is transmitted to a computing device for preparation for analysis.

[0221] Step 2:

[0222] The computing device on the server processes the received image data using OpenCV. Specifically, it extracts features from the images to recognize people and their movements, and sends these features to the AI ​​module. The input here is image data, and the output is a dataset containing the features.

[0223] Step 3:

[0224] The AI ​​module utilizes the TensorFlow framework to analyze feature data and detect suspicious behavior and recognize emotional states. The input is feature data sent from the server, and the output is an evaluation of the presence or absence of suspicious behavior and emotional state as a result of the analysis. This allows for the identification of abnormal behaviors and emotions exhibited by individuals.

[0225] Step 4:

[0226] The server dynamically adjusts the alert level based on emotional data analyzed using an emotion engine. The input is the analysis result of the AI ​​module, and the output is the adapted alert level. This adjustment changes the system's behavior to enable immediate responses as needed.

[0227] Step 5:

[0228] The device sends notifications to the user based on suspicious behavior and emotional state through an app developed using React Native. The input is a server-configured alert level and analysis results, and the output displayed to the user is a specific warning or notification message. For example, a prompt message such as "An anxious expression has been detected in venue area 3. Immediate verification is required." might be sent.

[0229] Step 6:

[0230] Users make decisions about on-site responses and take necessary actions based on notifications displayed on their devices. The output to the user is information provided on the device, and a corresponding response is required.

[0231] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0232] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0233] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0234] [Second Embodiment]

[0235] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0236] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0237] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0238] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0239] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0240] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0241] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0242] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0243] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0244] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0245] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0246] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0247] The system of the present invention is implemented by a computer program composed of multiple modules. Each module is responsible for a specific function, and together they function as a whole system. A specific embodiment is shown below.

[0248] The server first acquires video data in real time from the image acquisition device. This data is transmitted using a dedicated protocol to ensure security. The AI ​​module on the server analyzes the acquired video data frame by frame and detects suspicious behavior. For example, if a person moves erratically between vehicles in a parking lot, it is compared to a predefined dangerous behavior and recognized as suspicious behavior if it exceeds a threshold.

[0249] The device alerts the user to detected suspicious activity. This notification is sent, for example, via push notification through an application or email, allowing the user to immediately check the situation. Receiving alerts from the device enables the user to respond quickly.

[0250] Furthermore, the server extracts facial information from the video data and compares it with pre-registered authorized person information. If an unregistered person is identified through this process, they are flagged as a suspicious person. For example, in an office building, if a visitor whose face is not registered enters the premises, this system will identify that person as a suspicious person and notify the administrator.

[0251] Furthermore, the server analyzes environmental data and dynamically adjusts security levels according to weather and time of day. For example, under conditions where the risk is higher, such as at night or on weekends, the security level is increased and the detection sensitivity is enhanced. This reduces false alarms while enabling appropriate security responses.

[0252] Finally, the server collects event data from suspicious person detections and automatically generates a report. This report includes detailed information such as the time and location of the incident and the detected behavior. This report is provided to the user via their terminal, allowing administrators to review past events and use the information to improve future security measures.

[0253] Thus, the system of the present invention provides efficient and effective crime prevention measures by enabling real-time detection of suspicious behavior, identification of individuals through facial recognition, and dynamic adjustment of security levels according to the environment.

[0254] The following describes the processing flow.

[0255] Step 1:

[0256] The server receives video data from the image acquisition device in real time. The video data is transferred to the server in stream format and communicated over the network using a secure and highly efficient protocol.

[0257] Step 2:

[0258] The server analyzes the received video data frame by frame using an AI module. The AI ​​module evaluates the speed and direction of movement based on predefined suspicious behavior patterns and identifies suspicious behavior in real time.

[0259] Step 3:

[0260] The server immediately generates an alert if suspicious activity is detected. The alert includes the time of detection, a detailed description of the activity, and the scope of its impact. This information is recorded in the log.

[0261] Step 4:

[0262] The device receives alerts from the server and notifies the user. Notification methods include push notifications through the device's application and email notifications. Upon receiving a notification, the user can quickly check the situation.

[0263] Step 5:

[0264] The server extracts facial information from video data and compares it with pre-registered information on authorized individuals. If an unregistered person is detected during this matching process, they are marked as a suspicious person.

[0265] Step 6:

[0266] The server collects environmental data (e.g., weather, date and time, ambient light levels) and uses machine learning models to dynamically adjust security levels. Under certain conditions, it increases alarm sensitivity and optimizes the system to minimize false alarms.

[0267] Step 7:

[0268] The server collects detailed data on suspicious person detection and other alarm events. Based on this data, it automatically generates reports that administrators can use to analyze past events. These reports are then provided to users via their terminals.

[0269] (Example 1)

[0270] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0271] In security systems, it is necessary to quickly and accurately detect suspicious behavior and individuals, minimizing false alarms and enabling efficient responses. Conventional systems suffer from problems such as false alarms and excessive alerts, especially under complex environmental conditions, and these issues need to be resolved.

[0272] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0273] In this invention, the server includes means having an information processing device that analyzes visual data acquired from an image acquisition device to detect suspicious behavior; means extracting facial information from the visual data and comparing it with pre-registered authorized personal information to identify a suspicious person; and means analyzing external environmental information and dynamically adjusting the security level based on the analysis results to detect suspicious behavior with increased sensitivity. This enables the rapid identification of suspicious behavior and suspicious individuals, reduces false alarms through adjustment of security levels according to the environment, and allows for efficient crime prevention response.

[0274] "Image acquisition equipment" refers to devices used to acquire visual data, and includes equipment such as cameras and sensors.

[0275] "Visual data" refers to digital information acquired as images or videos, and is the data that is subject to analysis.

[0276] An "information processing device" refers to computing equipment such as computers and servers, which are devices that perform data analysis and processing.

[0277] "Suspicious behavior" refers to movements or actions that deviate from normal behavioral patterns and are judged to be dangerous or suspicious.

[0278] "Facial information" refers to distinctive digital data about a person's face, and is the information used for personal identification through facial recognition.

[0279] "Authorized personal information" refers to information about individuals whose access and actions have been approved and who have been registered in advance.

[0280] "Verification" refers to the process of comparing acquired data with registered data to check for matches or discrepancies.

[0281] A "suspicious person" refers to a person who is identified as not being registered or authenticated.

[0282] "External environmental information" refers to data related to external conditions of the system such as weather, time, illuminance, etc.

[0283] "Safety level" refers to an indicator indicating the degree of vigilance and response to suspicious actions and suspicious persons.

[0284] "Dynamically adjust" refers to the process of changing the settings and operations of the system in real time according to the situation and conditions.

[0285] "False alarm" refers to a state where the system erroneously issues an alert although there are no actual suspicious actions or suspicious persons.

[0286] The system in the present invention combines various hardware and software to efficiently achieve crime detection. First, the server acquires visual data in real time from a camera which is an image acquisition device. This visual data is securely transferred using a dedicated protocol and analyzed by an information processing device.

[0287] The server analyzes this data frame by frame using an AI module to detect suspicious actions. The AI module uses a generated AI model to evaluate the movements and action patterns in the visual data and compare them with predefined suspicious actions. As a specific example, in a parking lot, when a person moving irregularly between vehicles is detected, it is recognized as a suspicious action.

[0288] Also, the server extracts facial information from the visual data and performs a comparison with pre-registered permitted personal information. As a result, if an unregistered person is detected, a flag is set as a suspicious person. In the security of an office building, it is possible to immediately notify the administrator of the entry of a visitor who is not registered in the face recognition system.

[0289] The server also incorporates external environmental information and dynamically adjusts the security level according to weather, time of day, and other factors. For example, detection sensitivity is increased at night or during periods of higher risk. This operation minimizes false alarms and enables more appropriate security responses.

[0290] The device generates alerts regarding suspicious behavior or individuals detected by the server and notifies the user. Notifications are sent via smartphone apps or email, allowing users to quickly check the situation and take appropriate action.

[0291] An example of a prompt message is: "Identify suspicious behavior in the parking lot, and prioritize notifying you of individuals who move irregularly between vehicles multiple times." Based on this prompt, the AI ​​operates efficiently, highlighting and detecting anomalies that need to be found.

[0292] In this way, the system of the present invention can provide efficient and effective crime prevention measures by integrating real-time detection of suspicious behavior, identification of individuals through facial recognition, and dynamic adjustment of security levels according to the environment.

[0293] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0294] Step 1:

[0295] The server acquires visual data in real time from the camera, which is an image acquisition device. The input is the camera's video stream, which is received by the server in a secure, proprietary protocol. The output is digital video data ready for analysis.

[0296] Step 2:

[0297] The server analyzes the acquired visual data frame by frame using an AI module to detect suspicious behavior. The input is the video data acquired in step 1, and the movement and patterns of the data are analyzed using a generating AI model. The output is warning information about the detected abnormal behavior. Specifically, a person moving irregularly between vehicles in a parking lot is identified, and suspicious behavior is recognized by comparing it with predefined dangerous behaviors.

[0298] Step 3:

[0299] The server extracts facial information from visual data and compares it with registered, authorized personal information. The input is frame-by-frame facial data. By matching this with personal information in the database, a list of unregistered individuals is generated as output. Specifically, this is the process by which visitors to an office building are identified as suspicious individuals.

[0300] Step 4:

[0301] The server takes in external environmental information and dynamically adjusts the security level. Environmental data such as time of day and weather are used as input. Based on this data, security settings under risk conditions are re-evaluated, and the adjusted security level is applied as output. Specifically, it reduces false alarms by increasing detection sensitivity at night.

[0302] Step 5:

[0303] The terminal generates alerts for suspicious activity detected by the server and notifies the user. The input is warning information from the server, which is sent to the terminal via push notification or email. The output provides the user with details of the suspicious activity, prompting a quick response. Specifically, it supports administrators in immediately accessing the warning received on their smartphone to check the situation on-site.

[0304] (Application Example 1)

[0305] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0306] In order to ensure the safety of a home and its surroundings, a system that can immediately detect and notify suspicious behavior and visitors is necessary. However, ordinary monitoring systems cannot flexibly respond to environmental changes, and false alarms and omissions often occur. In addition, it is difficult to adjust the security level according to specific situations and accurately identify unregistered visitors. As a result, there is a problem that it is difficult to effectively manage home security.

[0307] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0308] In this invention, the server includes means for analyzing video data collected from an image acquisition device to detect abnormal behavior, means for comparing face information extracted from the video data with registered person information, and means for learning environmental data to dynamically adjust the security level. As a result, abnormalities inside and outside the home can be quickly detected, and immediate notification to the user becomes possible. This system realizes more accurate security management by pre-registering visitor information and sending an abnormal notification when an unregistered visitor arrives. In addition, by automatically adjusting the security sensitivity according to changes in time and weather, flexible countermeasures can be taken according to each situation while reducing false alarms.

[0309] The "image acquisition device" is a device for capturing video data and has the role of collecting the situation of the monitoring target in real time.

[0310] The "information processing device" is a device that analyzes the collected video data and performs processing for detecting abnormalities and suspicious behavior.

[0311] A "warning" refers to an alert message that notifies the user when abnormal behavior is detected and is a means for immediate attention.

[0312] "Human facial information" refers to facial images and feature data extracted from video data to identify individuals.

[0313] "Comparison" is the process of matching extracted facial information with pre-registered person information to determine whether or not they match.

[0314] An "abnormal person" refers to an individual whose registered information does not match their behavior and who exhibits unexpected actions.

[0315] "Safety level" refers to a standard value that adjusts the sensitivity of abnormal behavior detection and the intensity of response measures, and it changes depending on the situation.

[0316] "Dynamic adjustment" refers to the process of automatically changing settings in response to changes in environmental data and other factors.

[0317] "Managing safety within and around the home" refers to activities that involve tracking and responding to abnormal behavior in order to protect the inside and surrounding areas of the home.

[0318] "Visitor information" refers to data about people who visit a home, which is registered in the system in advance.

[0319] An "automatically generated report" is a document created based on data collected from detected events, intended to record system operation and abnormal incidents.

[0320] "Reducing false alarms" means reducing incorrect warning messages and irrelevant data to improve accuracy.

[0321] A "prompt message" refers to a sentence that is input to an AI model to give instructions and generate output.

[0322] To realize this invention, software consisting of multiple modules is required. The server plays a central role, acquiring video data in real time from cameras installed in and around the home. Standard home surveillance cameras can be used for this purpose. Suspicious activity is detected by using image analysis software such as TensorFlow or OpenCV on the server.

[0323] The server analyzes the acquired video data frame by frame to determine if the movement and behavior patterns match predefined criteria for "abnormal individuals" or "suspicious behavior." For example, if a suspicious person is loitering around a house late at night, the server can detect it. Upon detection, a push notification is immediately sent to the device to alert the user.

[0324] Furthermore, the server extracts facial information and compares it with visitor information previously registered by the user. This automatically identifies any visits by unregistered individuals as an anomaly. In addition, by using cloud services such as AWS Lambda, it is possible to dynamically adjust the "security level" based on environmental data (e.g., time of day and weather). This allows for increased security sensitivity at night, for example, to improve the probability of detecting anomalies.

[0325] As a concrete example, even when a homeowner is away from home, this system can be used to receive notifications if a suspicious person visits, allowing them to confirm the safety of their property. Furthermore, the reports generated using the AI ​​model allow for consideration of future crime prevention measures based on past data.

[0326] Examples of specific prompts for the generated AI model include, "Please suggest effective countermeasures for the abnormal behavior detected around my home." In this way, this system provides concrete means to achieve advanced security management.

[0327] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0328] Step 1:

[0329] The server acquires video data in real time from cameras installed in or around homes. The input is a video stream from the cameras, and the server receives and stores this data, resulting in analyzable video data as output. This video data is stored on the server via a secure protocol.

[0330] Step 2:

[0331] The server divides the acquired video data into frames and performs image analysis using TensorFlow and OpenCV. The input is the framed video data stored on the server. Specifically, it performs motion analysis and behavioral pattern recognition, and sets a flag if suspicious behavior is detected. The output is the result of the determination of whether or not suspicious behavior was detected.

[0332] Step 3:

[0333] If suspicious activity is detected, the server sends a push notification to the device. The input is the result of the detected suspicious activity, and the output is a warning notification to the user's device. This notification is sent immediately, allowing the user to check the situation on their smartphone or computer.

[0334] Step 4:

[0335] The server extracts facial information from video data and compares it with visitor information previously registered by the user. The input is facial feature data extracted for each frame, and the output is the matching result. If an unregistered person visits, the server detects the anomaly and records it as a flag.

[0336] Step 5:

[0337] The server uses cloud services such as AWS Lambda to collect environmental data (e.g., time of day and weather) and dynamically adjust the security level. The input is environmental data, and the output is the adjusted security level. During nighttime or inclement weather, the system switches to high-sensitivity mode. This process reduces false alarms and enables the provision of appropriate countermeasures.

[0338] Step 6:

[0339] The user receives an automatically generated report based on detection results and environmental data. The input is data collected from the server, and the output is a report displayed on the user's terminal. This report includes past anomaly detection events and suggestions for future security measures. A specific example is "the detection time and response when a person moving around in the backyard late at night was detected."

[0340] Step 7:

[0341] The server uses a generative AI model to perform analysis and provide information in response to user prompts. The input is a prompt such as, "Please suggest effective countermeasures for the abnormal behavior detected around my home," and the output is a suggested result generated by the AI. This allows users to take crime prevention measures with high accuracy.

[0342] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0343] This invention relates to a security system comprising an image acquisition device, an AI module, an emotion engine, and related algorithms. Specific embodiments of this invention are described below.

[0344] The server first receives video data acquired from the image acquisition device in real time. This data includes people and their movements within the monitoring area. The server uses an AI module to analyze this data and detect suspicious behavior and the faces of unauthorized individuals.

[0345] Furthermore, the server uses an emotion engine to analyze the user's facial expressions from the video data and recognize their emotional state. This emotion recognition process makes it possible to determine whether the user is showing anxiety, surprise, or other emotional states. For example, if a visitor shows an anxious expression as they pass through a security gate in the lobby of an office building, that emotional state is recognized and considered as one of the factors to adjust the level of alert.

[0346] The terminal, following instructions from the server, notifies the user of alerts that include emotional states assessed by the emotion engine. These alerts are tailored to the situation and, if necessary, help administrators and security personnel respond immediately on-site. Users can refer to the alerts provided by the terminal to assess the urgency and take appropriate action quickly.

[0347] Furthermore, the server utilizes accumulated sentiment data and historical environmental data, analyzing this information using machine learning techniques. This analysis reduces false alarms, predicts potential security risks, and optimizes the overall system operation.

[0348] Furthermore, automated reports generated based on past suspicious person detection events and sentiment data are regularly provided to users. This allows administrators and security personnel to understand past situations and use this information to develop more effective security measures.

[0349] Thus, the present invention is a system that improves overall crime prevention effectiveness by combining an emotion engine with conventional security systems to provide more accurate person recognition and situational response capabilities.

[0350] The following describes the processing flow.

[0351] Step 1:

[0352] The server receives video data in real time from the image acquisition device. The acquired video data is transferred to the server via essential security protocols.

[0353] Step 2:

[0354] The server uses an AI module to analyze video data and execute a motion detection algorithm. It analyzes movement patterns and speeds to evaluate whether there is any suspicious behavior or abnormal movement.

[0355] Step 3:

[0356] The server uses a facial recognition algorithm to extract human faces from video data. The extracted facial information is then compared against already registered authorized person information. If an unregistered person is identified, they are marked as a suspicious person.

[0357] Step 4:

[0358] The server uses an emotion engine to analyze the user's emotional state from facial information. Emotional states include joy, surprise, anger, and anxiety, and the recognition results for each are recorded in the system.

[0359] Step 5:

[0360] The server combines information on recognized emotional states and suspicious behavior to determine the priority of alarms. For example, if a person exhibiting anxious emotions is identified, the alarm priority is increased, and it is determined that immediate action is required.

[0361] Step 6:

[0362] The device receives notifications from the server and provides alerts and related information to the user. The user can receive notifications through the device's application and immediately check the content.

[0363] Step 7:

[0364] Users evaluate the situation on-site based on the alarm information provided via their devices and take necessary security measures. This may include promptly notifying administrators or security staff.

[0365] Step 8:

[0366] The server uses accumulated sentiment data and historical environmental data to perform machine learning. This reduces the frequency of false alarms and predicts future security risks.

[0367] Step 9:

[0368] The server automatically generates reports based on these analysis results and sends them to users and administrators periodically. These reports serve as important resources for improving security measures.

[0369] (Example 2)

[0370] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0371] Traditional security systems rely solely on detecting suspicious behavior and identifying unauthorized individuals, which leads to a high rate of false alarms and an inability to consider emotional shifts. Furthermore, the effectiveness of adjusting security levels and conducting post-incident analyses is limited, making rapid and accurate responses difficult.

[0372] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0373] In this invention, the server includes means having a data processing device that analyzes video data acquired from an image acquisition device to detect suspicious behavior; means analyzing the emotional state of a person based on the video data and dynamically adjusting the level of attention based on an assessment of urgency; and means performing analysis using accumulated emotional data and past situational data to reduce false alarms and predict potential risks. This enables more accurate detection of suspicious individuals and adjustment of the security level appropriately based on emotion recognition.

[0374] An "image acquisition device" is a device used to acquire video data within a surveillance area, and includes cameras and sensors.

[0375] A "data processing device" is a device that analyzes received video data to detect suspicious behavior and extract facial information of individuals.

[0376] "Suspicious behavior" refers to actions or behaviors that deviate from normal behavioral patterns and may indicate a security risk.

[0377] "Warning information" refers to notifications and alerts issued when suspicious behavior or individuals are detected.

[0378] "A person's emotional state" refers to data that represents a person's emotional response, obtained by analyzing facial expressions extracted from video data.

[0379] "Assessing urgency" is the process of determining the immediacy and necessity of a response based on information about detected suspicious behavior and emotional states.

[0380] "Attention level" is an indicator that shows the level of risk recognized by the system, and it is dynamically adjusted.

[0381] "Accumulated emotional data" refers to a collection of data on emotional states collected in the past, which is used for analyzing and predicting security systems.

[0382] "Situational data" refers to historical information about the environment and events, which is used for analysis and machine learning.

[0383] "Reducing false alarms" refers to a series of processes undertaken to minimize the occurrence of incorrect warnings and false detections.

[0384] "Predicting potential risks" is the process of using accumulated data to detect potential dangers that may occur in the future.

[0385] This invention is an advanced system for improving security in a surveillance area, and consists of an image acquisition device, a data processing device, an emotion recognition engine, and a server. The server first receives video data in real time from an image acquisition device such as a surveillance camera. The hardware used includes a network-connected high-resolution camera.

[0386] The server transfers the received video data to a data processing unit, where data analysis is performed using an AI module. This AI module utilizes common machine learning frameworks such as TensorFlow and PyTorch. This enables the detection of suspicious behavior and the identification of unauthorized individuals by matching their faces against a database.

[0387] Furthermore, the server uses an emotion recognition engine to analyze a person's emotional state. This emotion recognition engine utilizes common emotion analysis tools such as Face API or similar technologies. The emotional data derived from facial expressions is used to adjust security levels and set alert levels. For example, if a visitor in an office building lobby displays an anxious expression, the system recognizes their emotional state and issues an alert.

[0388] The device receives warning information from the server and notifies the user of an alert. This alert includes detailed information about the emotional state and suspicious behavior. Based on this information, the user assesses the urgency of the situation and takes appropriate action immediately. For example, a message such as "Anxious facial expression detected: On-site verification recommended" may be displayed on the device screen.

[0389] The server also uses accumulated sentiment data and historical situational data, employing machine learning techniques to perform a comprehensive analysis. This allows for a reduction in false alarms, prediction of potential risks, and optimization of the overall system operation. An example of a prompt using a generative AI model is, "Please provide the sentiment data analysis results based on yesterday's suspicious person detection event." Such prompts allow the server to generate a detailed report, which can be used to develop future security measures based on historical data.

[0390] Thus, the present invention combines data analysis and emotion recognition to provide advanced security measures and enable the construction of an efficient crime prevention system.

[0391] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0392] Step 1:

[0393] The server receives video data in real time from the image acquisition device. The input is video data from the surveillance camera, and the output is a raw video stream. Specifically, it uses a network protocol to receive the video signal and prevents data loss by buffering it.

[0394] Step 2:

[0395] The server passes the received video data to the AI ​​module to detect suspicious behavior. The input for this step is the video data obtained in step 1, and the output is identification information indicating suspicious behavior. Specifically, the AI ​​module uses a built-in machine learning algorithm (e.g., an object detection model using TensorFlow) to analyze the movement in the video and identify suspicious patterns.

[0396] Step 3:

[0397] The server further processes the output data from the AI ​​module, extracting facial information and matching it against the allow list. The input for this step is video data and identification information of suspicious behavior, and the output is facial information of unauthorized individuals. Specifically, it uses a facial recognition framework (e.g., OpenCV) to analyze facial images and compare them with a pre-registered database to identify non-matching faces.

[0398] Step 4:

[0399] The server analyzes facial expressions using an emotion engine and recognizes the emotional state. The input for this step is facial information, and the output is the recognized emotional state (e.g., anxiety, surprise). Specifically, it uses an emotion recognition library to analyze subtle facial movements and facial feature quantities to classify emotions.

[0400] Step 5:

[0401] The terminal notifies the user of an alert based on identification information and emotional state of suspicious behavior sent from the server. The input for this step is the identification information and emotional state from the server, and the output is the alert message displayed to the user. Specifically, the terminal screen displays text such as "Anxious facial expression detected: Site verification recommended."

[0402] Step 6:

[0403] The user assesses the urgency based on the device's alerts and takes the necessary actions. The input for this step is the device's alert message, and the output is the on-site response based on the user's actions. Specifically, the user immediately checks the surveillance camera footage and contacts the security personnel.

[0404] Step 7:

[0405] The server analyzes accumulated data to reduce false alarms and predict potential risks. The inputs for this step are sentiment data and historical environmental data, and the output is a security setting adjusted based on predictions. Specifically, it uses machine learning algorithms to update the model based on past patterns.

[0406] In this way, the entire system can function with greater precision and efficiency.

[0407] (Application Example 2)

[0408] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0409] In recent years, the importance of ensuring safety in facilities and event venues has increased. However, conventional security systems only detect people's movements and actions, and cannot grasp changes in emotions or underlying psychological states, which has made it difficult to respond appropriately and quickly.

[0410] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0411] In this invention, the server includes a computing device that analyzes image data acquired from an image acquisition device to detect suspicious behavior, means for issuing a warning when suspicious behavior is detected, means for analyzing emotional states from video data and adjusting the alert level based on the results, and means for notifying a mobile terminal of information based on the suspicious behavior and emotional state. This makes it possible to comprehensively understand a person's behavior and emotional changes and take appropriate and prompt safety measures.

[0412] An "image acquisition device" is a device used to collect image data from a target area.

[0413] "Image data" refers to visual information that records a person, their movements, and their surrounding environment.

[0414] A "computational device" is a combination of hardware and software that performs processing to analyze image data and detect suspicious behavior.

[0415] "Means of issuing warnings" refers to a function that generates alerts and sends notifications in response to detected suspicious behavior.

[0416] "Facial information of a person" refers to data that shows the characteristics of an individual person, extracted from image data.

[0417] "Authorized person information" refers to a database of authorized individuals that have been pre-registered in the system.

[0418] "Environmental information" refers to data that shows the past and present conditions within the monitoring area.

[0419] "Means for dynamically adjusting the security level" refers to a function that changes the system's security settings according to the analysis results.

[0420] "Means for analyzing emotional states and adjusting alert levels based on the results" refers to a process for estimating emotions from a person's facial expressions and adjusting the system's alert level based on those results.

[0421] "Means of notifying mobile devices" refers to a function for sending information about suspicious behavior or emotional states to the user's mobile device.

[0422] The system of the present invention consists of multiple components and aims to enhance security both inside and outside the facility. The server receives image data in real time from image acquisition devices. The hardware used here includes network cameras and other image sensors. The acquired image data is first sent to a processing unit executed on the server, which uses OpenCV to analyze human movement and facial information.

[0423] The AI ​​module uses deep learning frameworks such as TensorFlow to detect suspicious behavior and analyze emotional states from this image data. This AI module recognizes emotions from facial expressions and extracts key emotional indicators such as surprise and anxiety. Based on this, the emotion engine dynamically adjusts the alert level and issues warnings as needed.

[0424] The device sends notifications to the user based on detected suspicious behavior and emotional states through a mobile application built with React Native. This notification feature allows the user to immediately understand the situation and take necessary actions. The notified data is stored in a database using AWS SageMaker and used for analyzing historical environmental information.

[0425] As a concrete example, let's consider an application in a large commercial facility. Cameras installed in various areas of the facility capture visitors' movements and facial expressions, transmitting the data to a server in real time. For example, if many visitors in a certain area suddenly show signs of anxiety, the system immediately analyzes the data and sends an alert to the security team. This allows potential problems to be detected in advance, enabling swift countermeasures to be taken.

[0426] Using a generative AI model, the following text is used as an example of a prompt:

[0427] "We detected an expression of anxiety in venue area 3. Immediate investigation is required."

[0428] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0429] Step 1:

[0430] The server receives image data in real time from the image acquisition device. This data is video of the monitored area and is raw data for analyzing behavior and emotional states. The received image data is transmitted to a computing device for preparation for analysis.

[0431] Step 2:

[0432] The computing device on the server processes the received image data using OpenCV. Specifically, it extracts features from the images to recognize people and their movements, and sends these features to the AI ​​module. The input here is image data, and the output is a dataset containing the features.

[0433] Step 3:

[0434] The AI ​​module utilizes the TensorFlow framework to analyze feature data and detect suspicious behavior and recognize emotional states. The input is feature data sent from the server, and the output is an evaluation of the presence or absence of suspicious behavior and emotional state as a result of the analysis. This allows for the identification of abnormal behaviors and emotions exhibited by individuals.

[0435] Step 4:

[0436] The server dynamically adjusts the alert level based on emotional data analyzed using an emotion engine. The input is the analysis result of the AI ​​module, and the output is the adapted alert level. This adjustment changes the system's behavior to enable immediate responses as needed.

[0437] Step 5:

[0438] The device sends notifications to the user based on suspicious behavior and emotional state through an app developed using React Native. The input is a server-configured alert level and analysis results, and the output displayed to the user is a specific warning or notification message. For example, a prompt message such as "An anxious expression has been detected in venue area 3. Immediate verification is required." might be sent.

[0439] Step 6:

[0440] Users make decisions about on-site responses and take necessary actions based on notifications displayed on their devices. The output to the user is information provided on the device, and a corresponding response is required.

[0441] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0442] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0443] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0444] [Third Embodiment]

[0445] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0446] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0447] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0448] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0449] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0450] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0451] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0452] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0453] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0454] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0455] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0456] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0457] The system of the present invention is implemented by a computer program composed of multiple modules. Each module is responsible for a specific function, and together they function as a whole system. A specific embodiment is shown below.

[0458] The server first acquires video data in real time from the image acquisition device. This data is transmitted using a dedicated protocol to ensure security. The AI ​​module on the server analyzes the acquired video data frame by frame and detects suspicious behavior. For example, if a person moves erratically between vehicles in a parking lot, it is compared to a predefined dangerous behavior and recognized as suspicious behavior if it exceeds a threshold.

[0459] The device alerts the user to detected suspicious activity. This notification is sent, for example, via push notification through an application or email, allowing the user to immediately check the situation. Receiving alerts from the device enables the user to respond quickly.

[0460] Furthermore, the server extracts facial information from the video data and compares it with pre-registered authorized person information. If an unregistered person is identified through this process, they are flagged as a suspicious person. For example, in an office building, if a visitor whose face is not registered enters the premises, this system will identify that person as a suspicious person and notify the administrator.

[0461] Furthermore, the server analyzes environmental data and dynamically adjusts security levels according to weather and time of day. For example, under conditions where the risk is higher, such as at night or on weekends, the security level is increased and the detection sensitivity is enhanced. This reduces false alarms while enabling appropriate security responses.

[0462] Finally, the server collects event data from suspicious person detections and automatically generates a report. This report includes detailed information such as the time and location of the incident and the detected behavior. This report is provided to the user via their terminal, allowing administrators to review past events and use the information to improve future security measures.

[0463] Thus, the system of the present invention provides efficient and effective crime prevention measures by enabling real-time detection of suspicious behavior, identification of individuals through facial recognition, and dynamic adjustment of security levels according to the environment.

[0464] The following describes the processing flow.

[0465] Step 1:

[0466] The server receives video data from the image acquisition device in real time. The video data is transferred to the server in stream format and communicated over the network using a secure and highly efficient protocol.

[0467] Step 2:

[0468] The server analyzes the received video data frame by frame using an AI module. The AI ​​module evaluates the speed and direction of movement based on predefined suspicious behavior patterns and identifies suspicious behavior in real time.

[0469] Step 3:

[0470] The server immediately generates an alert if suspicious activity is detected. The alert includes the time of detection, a detailed description of the activity, and the scope of its impact. This information is recorded in the log.

[0471] Step 4:

[0472] The device receives alerts from the server and notifies the user. Notification methods include push notifications through the device's application and email notifications. Upon receiving a notification, the user can quickly check the situation.

[0473] Step 5:

[0474] The server extracts facial information from video data and compares it with pre-registered information on authorized individuals. If an unregistered person is detected during this matching process, they are marked as a suspicious person.

[0475] Step 6:

[0476] The server collects environmental data (e.g., weather, date and time, ambient light levels) and uses machine learning models to dynamically adjust security levels. Under certain conditions, it increases alarm sensitivity and optimizes the system to minimize false alarms.

[0477] Step 7:

[0478] The server collects detailed data on suspicious person detection and other alarm events. Based on this data, it automatically generates reports that administrators can use to analyze past events. These reports are then provided to users via their terminals.

[0479] (Example 1)

[0480] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0481] In security systems, it is necessary to quickly and accurately detect suspicious behavior and individuals, minimizing false alarms and enabling efficient responses. Conventional systems suffer from problems such as false alarms and excessive alerts, especially under complex environmental conditions, and these issues need to be resolved.

[0482] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0483] In this invention, the server includes means having an information processing device that analyzes visual data acquired from an image acquisition device to detect suspicious behavior; means extracting facial information from the visual data and comparing it with pre-registered authorized personal information to identify a suspicious person; and means analyzing external environmental information and dynamically adjusting the security level based on the analysis results to detect suspicious behavior with increased sensitivity. This enables the rapid identification of suspicious behavior and suspicious individuals, reduces false alarms through adjustment of security levels according to the environment, and allows for efficient crime prevention response.

[0484] "Image acquisition equipment" refers to devices used to acquire visual data, and includes equipment such as cameras and sensors.

[0485] "Visual data" refers to digital information acquired as images or videos, and is the data that is subject to analysis.

[0486] An "information processing device" refers to computing equipment such as computers and servers, which are devices that perform data analysis and processing.

[0487] "Suspicious behavior" refers to movements or actions that deviate from normal behavioral patterns and are judged to be dangerous or suspicious.

[0488] "Facial information" refers to distinctive digital data about a person's face, and is the information used for personal identification through facial recognition.

[0489] "Authorized personal information" refers to information about individuals whose access and actions have been approved and who have been registered in advance.

[0490] "Verification" refers to the process of comparing acquired data with registered data to check for matches or discrepancies.

[0491] A "suspicious person" refers to a person who is identified as not being registered or authenticated.

[0492] "External environmental information" refers to data related to the system's external conditions, such as weather, time of day, and illuminance.

[0493] "Safety level" refers to an indicator that shows the degree of vigilance and response to suspicious behavior or individuals.

[0494] "Dynamic adjustment" refers to the process of changing system settings and operation in real time according to the situation and conditions.

[0495] A "false alarm" refers to a situation where the system mistakenly issues an alert even though there is no actual suspicious activity or person present.

[0496] The system in this invention efficiently realizes security detection by combining various hardware and software. First, the server acquires visual data in real time from a camera, which is an image acquisition device. This visual data is securely transferred using a dedicated protocol and analyzed by an information processing device.

[0497] The server analyzes this data frame by frame using an AI module to detect suspicious behavior. The AI ​​module uses a generative AI model to evaluate movement and behavioral patterns within the visual data and compare them to predefined suspicious behaviors. For example, in a parking lot, if a person moving irregularly between vehicles is detected, it is recognized as suspicious behavior.

[0498] Furthermore, the server extracts facial information from visual data and compares it with pre-registered, authorized personal information. If an unregistered person is detected, they are flagged as a suspicious individual. In office building security, this allows for immediate notification to administrators of visitors not registered in the facial recognition system.

[0499] The server also incorporates external environmental information and dynamically adjusts the security level according to weather, time of day, and other factors. For example, detection sensitivity is increased at night or during periods of higher risk. This operation minimizes false alarms and enables more appropriate security responses.

[0500] The device generates alerts regarding suspicious behavior or individuals detected by the server and notifies the user. Notifications are sent via smartphone apps or email, allowing users to quickly check the situation and take appropriate action.

[0501] An example of a prompt message is: "Identify suspicious behavior in the parking lot, and prioritize notifying you of individuals who move irregularly between vehicles multiple times." Based on this prompt, the AI ​​operates efficiently, highlighting and detecting anomalies that need to be found.

[0502] In this way, the system of the present invention can provide efficient and effective crime prevention measures by integrating real-time detection of suspicious behavior, identification of individuals through facial recognition, and dynamic adjustment of security levels according to the environment.

[0503] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0504] Step 1:

[0505] The server acquires visual data in real time from the camera, which is an image acquisition device. The input is the camera's video stream, which is received by the server in a secure, proprietary protocol. The output is digital video data ready for analysis.

[0506] Step 2:

[0507] The server analyzes the acquired visual data frame by frame using an AI module to detect suspicious behavior. The input is the video data acquired in step 1, and the movement and patterns of the data are analyzed using a generating AI model. The output is warning information about the detected abnormal behavior. Specifically, a person moving irregularly between vehicles in a parking lot is identified, and suspicious behavior is recognized by comparing it with predefined dangerous behaviors.

[0508] Step 3:

[0509] The server extracts facial information from visual data and compares it with registered, authorized personal information. The input is frame-by-frame facial data. By matching this with personal information in the database, a list of unregistered individuals is generated as output. Specifically, this is the process by which visitors to an office building are identified as suspicious individuals.

[0510] Step 4:

[0511] The server takes in external environmental information and dynamically adjusts the security level. Environmental data such as time of day and weather are used as input. Based on this data, security settings under risk conditions are re-evaluated, and the adjusted security level is applied as output. Specifically, it reduces false alarms by increasing detection sensitivity at night.

[0512] Step 5:

[0513] The terminal generates alerts for suspicious activity detected by the server and notifies the user. The input is warning information from the server, which is sent to the terminal via push notification or email. The output provides the user with details of the suspicious activity, prompting a quick response. Specifically, it supports administrators in immediately accessing the warning received on their smartphone to check the situation on-site.

[0514] (Application Example 1)

[0515] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0516] Ensuring the safety of homes and their surroundings requires a system that can immediately detect and notify of suspicious behavior and visitors. However, conventional surveillance systems often fail to adapt flexibly to environmental changes, leading to false alarms and missed alerts. Furthermore, adjusting security levels to suit specific situations and accurately identifying unregistered visitors are difficult. This presents a challenge in effectively managing home security.

[0517] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0518] In this invention, the server includes means for analyzing video data collected from an image acquisition device to detect abnormal behavior, means for comparing facial information extracted from the video data with registered person information, and means for learning environmental data to dynamically adjust the security level. This enables rapid detection of abnormalities inside and outside the home and immediate notification to the user. This system achieves more accurate security management by pre-registering visitor information and sending an abnormality notification when an unregistered visitor arrives. Furthermore, by automatically adjusting security sensitivity according to changes in time and weather, it becomes possible to take flexible measures according to each situation while reducing false alarms.

[0519] An "image acquisition device" is a device used to capture video data and plays a role in collecting real-time information about the monitored area.

[0520] An "information processing device" is a device that analyzes collected video data and performs processing to detect abnormalities or suspicious behavior.

[0521] A "warning" refers to an alert message sent to the user when abnormal behavior is detected, serving as a means of immediate alerting.

[0522] "Human facial information" refers to facial images and feature data extracted from video data to identify individuals.

[0523] "Comparison" is the process of matching extracted facial information with pre-registered person information to determine whether or not they match.

[0524] An "abnormal person" refers to an individual whose registered information does not match their behavior and who exhibits unexpected actions.

[0525] "Safety level" refers to a standard value that adjusts the sensitivity of abnormal behavior detection and the intensity of response measures, and it changes depending on the situation.

[0526] "Dynamic adjustment" refers to the process of automatically changing settings in response to changes in environmental data and other factors.

[0527] "Managing safety within and around the home" refers to activities that involve tracking and responding to abnormal behavior in order to protect the inside and surrounding areas of the home.

[0528] "Visitor information" refers to data about people who visit a home, which is registered in the system in advance.

[0529] An "automatically generated report" is a document created based on data collected from detected events, intended to record system operation and abnormal incidents.

[0530] "Reducing false alarms" means reducing incorrect warning messages and irrelevant data to improve accuracy.

[0531] A "prompt message" refers to a sentence that is input to an AI model to give instructions and generate output.

[0532] To realize this invention, software consisting of multiple modules is required. The server plays a central role, acquiring video data in real time from cameras installed in and around the home. Standard home surveillance cameras can be used for this purpose. Suspicious activity is detected by using image analysis software such as TensorFlow or OpenCV on the server.

[0533] The server analyzes the acquired video data frame by frame to determine if the movement and behavior patterns match predefined criteria for "abnormal individuals" or "suspicious behavior." For example, if a suspicious person is loitering around a house late at night, the server can detect it. Upon detection, a push notification is immediately sent to the device to alert the user.

[0534] Furthermore, the server extracts facial information and compares it with visitor information previously registered by the user. This automatically identifies any visits by unregistered individuals as an anomaly. In addition, by using cloud services such as AWS Lambda, it is possible to dynamically adjust the "security level" based on environmental data (e.g., time of day and weather). This allows for increased security sensitivity at night, for example, to improve the probability of detecting anomalies.

[0535] As a concrete example, even when a homeowner is away from home, this system can be used to receive notifications if a suspicious person visits, allowing them to confirm the safety of their property. Furthermore, the reports generated using the AI ​​model allow for consideration of future crime prevention measures based on past data.

[0536] Examples of specific prompts for the generated AI model include, "Please suggest effective countermeasures for the abnormal behavior detected around my home." In this way, this system provides concrete means to achieve advanced security management.

[0537] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0538] Step 1:

[0539] The server acquires video data in real time from cameras installed in or around homes. The input is a video stream from the cameras, and the server receives and stores this data, resulting in analyzable video data as output. This video data is stored on the server via a secure protocol.

[0540] Step 2:

[0541] The server divides the acquired video data into frames and performs image analysis using TensorFlow and OpenCV. The input is the framed video data stored on the server. Specifically, it performs motion analysis and behavioral pattern recognition, and sets a flag if suspicious behavior is detected. The output is the result of the determination of whether or not suspicious behavior was detected.

[0542] Step 3:

[0543] If suspicious activity is detected, the server sends a push notification to the device. The input is the result of the detected suspicious activity, and the output is a warning notification to the user's device. This notification is sent immediately, allowing the user to check the situation on their smartphone or computer.

[0544] Step 4:

[0545] The server extracts facial information from video data and compares it with visitor information previously registered by the user. The input is facial feature data extracted for each frame, and the output is the matching result. If an unregistered person visits, the server detects the anomaly and records it as a flag.

[0546] Step 5:

[0547] The server uses cloud services such as AWS Lambda to collect environmental data (e.g., time of day and weather) and dynamically adjust the security level. The input is environmental data, and the output is the adjusted security level. During nighttime or inclement weather, the system switches to high-sensitivity mode. This process reduces false alarms and enables the provision of appropriate countermeasures.

[0548] Step 6:

[0549] The user receives an automatically generated report based on detection results and environmental data. The input is data collected from the server, and the output is a report displayed on the user's terminal. This report includes past anomaly detection events and suggestions for future security measures. A specific example is "the detection time and response when a person moving around in the backyard late at night was detected."

[0550] Step 7:

[0551] The server uses a generative AI model to perform analysis and provide information in response to user prompts. The input is a prompt such as, "Please suggest effective countermeasures for the abnormal behavior detected around my home," and the output is a suggested result generated by the AI. This allows users to take crime prevention measures with high accuracy.

[0552] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0553] This invention relates to a security system comprising an image acquisition device, an AI module, an emotion engine, and related algorithms. Specific embodiments of this invention are described below.

[0554] The server first receives video data acquired from the image acquisition device in real time. This data includes people and their movements within the monitoring area. The server uses an AI module to analyze this data and detect suspicious behavior and the faces of unauthorized individuals.

[0555] Furthermore, the server uses an emotion engine to analyze the user's facial expressions from the video data and recognize their emotional state. This emotion recognition process makes it possible to determine whether the user is showing anxiety, surprise, or other emotional states. For example, if a visitor shows an anxious expression as they pass through a security gate in the lobby of an office building, that emotional state is recognized and considered as one of the factors to adjust the level of alert.

[0556] The terminal, following instructions from the server, notifies the user of alerts that include emotional states assessed by the emotion engine. These alerts are tailored to the situation and, if necessary, help administrators and security personnel respond immediately on-site. Users can refer to the alerts provided by the terminal to assess the urgency and take appropriate action quickly.

[0557] Furthermore, the server utilizes accumulated sentiment data and historical environmental data, analyzing this information using machine learning techniques. This analysis reduces false alarms, predicts potential security risks, and optimizes the overall system operation.

[0558] Furthermore, automated reports generated based on past suspicious person detection events and sentiment data are regularly provided to users. This allows administrators and security personnel to understand past situations and use this information to develop more effective security measures.

[0559] Thus, the present invention is a system that improves overall crime prevention effectiveness by combining an emotion engine with conventional security systems to provide more accurate person recognition and situational response capabilities.

[0560] The following describes the processing flow.

[0561] Step 1:

[0562] The server receives video data in real time from the image acquisition device. The acquired video data is transferred to the server via essential security protocols.

[0563] Step 2:

[0564] The server uses an AI module to analyze video data and execute a motion detection algorithm. It analyzes movement patterns and speeds to evaluate whether there is any suspicious behavior or abnormal movement.

[0565] Step 3:

[0566] The server uses a facial recognition algorithm to extract human faces from video data. The extracted facial information is then compared against already registered authorized person information. If an unregistered person is identified, they are marked as a suspicious person.

[0567] Step 4:

[0568] The server uses an emotion engine to analyze the user's emotional state from facial information. Emotional states include joy, surprise, anger, and anxiety, and the recognition results for each are recorded in the system.

[0569] Step 5:

[0570] The server combines information on recognized emotional states and suspicious behavior to determine the priority of alarms. For example, if a person exhibiting anxious emotions is identified, the alarm priority is increased, and it is determined that immediate action is required.

[0571] Step 6:

[0572] The device receives notifications from the server and provides alerts and related information to the user. The user can receive notifications through the device's application and immediately check the content.

[0573] Step 7:

[0574] Users evaluate the situation on-site based on the alarm information provided via their devices and take necessary security measures. This may include promptly notifying administrators or security staff.

[0575] Step 8:

[0576] The server uses accumulated sentiment data and historical environmental data to perform machine learning. This reduces the frequency of false alarms and predicts future security risks.

[0577] Step 9:

[0578] The server automatically generates reports based on these analysis results and sends them to users and administrators periodically. These reports serve as important resources for improving security measures.

[0579] (Example 2)

[0580] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0581] Traditional security systems rely solely on detecting suspicious behavior and identifying unauthorized individuals, which leads to a high rate of false alarms and an inability to consider emotional shifts. Furthermore, the effectiveness of adjusting security levels and conducting post-incident analyses is limited, making rapid and accurate responses difficult.

[0582] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0583] In this invention, the server includes means having a data processing device that analyzes video data acquired from an image acquisition device to detect suspicious behavior; means analyzing the emotional state of a person based on the video data and dynamically adjusting the level of attention based on an assessment of urgency; and means performing analysis using accumulated emotional data and past situational data to reduce false alarms and predict potential risks. This enables more accurate detection of suspicious individuals and adjustment of the security level appropriately based on emotion recognition.

[0584] An "image acquisition device" is a device used to acquire video data within a surveillance area, and includes cameras and sensors.

[0585] A "data processing device" is a device that analyzes received video data to detect suspicious behavior and extract facial information of individuals.

[0586] "Suspicious behavior" refers to actions or behaviors that deviate from normal behavioral patterns and may indicate a security risk.

[0587] "Warning information" refers to notifications and alerts issued when suspicious behavior or individuals are detected.

[0588] "A person's emotional state" refers to data that represents a person's emotional response, obtained by analyzing facial expressions extracted from video data.

[0589] "Assessing urgency" is the process of determining the immediacy and necessity of a response based on information about detected suspicious behavior and emotional states.

[0590] "Attention level" is an indicator that shows the level of risk recognized by the system, and it is dynamically adjusted.

[0591] "Accumulated emotional data" refers to a collection of data on emotional states collected in the past, which is used for analyzing and predicting security systems.

[0592] "Situational data" refers to historical information about the environment and events, which is used for analysis and machine learning.

[0593] "Reducing false alarms" refers to a series of processes undertaken to minimize the occurrence of incorrect warnings and false detections.

[0594] "Predicting potential risks" is the process of using accumulated data to detect potential dangers that may occur in the future.

[0595] This invention is an advanced system for improving security in a surveillance area, and consists of an image acquisition device, a data processing device, an emotion recognition engine, and a server. The server first receives video data in real time from an image acquisition device such as a surveillance camera. The hardware used includes a network-connected high-resolution camera.

[0596] The server transfers the received video data to a data processing unit, where data analysis is performed using an AI module. This AI module utilizes common machine learning frameworks such as TensorFlow and PyTorch. This enables the detection of suspicious behavior and the identification of unauthorized individuals by matching their faces against a database.

[0597] Furthermore, the server uses an emotion recognition engine to analyze a person's emotional state. This emotion recognition engine utilizes common emotion analysis tools such as Face API or similar technologies. The emotional data derived from facial expressions is used to adjust security levels and set alert levels. For example, if a visitor in an office building lobby displays an anxious expression, the system recognizes their emotional state and issues an alert.

[0598] The device receives warning information from the server and notifies the user of an alert. This alert includes detailed information about the emotional state and suspicious behavior. Based on this information, the user assesses the urgency of the situation and takes appropriate action immediately. For example, a message such as "Anxious facial expression detected: On-site verification recommended" may be displayed on the device screen.

[0599] The server also uses accumulated sentiment data and historical situational data, employing machine learning techniques to perform a comprehensive analysis. This allows for a reduction in false alarms, prediction of potential risks, and optimization of the overall system operation. An example of a prompt using a generative AI model is, "Please provide the sentiment data analysis results based on yesterday's suspicious person detection event." Such prompts allow the server to generate a detailed report, which can be used to develop future security measures based on historical data.

[0600] Thus, the present invention combines data analysis and emotion recognition to provide advanced security measures and enable the construction of an efficient crime prevention system.

[0601] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0602] Step 1:

[0603] The server receives video data in real time from the image acquisition device. The input is video data from the surveillance camera, and the output is a raw video stream. Specifically, it uses a network protocol to receive the video signal and prevents data loss by buffering it.

[0604] Step 2:

[0605] The server passes the received video data to the AI ​​module to detect suspicious behavior. The input for this step is the video data obtained in step 1, and the output is identification information indicating suspicious behavior. Specifically, the AI ​​module uses a built-in machine learning algorithm (e.g., an object detection model using TensorFlow) to analyze the movement in the video and identify suspicious patterns.

[0606] Step 3:

[0607] The server further processes the output data from the AI ​​module, extracting facial information and matching it against the allow list. The input for this step is video data and identification information of suspicious behavior, and the output is facial information of unauthorized individuals. Specifically, it uses a facial recognition framework (e.g., OpenCV) to analyze facial images and compare them with a pre-registered database to identify non-matching faces.

[0608] Step 4:

[0609] The server analyzes facial expressions using an emotion engine and recognizes the emotional state. The input for this step is facial information, and the output is the recognized emotional state (e.g., anxiety, surprise). Specifically, it uses an emotion recognition library to analyze subtle facial movements and facial feature quantities to classify emotions.

[0610] Step 5:

[0611] The terminal notifies the user of an alert based on identification information and emotional state of suspicious behavior sent from the server. The input for this step is the identification information and emotional state from the server, and the output is the alert message displayed to the user. Specifically, the terminal screen displays text such as "Anxious facial expression detected: Site verification recommended."

[0612] Step 6:

[0613] The user assesses the urgency based on the device's alerts and takes the necessary actions. The input for this step is the device's alert message, and the output is the on-site response based on the user's actions. Specifically, the user immediately checks the surveillance camera footage and contacts the security personnel.

[0614] Step 7:

[0615] The server analyzes accumulated data to reduce false alarms and predict potential risks. The inputs for this step are sentiment data and historical environmental data, and the output is a security setting adjusted based on predictions. Specifically, it uses machine learning algorithms to update the model based on past patterns.

[0616] In this way, the entire system can function with greater precision and efficiency.

[0617] (Application Example 2)

[0618] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0619] In recent years, the importance of ensuring safety in facilities and event venues has increased. However, conventional security systems only detect people's movements and actions, and cannot grasp changes in emotions or underlying psychological states, which has made it difficult to respond appropriately and quickly.

[0620] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0621] In this invention, the server includes a computing device that analyzes image data acquired from an image acquisition device to detect suspicious behavior, means for issuing a warning when suspicious behavior is detected, means for analyzing emotional states from video data and adjusting the alert level based on the results, and means for notifying a mobile terminal of information based on the suspicious behavior and emotional state. This makes it possible to comprehensively understand a person's behavior and emotional changes and take appropriate and prompt safety measures.

[0622] An "image acquisition device" is a device used to collect image data from a target area.

[0623] "Image data" refers to visual information that records a person, their movements, and their surrounding environment.

[0624] A "computational device" is a combination of hardware and software that performs processing to analyze image data and detect suspicious behavior.

[0625] "Means of issuing warnings" refers to a function that generates alerts and sends notifications in response to detected suspicious behavior.

[0626] "Facial information of a person" refers to data that shows the characteristics of an individual person, extracted from image data.

[0627] "Authorized person information" refers to a database of authorized individuals that have been pre-registered in the system.

[0628] "Environmental information" refers to data that shows the past and present conditions within the monitoring area.

[0629] "Means for dynamically adjusting the security level" refers to a function that changes the system's security settings according to the analysis results.

[0630] "Means for analyzing emotional states and adjusting alert levels based on the results" refers to a process for estimating emotions from a person's facial expressions and adjusting the system's alert level based on those results.

[0631] "Means of notifying mobile devices" refers to a function for sending information about suspicious behavior or emotional states to the user's mobile device.

[0632] The system of the present invention consists of multiple components and aims to enhance security both inside and outside the facility. The server receives image data in real time from image acquisition devices. The hardware used here includes network cameras and other image sensors. The acquired image data is first sent to a processing unit executed on the server, which uses OpenCV to analyze human movement and facial information.

[0633] The AI ​​module uses deep learning frameworks such as TensorFlow to detect suspicious behavior and analyze emotional states from this image data. This AI module recognizes emotions from facial expressions and extracts key emotional indicators such as surprise and anxiety. Based on this, the emotion engine dynamically adjusts the alert level and issues warnings as needed.

[0634] The device sends notifications to the user based on detected suspicious behavior and emotional states through a mobile application built with React Native. This notification feature allows the user to immediately understand the situation and take necessary actions. The notified data is stored in a database using AWS SageMaker and used for analyzing historical environmental information.

[0635] As a concrete example, let's consider an application in a large commercial facility. Cameras installed in various areas of the facility capture visitors' movements and facial expressions, transmitting the data to a server in real time. For example, if many visitors in a certain area suddenly show signs of anxiety, the system immediately analyzes the data and sends an alert to the security team. This allows potential problems to be detected in advance, enabling swift countermeasures to be taken.

[0636] Using a generative AI model, the following text is used as an example of a prompt:

[0637] "We detected an expression of anxiety in venue area 3. Immediate investigation is required."

[0638] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0639] Step 1:

[0640] The server receives image data in real time from the image acquisition device. This data is video of the monitored area and is raw data for analyzing behavior and emotional states. The received image data is transmitted to a computing device for preparation for analysis.

[0641] Step 2:

[0642] The computing device on the server processes the received image data using OpenCV. Specifically, it extracts features from the images to recognize people and their movements, and sends these features to the AI ​​module. The input here is image data, and the output is a dataset containing the features.

[0643] Step 3:

[0644] The AI ​​module utilizes the TensorFlow framework to analyze feature data and detect suspicious behavior and recognize emotional states. The input is feature data sent from the server, and the output is an evaluation of the presence or absence of suspicious behavior and emotional state as a result of the analysis. This allows for the identification of abnormal behaviors and emotions exhibited by individuals.

[0645] Step 4:

[0646] The server dynamically adjusts the alert level based on emotional data analyzed using an emotion engine. The input is the analysis result of the AI ​​module, and the output is the adapted alert level. This adjustment changes the system's behavior to enable immediate responses as needed.

[0647] Step 5:

[0648] The device sends notifications to the user based on suspicious behavior and emotional state through an app developed using React Native. The input is a server-configured alert level and analysis results, and the output displayed to the user is a specific warning or notification message. For example, a prompt message such as "An anxious expression has been detected in venue area 3. Immediate verification is required." might be sent.

[0649] Step 6:

[0650] Users make decisions about on-site responses and take necessary actions based on notifications displayed on their devices. The output to the user is information provided on the device, and a corresponding response is required.

[0651] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0652] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0653] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0654] [Fourth Embodiment]

[0655] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0656] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0657] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0658] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0659] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0660] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0661] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0662] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0663] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0664] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0665] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0666] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0667] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0668] The system of the present invention is implemented by a computer program composed of multiple modules. Each module is responsible for a specific function, and together they function as a whole system. A specific embodiment is shown below.

[0669] The server first acquires video data in real time from the image acquisition device. This data is transmitted using a dedicated protocol to ensure security. The AI ​​module on the server analyzes the acquired video data frame by frame and detects suspicious behavior. For example, if a person moves erratically between vehicles in a parking lot, it is compared to a predefined dangerous behavior and recognized as suspicious behavior if it exceeds a threshold.

[0670] The device alerts the user to detected suspicious activity. This notification is sent, for example, via push notification through an application or email, allowing the user to immediately check the situation. Receiving alerts from the device enables the user to respond quickly.

[0671] Furthermore, the server extracts facial information from the video data and compares it with pre-registered authorized person information. If an unregistered person is identified through this process, they are flagged as a suspicious person. For example, in an office building, if a visitor whose face is not registered enters the premises, this system will identify that person as a suspicious person and notify the administrator.

[0672] Furthermore, the server analyzes environmental data and dynamically adjusts security levels according to weather and time of day. For example, under conditions where the risk is higher, such as at night or on weekends, the security level is increased and the detection sensitivity is enhanced. This reduces false alarms while enabling appropriate security responses.

[0673] Finally, the server collects event data from suspicious person detections and automatically generates a report. This report includes detailed information such as the time and location of the incident and the detected behavior. This report is provided to the user via their terminal, allowing administrators to review past events and use the information to improve future security measures.

[0674] Thus, the system of the present invention provides efficient and effective crime prevention measures by enabling real-time detection of suspicious behavior, identification of individuals through facial recognition, and dynamic adjustment of security levels according to the environment.

[0675] The following describes the processing flow.

[0676] Step 1:

[0677] The server receives video data from the image acquisition device in real time. The video data is transferred to the server in stream format and communicated over the network using a secure and highly efficient protocol.

[0678] Step 2:

[0679] The server analyzes the received video data frame by frame using an AI module. The AI ​​module evaluates the speed and direction of movement based on predefined suspicious behavior patterns and identifies suspicious behavior in real time.

[0680] Step 3:

[0681] The server immediately generates an alert if suspicious activity is detected. The alert includes the time of detection, a detailed description of the activity, and the scope of its impact. This information is recorded in the log.

[0682] Step 4:

[0683] The device receives alerts from the server and notifies the user. Notification methods include push notifications through the device's application and email notifications. Upon receiving a notification, the user can quickly check the situation.

[0684] Step 5:

[0685] The server extracts facial information from video data and compares it with pre-registered information on authorized individuals. If an unregistered person is detected during this matching process, they are marked as a suspicious person.

[0686] Step 6:

[0687] The server collects environmental data (e.g., weather, date and time, ambient light levels) and uses machine learning models to dynamically adjust security levels. Under certain conditions, it increases alarm sensitivity and optimizes the system to minimize false alarms.

[0688] Step 7:

[0689] The server collects detailed data on suspicious person detection and other alarm events. Based on this data, it automatically generates reports that administrators can use to analyze past events. These reports are then provided to users via their terminals.

[0690] (Example 1)

[0691] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0692] In security systems, it is necessary to quickly and accurately detect suspicious behavior and individuals, minimizing false alarms and enabling efficient responses. Conventional systems suffer from problems such as false alarms and excessive alerts, especially under complex environmental conditions, and these issues need to be resolved.

[0693] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0694] In this invention, the server includes means having an information processing device that analyzes visual data acquired from an image acquisition device to detect suspicious behavior; means extracting facial information from the visual data and comparing it with pre-registered authorized personal information to identify a suspicious person; and means analyzing external environmental information and dynamically adjusting the security level based on the analysis results to detect suspicious behavior with increased sensitivity. This enables the rapid identification of suspicious behavior and suspicious individuals, reduces false alarms through adjustment of security levels according to the environment, and allows for efficient crime prevention response.

[0695] "Image acquisition equipment" refers to devices used to acquire visual data, and includes equipment such as cameras and sensors.

[0696] "Visual data" refers to digital information acquired as images or videos, and is the data that is subject to analysis.

[0697] An "information processing device" refers to computing equipment such as computers and servers, which are devices that perform data analysis and processing.

[0698] "Suspicious behavior" refers to movements or actions that deviate from normal behavioral patterns and are judged to be dangerous or suspicious.

[0699] "Facial information" refers to distinctive digital data about a person's face, and is the information used for personal identification through facial recognition.

[0700] "Authorized personal information" refers to information about individuals whose access and actions have been approved and who have been registered in advance.

[0701] "Verification" refers to the process of comparing acquired data with registered data to check for matches or discrepancies.

[0702] A "suspicious person" refers to a person who is identified as not being registered or authenticated.

[0703] "External environmental information" refers to data related to the system's external conditions, such as weather, time of day, and illuminance.

[0704] "Safety level" refers to an indicator that shows the degree of vigilance and response to suspicious behavior or individuals.

[0705] "Dynamic adjustment" refers to the process of changing system settings and operation in real time according to the situation and conditions.

[0706] A "false alarm" refers to a situation where the system mistakenly issues an alert even though there is no actual suspicious activity or person present.

[0707] The system in this invention efficiently realizes security detection by combining various hardware and software. First, the server acquires visual data in real time from a camera, which is an image acquisition device. This visual data is securely transferred using a dedicated protocol and analyzed by an information processing device.

[0708] The server analyzes this data frame by frame using an AI module to detect suspicious behavior. The AI ​​module uses a generative AI model to evaluate movement and behavioral patterns within the visual data and compare them to predefined suspicious behaviors. For example, in a parking lot, if a person moving irregularly between vehicles is detected, it is recognized as suspicious behavior.

[0709] Furthermore, the server extracts facial information from visual data and compares it with pre-registered, authorized personal information. If an unregistered person is detected, they are flagged as a suspicious individual. In office building security, this allows for immediate notification to administrators of visitors not registered in the facial recognition system.

[0710] The server also incorporates external environmental information and dynamically adjusts the security level according to weather, time of day, and other factors. For example, detection sensitivity is increased at night or during periods of higher risk. This operation minimizes false alarms and enables more appropriate security responses.

[0711] The device generates alerts regarding suspicious behavior or individuals detected by the server and notifies the user. Notifications are sent via smartphone apps or email, allowing users to quickly check the situation and take appropriate action.

[0712] An example of a prompt message is: "Identify suspicious behavior in the parking lot, and prioritize notifying you of individuals who move irregularly between vehicles multiple times." Based on this prompt, the AI ​​operates efficiently, highlighting and detecting anomalies that need to be found.

[0713] In this way, the system of the present invention can provide efficient and effective crime prevention measures by integrating real-time detection of suspicious behavior, identification of individuals through facial recognition, and dynamic adjustment of security levels according to the environment.

[0714] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0715] Step 1:

[0716] The server acquires visual data in real time from the camera, which is an image acquisition device. The input is the camera's video stream, which is received by the server in a secure, proprietary protocol. The output is digital video data ready for analysis.

[0717] Step 2:

[0718] The server analyzes the acquired visual data frame by frame using an AI module to detect suspicious behavior. The input is the video data acquired in step 1, and the movement and patterns of the data are analyzed using a generating AI model. The output is warning information about the detected abnormal behavior. Specifically, a person moving irregularly between vehicles in a parking lot is identified, and suspicious behavior is recognized by comparing it with predefined dangerous behaviors.

[0719] Step 3:

[0720] The server extracts facial information from visual data and compares it with registered, authorized personal information. The input is frame-by-frame facial data. By matching this with personal information in the database, a list of unregistered individuals is generated as output. Specifically, this is the process by which visitors to an office building are identified as suspicious individuals.

[0721] Step 4:

[0722] The server takes in external environmental information and dynamically adjusts the security level. Environmental data such as time of day and weather are used as input. Based on this data, security settings under risk conditions are re-evaluated, and the adjusted security level is applied as output. Specifically, it reduces false alarms by increasing detection sensitivity at night.

[0723] Step 5:

[0724] The terminal generates alerts for suspicious activity detected by the server and notifies the user. The input is warning information from the server, which is sent to the terminal via push notification or email. The output provides the user with details of the suspicious activity, prompting a quick response. Specifically, it supports administrators in immediately accessing the warning received on their smartphone to check the situation on-site.

[0725] (Application Example 1)

[0726] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0727] Ensuring the safety of homes and their surroundings requires a system that can immediately detect and notify of suspicious behavior and visitors. However, conventional surveillance systems often fail to adapt flexibly to environmental changes, leading to false alarms and missed alerts. Furthermore, adjusting security levels to suit specific situations and accurately identifying unregistered visitors are difficult. This presents a challenge in effectively managing home security.

[0728] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0729] In this invention, the server includes means for analyzing video data collected from an image acquisition device to detect abnormal behavior, means for comparing facial information extracted from the video data with registered person information, and means for learning environmental data to dynamically adjust the security level. This enables rapid detection of abnormalities inside and outside the home and immediate notification to the user. This system achieves more accurate security management by pre-registering visitor information and sending an abnormality notification when an unregistered visitor arrives. Furthermore, by automatically adjusting security sensitivity according to changes in time and weather, it becomes possible to take flexible measures according to each situation while reducing false alarms.

[0730] An "image acquisition device" is a device used to capture video data and plays a role in collecting real-time information about the monitored area.

[0731] An "information processing device" is a device that analyzes collected video data and performs processing to detect abnormalities or suspicious behavior.

[0732] A "warning" refers to an alert message sent to the user when abnormal behavior is detected, serving as a means of immediate alerting.

[0733] "Human facial information" refers to facial images and feature data extracted from video data to identify individuals.

[0734] "Comparison" is the process of matching extracted facial information with pre-registered person information to determine whether or not they match.

[0735] An "abnormal person" refers to an individual whose registered information does not match their behavior and who exhibits unexpected actions.

[0736] "Safety level" refers to a standard value that adjusts the sensitivity of abnormal behavior detection and the intensity of response measures, and it changes depending on the situation.

[0737] "Dynamic adjustment" refers to the process of automatically changing settings in response to changes in environmental data and other factors.

[0738] "Managing safety within and around the home" refers to activities that involve tracking and responding to abnormal behavior in order to protect the inside and surrounding areas of the home.

[0739] "Visitor information" refers to data about people who visit a home, which is registered in the system in advance.

[0740] An "automatically generated report" is a document created based on data collected from detected events, intended to record system operation and abnormal incidents.

[0741] "Reducing false alarms" means reducing incorrect warning messages and irrelevant data to improve accuracy.

[0742] A "prompt message" refers to a sentence that is input to an AI model to give instructions and generate output.

[0743] To realize this invention, software consisting of multiple modules is required. The server plays a central role, acquiring video data in real time from cameras installed in and around the home. Standard home surveillance cameras can be used for this purpose. Suspicious activity is detected by using image analysis software such as TensorFlow or OpenCV on the server.

[0744] The server analyzes the acquired video data frame by frame to determine if the movement and behavior patterns match predefined criteria for "abnormal individuals" or "suspicious behavior." For example, if a suspicious person is loitering around a house late at night, the server can detect it. Upon detection, a push notification is immediately sent to the device to alert the user.

[0745] Furthermore, the server extracts facial information and compares it with visitor information previously registered by the user. This automatically identifies any visits by unregistered individuals as an anomaly. In addition, by using cloud services such as AWS Lambda, it is possible to dynamically adjust the "security level" based on environmental data (e.g., time of day and weather). This allows for increased security sensitivity at night, for example, to improve the probability of detecting anomalies.

[0746] As a concrete example, even when a homeowner is away from home, this system can be used to receive notifications if a suspicious person visits, allowing them to confirm the safety of their property. Furthermore, the reports generated using the AI ​​model allow for consideration of future crime prevention measures based on past data.

[0747] Examples of specific prompts for the generated AI model include, "Please suggest effective countermeasures for the abnormal behavior detected around my home." In this way, this system provides concrete means to achieve advanced security management.

[0748] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0749] Step 1:

[0750] The server acquires video data in real time from cameras installed in or around homes. The input is a video stream from the cameras, and the server receives and stores this data, resulting in analyzable video data as output. This video data is stored on the server via a secure protocol.

[0751] Step 2:

[0752] The server divides the acquired video data into frames and performs image analysis using TensorFlow and OpenCV. The input is the framed video data stored on the server. Specifically, it performs motion analysis and behavioral pattern recognition, and sets a flag if suspicious behavior is detected. The output is the result of the determination of whether or not suspicious behavior was detected.

[0753] Step 3:

[0754] If suspicious activity is detected, the server sends a push notification to the device. The input is the result of the detected suspicious activity, and the output is a warning notification to the user's device. This notification is sent immediately, allowing the user to check the situation on their smartphone or computer.

[0755] Step 4:

[0756] The server extracts facial information from video data and compares it with visitor information previously registered by the user. The input is facial feature data extracted for each frame, and the output is the matching result. If an unregistered person visits, the server detects the anomaly and records it as a flag.

[0757] Step 5:

[0758] The server uses cloud services such as AWS Lambda to collect environmental data (e.g., time of day and weather) and dynamically adjust the security level. The input is environmental data, and the output is the adjusted security level. During nighttime or inclement weather, the system switches to high-sensitivity mode. This process reduces false alarms and enables the provision of appropriate countermeasures.

[0759] Step 6:

[0760] The user receives an automatically generated report based on detection results and environmental data. The input is data collected from the server, and the output is a report displayed on the user's terminal. This report includes past anomaly detection events and suggestions for future security measures. A specific example is "the detection time and response when a person moving around in the backyard late at night was detected."

[0761] Step 7:

[0762] The server uses a generative AI model to perform analysis and provide information in response to user prompts. The input is a prompt such as, "Please suggest effective countermeasures for the abnormal behavior detected around my home," and the output is a suggested result generated by the AI. This allows users to take crime prevention measures with high accuracy.

[0763] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0764] This invention relates to a security system comprising an image acquisition device, an AI module, an emotion engine, and related algorithms. Specific embodiments of this invention are described below.

[0765] The server first receives video data acquired from the image acquisition device in real time. This data includes people and their movements within the monitoring area. The server uses an AI module to analyze this data and detect suspicious behavior and the faces of unauthorized individuals.

[0766] Furthermore, the server uses an emotion engine to analyze the user's facial expressions from the video data and recognize their emotional state. This emotion recognition process makes it possible to determine whether the user is showing anxiety, surprise, or other emotional states. For example, if a visitor shows an anxious expression as they pass through a security gate in the lobby of an office building, that emotional state is recognized and considered as one of the factors to adjust the level of alert.

[0767] The terminal, following instructions from the server, notifies the user of alerts that include emotional states assessed by the emotion engine. These alerts are tailored to the situation and, if necessary, help administrators and security personnel respond immediately on-site. Users can refer to the alerts provided by the terminal to assess the urgency and take appropriate action quickly.

[0768] Furthermore, the server utilizes accumulated sentiment data and historical environmental data, analyzing this information using machine learning techniques. This analysis reduces false alarms, predicts potential security risks, and optimizes the overall system operation.

[0769] Furthermore, automated reports generated based on past suspicious person detection events and sentiment data are regularly provided to users. This allows administrators and security personnel to understand past situations and use this information to develop more effective security measures.

[0770] Thus, the present invention is a system that improves overall crime prevention effectiveness by combining an emotion engine with conventional security systems to provide more accurate person recognition and situational response capabilities.

[0771] The following describes the processing flow.

[0772] Step 1:

[0773] The server receives video data in real time from the image acquisition device. The acquired video data is transferred to the server via essential security protocols.

[0774] Step 2:

[0775] The server uses an AI module to analyze video data and execute a motion detection algorithm. It analyzes movement patterns and speeds to evaluate whether there is any suspicious behavior or abnormal movement.

[0776] Step 3:

[0777] The server uses a facial recognition algorithm to extract human faces from video data. The extracted facial information is then compared against already registered authorized person information. If an unregistered person is identified, they are marked as a suspicious person.

[0778] Step 4:

[0779] The server uses an emotion engine to analyze the user's emotional state from facial information. Emotional states include joy, surprise, anger, and anxiety, and the recognition results for each are recorded in the system.

[0780] Step 5:

[0781] The server combines information on recognized emotional states and suspicious behavior to determine the priority of alarms. For example, if a person exhibiting anxious emotions is identified, the alarm priority is increased, and it is determined that immediate action is required.

[0782] Step 6:

[0783] The device receives notifications from the server and provides alerts and related information to the user. The user can receive notifications through the device's application and immediately check the content.

[0784] Step 7:

[0785] Users evaluate the situation on-site based on the alarm information provided via their devices and take necessary security measures. This may include promptly notifying administrators or security staff.

[0786] Step 8:

[0787] The server uses accumulated sentiment data and historical environmental data to perform machine learning. This reduces the frequency of false alarms and predicts future security risks.

[0788] Step 9:

[0789] The server automatically generates reports based on these analysis results and sends them to users and administrators periodically. These reports serve as important resources for improving security measures.

[0790] (Example 2)

[0791] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0792] Traditional security systems rely solely on detecting suspicious behavior and identifying unauthorized individuals, which leads to a high rate of false alarms and an inability to consider emotional shifts. Furthermore, the effectiveness of adjusting security levels and conducting post-incident analyses is limited, making rapid and accurate responses difficult.

[0793] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0794] In this invention, the server includes means having a data processing device that analyzes video data acquired from an image acquisition device to detect suspicious behavior; means analyzing the emotional state of a person based on the video data and dynamically adjusting the level of attention based on an assessment of urgency; and means performing analysis using accumulated emotional data and past situational data to reduce false alarms and predict potential risks. This enables more accurate detection of suspicious individuals and adjustment of the security level appropriately based on emotion recognition.

[0795] An "image acquisition device" is a device used to acquire video data within a surveillance area, and includes cameras and sensors.

[0796] A "data processing device" is a device that analyzes received video data to detect suspicious behavior and extract facial information of individuals.

[0797] "Suspicious behavior" refers to actions or behaviors that deviate from normal behavioral patterns and may indicate a security risk.

[0798] "Warning information" refers to notifications and alerts issued when suspicious behavior or individuals are detected.

[0799] "A person's emotional state" refers to data that represents a person's emotional response, obtained by analyzing facial expressions extracted from video data.

[0800] "Assessing urgency" is the process of determining the immediacy and necessity of a response based on information about detected suspicious behavior and emotional states.

[0801] "Attention level" is an indicator that shows the level of risk recognized by the system, and it is dynamically adjusted.

[0802] "Accumulated emotional data" refers to a collection of data on emotional states collected in the past, which is used for analyzing and predicting security systems.

[0803] "Situational data" refers to historical information about the environment and events, which is used for analysis and machine learning.

[0804] "Reducing false alarms" refers to a series of processes undertaken to minimize the occurrence of incorrect warnings and false detections.

[0805] "Predicting potential risks" is the process of using accumulated data to detect potential dangers that may occur in the future.

[0806] This invention is an advanced system for improving security in a surveillance area, and consists of an image acquisition device, a data processing device, an emotion recognition engine, and a server. The server first receives video data in real time from an image acquisition device such as a surveillance camera. The hardware used includes a network-connected high-resolution camera.

[0807] The server transfers the received video data to a data processing unit, where data analysis is performed using an AI module. This AI module utilizes common machine learning frameworks such as TensorFlow and PyTorch. This enables the detection of suspicious behavior and the identification of unauthorized individuals by matching their faces against a database.

[0808] Furthermore, the server uses an emotion recognition engine to analyze a person's emotional state. This emotion recognition engine utilizes common emotion analysis tools such as Face API or similar technologies. The emotional data derived from facial expressions is used to adjust security levels and set alert levels. For example, if a visitor in an office building lobby displays an anxious expression, the system recognizes their emotional state and issues an alert.

[0809] The device receives warning information from the server and notifies the user of an alert. This alert includes detailed information about the emotional state and suspicious behavior. Based on this information, the user assesses the urgency of the situation and takes appropriate action immediately. For example, a message such as "Anxious facial expression detected: On-site verification recommended" may be displayed on the device screen.

[0810] The server also uses accumulated sentiment data and historical situational data, employing machine learning techniques to perform a comprehensive analysis. This allows for a reduction in false alarms, prediction of potential risks, and optimization of the overall system operation. An example of a prompt using a generative AI model is, "Please provide the sentiment data analysis results based on yesterday's suspicious person detection event." Such prompts allow the server to generate a detailed report, which can be used to develop future security measures based on historical data.

[0811] Thus, the present invention combines data analysis and emotion recognition to provide advanced security measures and enable the construction of an efficient crime prevention system.

[0812] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0813] Step 1:

[0814] The server receives video data in real time from the image acquisition device. The input is video data from the surveillance camera, and the output is a raw video stream. Specifically, it uses a network protocol to receive the video signal and prevents data loss by buffering it.

[0815] Step 2:

[0816] The server passes the received video data to the AI ​​module to detect suspicious behavior. The input for this step is the video data obtained in step 1, and the output is identification information indicating suspicious behavior. Specifically, the AI ​​module uses a built-in machine learning algorithm (e.g., an object detection model using TensorFlow) to analyze the movement in the video and identify suspicious patterns.

[0817] Step 3:

[0818] The server further processes the output data from the AI ​​module, extracting facial information and matching it against the allow list. The input for this step is video data and identification information of suspicious behavior, and the output is facial information of unauthorized individuals. Specifically, it uses a facial recognition framework (e.g., OpenCV) to analyze facial images and compare them with a pre-registered database to identify non-matching faces.

[0819] Step 4:

[0820] The server analyzes facial expressions using an emotion engine and recognizes the emotional state. The input for this step is facial information, and the output is the recognized emotional state (e.g., anxiety, surprise). Specifically, it uses an emotion recognition library to analyze subtle facial movements and facial feature quantities to classify emotions.

[0821] Step 5:

[0822] The terminal notifies the user of an alert based on identification information and emotional state of suspicious behavior sent from the server. The input for this step is the identification information and emotional state from the server, and the output is the alert message displayed to the user. Specifically, the terminal screen displays text such as "Anxious facial expression detected: Site verification recommended."

[0823] Step 6:

[0824] The user assesses the urgency based on the device's alerts and takes the necessary actions. The input for this step is the device's alert message, and the output is the on-site response based on the user's actions. Specifically, the user immediately checks the surveillance camera footage and contacts the security personnel.

[0825] Step 7:

[0826] The server analyzes accumulated data to reduce false alarms and predict potential risks. The inputs for this step are sentiment data and historical environmental data, and the output is a security setting adjusted based on predictions. Specifically, it uses machine learning algorithms to update the model based on past patterns.

[0827] In this way, the entire system can function with greater precision and efficiency.

[0828] (Application Example 2)

[0829] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0830] In recent years, the importance of ensuring safety in facilities and event venues has increased. However, conventional security systems only detect people's movements and actions, and cannot grasp changes in emotions or underlying psychological states, which has made it difficult to respond appropriately and quickly.

[0831] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0832] In this invention, the server includes a computing device that analyzes image data acquired from an image acquisition device to detect suspicious behavior, means for issuing a warning when suspicious behavior is detected, means for analyzing emotional states from video data and adjusting the alert level based on the results, and means for notifying a mobile terminal of information based on the suspicious behavior and emotional state. This makes it possible to comprehensively understand a person's behavior and emotional changes and take appropriate and prompt safety measures.

[0833] An "image acquisition device" is a device used to collect image data from a target area.

[0834] "Image data" refers to visual information that records a person, their movements, and their surrounding environment.

[0835] A "computational device" is a combination of hardware and software that performs processing to analyze image data and detect suspicious behavior.

[0836] "Means of issuing warnings" refers to a function that generates alerts and sends notifications in response to detected suspicious behavior.

[0837] "Facial information of a person" refers to data that shows the characteristics of an individual person, extracted from image data.

[0838] "Authorized person information" refers to a database of authorized individuals that have been pre-registered in the system.

[0839] "Environmental information" refers to data that shows the past and present conditions within the monitoring area.

[0840] "Means for dynamically adjusting the security level" refers to a function that changes the system's security settings according to the analysis results.

[0841] "Means for analyzing emotional states and adjusting alert levels based on the results" refers to a process for estimating emotions from a person's facial expressions and adjusting the system's alert level based on those results.

[0842] "Means of notifying mobile devices" refers to a function for sending information about suspicious behavior or emotional states to the user's mobile device.

[0843] The system of the present invention consists of multiple components and aims to enhance security both inside and outside the facility. The server receives image data in real time from image acquisition devices. The hardware used here includes network cameras and other image sensors. The acquired image data is first sent to a processing unit executed on the server, which uses OpenCV to analyze human movement and facial information.

[0844] The AI ​​module uses deep learning frameworks such as TensorFlow to detect suspicious behavior and analyze emotional states from this image data. This AI module recognizes emotions from facial expressions and extracts key emotional indicators such as surprise and anxiety. Based on this, the emotion engine dynamically adjusts the alert level and issues warnings as needed.

[0845] The device sends notifications to the user based on detected suspicious behavior and emotional states through a mobile application built with React Native. This notification feature allows the user to immediately understand the situation and take necessary actions. The notified data is stored in a database using AWS SageMaker and used for analyzing historical environmental information.

[0846] As a concrete example, let's consider an application in a large commercial facility. Cameras installed in various areas of the facility capture visitors' movements and facial expressions, transmitting the data to a server in real time. For example, if many visitors in a certain area suddenly show signs of anxiety, the system immediately analyzes the data and sends an alert to the security team. This allows potential problems to be detected in advance, enabling swift countermeasures to be taken.

[0847] Using a generative AI model, the following text is used as an example of a prompt:

[0848] "We detected an expression of anxiety in venue area 3. Immediate investigation is required."

[0849] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0850] Step 1:

[0851] The server receives image data in real time from the image acquisition device. This data is video of the monitored area and is raw data for analyzing behavior and emotional states. The received image data is transmitted to a computing device for preparation for analysis.

[0852] Step 2:

[0853] The computing device on the server processes the received image data using OpenCV. Specifically, it extracts features from the images to recognize people and their movements, and sends these features to the AI ​​module. The input here is image data, and the output is a dataset containing the features.

[0854] Step 3:

[0855] The AI ​​module utilizes the TensorFlow framework to analyze feature data and detect suspicious behavior and recognize emotional states. The input is feature data sent from the server, and the output is an evaluation of the presence or absence of suspicious behavior and emotional state as a result of the analysis. This allows for the identification of abnormal behaviors and emotions exhibited by individuals.

[0856] Step 4:

[0857] The server dynamically adjusts the alert level based on emotional data analyzed using an emotion engine. The input is the analysis result of the AI ​​module, and the output is the adapted alert level. This adjustment changes the system's behavior to enable immediate responses as needed.

[0858] Step 5:

[0859] The device sends notifications to the user based on suspicious behavior and emotional state through an app developed using React Native. The input is a server-configured alert level and analysis results, and the output displayed to the user is a specific warning or notification message. For example, a prompt message such as "An anxious expression has been detected in venue area 3. Immediate verification is required." might be sent.

[0860] Step 6:

[0861] Users make decisions about on-site responses and take necessary actions based on notifications displayed on their devices. The output to the user is information provided on the device, and a corresponding response is required.

[0862] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0863] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0864] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0865] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0866] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0867] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0868] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0869] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0870] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0871] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0872] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0873] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0874] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0875] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0876] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0877] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using this memory.

[0878] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0879] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0880] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0881] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0882] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0883] The following is further disclosed regarding the embodiments described above.

[0884] (Claim 1)

[0885] The system includes a computing device that analyzes video data acquired from an image acquisition device to detect suspicious behavior, and means for issuing an alert when such suspicious behavior is detected.

[0886] A means of identifying a suspicious person by extracting human facial information from the aforementioned video data and comparing it with pre-registered authorized person information,

[0887] A means of learning from environmental data and dynamically adjusting the security level based on the learning results,

[0888] A system that includes this.

[0889] (Claim 2)

[0890] A means for collecting data on suspicious person detection events and generating an automatically generated report based on the collected data,

[0891] The system according to claim 1, which notifies the user of the automatically generated report.

[0892] (Claim 3)

[0893] This includes a means of using machine learning to analyze patterns based on past environmental data to reduce false alarms.

[0894] The system according to claim 1.

[0895] "Example 1"

[0896] (Claim 1)

[0897] It has an information processing device that analyzes visual data acquired from an image acquisition device to detect suspicious behavior, and means for issuing a warning when suspicious behavior is detected,

[0898] A means of identifying a suspicious person by extracting facial information from the aforementioned visual data and comparing it with pre-registered authorized personal information,

[0899] A means of analyzing external environmental information and dynamically adjusting the safety level based on the analysis results,

[0900] A means for detecting suspicious behavior with increased sensitivity according to the adjusted safety level,

[0901] A system that includes this.

[0902] (Claim 2)

[0903] A means for integrating information on suspicious person detection events and creating an automatically generated report based on the integrated information,

[0904] The system according to claim 1, which notifies the user of the automatically generated report.

[0905] (Claim 3)

[0906] This includes a means of analyzing training data using machine learning to reduce false alarms based on past environmental information.

[0907] The system according to claim 1.

[0908] "Application Example 1"

[0909] (Claim 1)

[0910] It has an information processing device that analyzes video data acquired from an image acquisition device to detect suspicious behavior, and means for issuing a warning when suspicious behavior is detected,

[0911] A means for extracting human facial information from the aforementioned video data and comparing it with pre-registered authorized person information to identify abnormal individuals,

[0912] A means of learning environmental data and dynamically adjusting the safety level based on the learning results,

[0913] A means of managing safety within the home and its surroundings, and immediately notifying the user when abnormal behavior is detected,

[0914] A method for pre-registering visitor information and sending an anomaly notification when an unregistered visitor arrives,

[0915] A means to automatically change security sensitivity according to time and weather conditions,

[0916] ...

[0917] A system that includes this.

[0918] (Claim 2)

[0919] The system according to claim 1, which collects data on abnormal person detection events, generates an automatically generated report based on the collected data, and notifies the user of the automatically generated report.

[0920] (Claim 3)

[0921] The system according to claim 1, comprising means for extracting patterns for reducing false alarms based on past environmental data using machine learning.

[0922] "Example 2 of combining an emotion engine"

[0923] (Claim 1)

[0924] The system includes a data processing device that analyzes video data acquired from an image acquisition device to detect suspicious behavior, and means for transmitting warning information when suspicious behavior is detected.

[0925] A means of identifying a suspicious person by extracting facial information from the aforementioned video data and comparing it with authorized personal information stored in advance,

[0926] A means of analyzing a person's emotional state based on video data and dynamically adjusting the level of attention based on an assessment of urgency,

[0927] A means of performing analysis to reduce misinformation and predict potential risks using accumulated emotional data and historical situational data,

[0928] A system that includes this.

[0929] (Claim 2)

[0930] The system according to claim 1, comprising means for collecting information on suspicious person detection events, generating an automatically generated report based on the collected information, and notifying the user of the automatically generated report.

[0931] (Claim 3)

[0932] The system according to claim 1, comprising means for analyzing patterns to reduce false alarms based on past situational data using machine learning techniques.

[0933] "Application example 2 when combining with an emotional engine"

[0934] (Claim 1)

[0935] It has a computing device that analyzes image data acquired from an image acquisition device to detect suspicious behavior, and means for issuing a warning when such suspicious behavior is detected,

[0936] A means for extracting facial information of a person from the aforementioned image data and identifying a suspicious person by comparing it with pre-registered authorized person information,

[0937] A means for learning environmental information and dynamically adjusting the safety level based on the learning results,

[0938] A means of analyzing emotional states from video data and adjusting the alert level based on the results,

[0939] A means of notifying mobile devices of information based on suspicious behavior or emotional state,

[0940] A system that includes this.

[0941] (Claim 2)

[0942] A means for aggregating data on suspicious person detection incidents and generating an automatically generated report based on the aggregated data,

[0943] The system according to claim 1, which notifies the user of the automatically generated report.

[0944] (Claim 3)

[0945] This includes a means of analyzing patterns to reduce false alarms based on past environmental information using machine learning.

[0946] The system according to claim 1. [Explanation of Symbols]

[0947] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. The system includes a computing device that analyzes video data acquired from an image acquisition device to detect suspicious behavior, and means for issuing an alert when such suspicious behavior is detected. A means of identifying a suspicious person by extracting human facial information from the aforementioned video data and comparing it with pre-registered authorized person information, A means of learning from environmental data and dynamically adjusting the security level based on the learning results, A system that includes this.

2. A means for collecting data on suspicious person detection events and generating an automatically generated report based on the collected data, The system according to claim 1, which notifies the user of the automatically generated report.

3. This includes a means of using machine learning to analyze patterns based on past environmental data to reduce false alarms. The system according to claim 1.

Images