system

Abstract

To provide a system that automatically and efficiently identifies suspicious individuals and issues warnings and reports. [Solution] A system including: means for capturing video from a surveillance camera in real time; means for transmitting the captured video at regular intervals; AI means for analyzing the received video and recognizing people in the video; means for comparing the characteristics of the recognized people with a pre-registered database; means for identifying unregistered people as a result of the comparison; means for outputting an alarm signal to the identified unregistered people; means for issuing an alarm with light and sound based on the alarm signal; means for automatically reporting information about the unregistered person to an external reporting destination; and means for saving the series of processes as a log.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Crime prevention measures are important for buildings and facilities, but conventional surveillance camera systems require manual monitoring, which can lead to missed spots and making it difficult to respond quickly to suspicious individuals. Rapid, automatic responses are especially needed to detect intruders late at night or during closed hours. Therefore, a system that can automatically and efficiently identify suspicious individuals and issue alarms and reports is needed. [Means for solving the problem]

[0005] In order to solve the above problem, the present invention provides a system including: means for capturing video from a surveillance camera in real time; means for transmitting the captured video at regular intervals; AI means for analyzing the received video and recognizing people in the video; means for comparing the characteristics of the recognized people with a pre-registered database; means for identifying unregistered people as a result of the comparison; means for outputting an alarm signal to the identified unregistered people; means for issuing an alarm with light and sound based on the alarm signal; means for automatically reporting information about the unregistered person to an external reporting destination; and means for saving the above series of processes as a log.

[0006] A "surveillance camera" is a device that captures real-time images of the inside and outside of a building or facility.

[0007] "Real-time" refers to the ability to process or display data almost immediately.

[0008] "Video" refers to visual information captured by a device such as a camera.

[0009] "Capture" refers to the process of acquiring and saving information such as video or images.

[0010] "Means" refers to a method, device, or process used to perform a particular function or role.

[0011] "Transmitting" refers to moving information from one point to another.

[0012] "Analysis" refers to the act of breaking down data or information, extracting its elements, and understanding them.

[0013] "AI means" refers to a process or device that uses artificial intelligence techniques to perform a specific function.

[0014] "Recognition" refers to the process of grasping and identifying a particular object.

[0015] A "feature" refers to a distinctive attribute or property that distinguishes an object.

[0016] A "database" refers to a system that organizes and manages large amounts of data, making it easy to access, search, and update.

[0017] "Matching" refers to the process of comparing two or more pieces of data to identify matches or differences between them.

[0018] "Identification" refers to the act of distinguishing a particular object from others and clarifying its characteristics.

[0019] "Warning signal" refers to a signal emitted to warn of a particular situation.

[0020] "Optical and acoustic alarm means" refers to a device or process that receives an alarm signal and issues a warning by visual and audible means.

[0021] "Automatic reporting" refers to the system's ability to report when certain conditions are met without manual intervention.

[0022] "External reporting destination" refers to a third-party agency or organization that receives important information such as the detection of suspicious persons.

[0023] "Log" refers to data that records the operations performed by the system and the events that occurred.

[0024] "Storage" refers to the process of retaining data for future reference or use. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0026] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0027] First, the terms used in the following description will be explained.

[0028] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0029] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0030] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0031] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0032] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0033] [First embodiment]

[0034] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0035] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0036] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0037] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0038] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0039] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0040] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0041] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0042] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0043] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0044] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0045] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] This invention is a system that enhances the security of monitored objects by capturing video in real time using a surveillance camera and recognizing and identifying people in the video. This system is composed of a surveillance camera, a server, and a terminal, which operate in cooperation with each other.

[0047] System configuration

[0048] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time. They are connected to devices via a network.

[0049] Terminal: Receives images sent from the surveillance cameras and periodically sends them to the server. It also receives alarm signals from the server and activates light and sound alarms within the building.

[0050] Server: Analyzes the captured video using an AI model and recognizes people in the video. The characteristics of the recognized people are compared with a database to identify unregistered people. If an unregistered person is identified, an alarm signal is sent to the terminal and an automatic report is made to an external reporting point.

[0051] Specific Modes for Carrying Out the Invention

[0052] Video capture and transmission

[0053] The device captures video from the surveillance camera in real time. The captured video is sent to the server frame by frame at regular intervals. This process ensures that the latest surveillance video is always provided to the server in a format that can be analyzed.

[0054] Video analysis and matching

[0055] The server analyzes the received video frames using an AI model. This AI model is based on facial recognition technology and recognizes the faces of people in the video and extracts their features. The extracted features are compared with a pre-registered database, which contains facial data of people involved in the building or facility.

[0056] Identifying suspicious people

[0057] If the result of the check is that there is no matching person in the database, the server identifies the person as an unregistered person (suspicious person), and it is determined that the identified unregistered person requires further action.

[0058] Activation of an alarm

[0059] The server sends an alarm signal to the terminal for any identified unregistered person. When the terminal receives the alarm signal, it immediately activates a light and sound alarm within the building to alert those in the vicinity. This alarm increases the likelihood that the suspicious person will leave the area, and allows those in the vicinity to become more vigilant.

[0060] Automatic reporting

[0061] At the same time, the server automatically sends information about the suspicious person to an external reporting point, including the video frame and the suspicious person's characteristics, so that the security company or relevant security department can respond immediately.

[0062] Log storage

[0063] The server records the results of the series of processes as a log. This log includes the results of video analysis, the identification of suspicious individuals, the issuance of alarms, and the details of reports. The log can be accessed and checked later by an administrator, helping to understand the situation and prevent recurrence.

[0064] Specific examples

[0065] Example 1: If the candidate is eligible, they will pass.

[0066] 1. User: The device captures video from a surveillance camera installed in the building and sends it to the server.

[0067] 2. Server: Receives the video and analyzes the person using an AI model.

[0068] 3. Server: Compares the analyzed person's characteristics with a database and confirms that the person is a registered person (e.g., an employee).

[0069] 4. Server: No alarm signal is generated, so no action is taken by the system. The user can enter the building without any problems.

[0070] Example 2: When a suspicious person passes through

[0071] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[0072] 2. Server: Receives the video and analyzes the person using an AI model.

[0073] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people.

[0074] 4. Server: Sends an alarm signal to the device.

[0075] 5. Terminal: Receives alarm signals and activates light and sound alarms within the building.

[0076] 6. Server: Automatically reports information about suspicious individuals to security companies.

[0077] 7. Server: This process is logged so that an administrator can review it later.

[0078] In this way, the present invention makes it possible to automatically and effectively strengthen the security of buildings and facilities.

[0079] The processing flow will be explained below.

[0080] Step 1:

[0081] The device captures video from the surveillance camera in real time, and the captured video is buffered in the device and awaits the next transmission timing.

[0082] Step 2:

[0083] The device transmits the captured video frames to the server at regular intervals. This transmission is carried out over a network, and the data is efficiently compressed to ensure real-time performance.

[0084] Step 3:

[0085] The server receives the video frames sent from the device, which are then immediately passed to the AI ​​analysis module.

[0086] Step 4:

[0087] The server's AI analysis module identifies people in the video frame and extracts their facial features, using a facial recognition algorithm.

[0088] Step 5:

[0089] The server compares the extracted person's features with a pre-registered facial database of building personnel. During the matching process, it calculates the degree of match between the features and identifies those with a high degree of match.

[0090] Step 6:

[0091] The server uses the results of the comparison to determine whether an unregistered person exists. If there is no match in the database, the person is identified as suspicious.

[0092] Step 7:

[0093] When the server identifies a suspicious person, it generates an alarm signal and sends it to the terminal. The alarm signal includes detailed information about the identified person (e.g., video frames and features).

[0094] Step 8:

[0095] The terminal receives the alarm signal and immediately activates a light and sound alarm within the building, with preset light and sound intensities and patterns.

[0096] Step 9:

[0097] At the same time, the server automatically reports the suspicious person's information to an external reporting point (e.g., a security company or administrator). The report includes important information (such as the suspicious person's characteristics, video frames, and the time of the incident).

[0098] Step 10:

[0099] The server records a detailed log of the process results, including the details of each step and a timestamp, and saves it so that an administrator can review the situation later.

[0100] Example 1

[0101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0102] Conventional security systems often rely on manual monitoring, which can lead to incomplete monitoring and human error. Furthermore, existing video surveillance systems have difficulty recognizing and identifying people in real time, which can result in delays in detecting suspicious individuals. A system that can resolve these issues and perform monitoring more accurately and quickly is needed.

[0103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0104] In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, means for analyzing the received video using a generative AI model and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for identifying unregistered people based on the results of the comparison, means for outputting an alarm signal for the identified unregistered people, means for issuing an alarm with light and sound based on the alarm signal, means for automatically reporting information about the unregistered person to an external reporting destination, and means for saving the series of processes as a log. This enables real-time recognition and identification of people, enabling quick and accurate detection of suspicious people and appropriate response.

[0105] A "surveillance camera" is a device that captures video footage in real time and transmits the video data over a network.

[0106] A "capturing means" is a method or device by which a surveillance camera generates a video stream and captures that video footage.

[0107] "Transmitting means" refers to a method or device for transmitting captured video data to a server via a network at regular intervals.

[0108] A "generative AI model" is an algorithm or program that uses machine learning techniques to recognize and identify people.

[0109] The "analyzing means" refers to a method or device for analyzing received video data and recognizing people in the video.

[0110] A "means for extracting features" is a method or device for extracting identifiable features such as the face or posture of a recognized person.

[0111] A "database" is a storage device or system that stores facial data and feature data of registered people and uses them for matching purposes.

[0112] The "matching means" refers to a method or device for comparing extracted feature data with registered data in a database to find a matching person.

[0113] The "means for identifying" refers to a method or device for detecting an unregistered person identified as a result of the matching and recognizing him or her as a suspicious person.

[0114] The "means for outputting an alert signal" refers to a method or device for generating and transmitting a signal to alert an identified unregistered person.

[0115] The "means for activating an alarm" refers to a method or device for actually activating an alarm using light and sound within the building based on the alarm signal received by the terminal.

[0116] "Means for reporting" refers to a method or device for automatically sending information about a suspicious person to an external reporting destination.

[0117] "Means for saving as a log" refers to a method or device for recording the results of a series of processes and saving them so that they can be checked later.

[0118] "Real-time" refers to the state in which video is processed immediately after it is captured, with almost no delay.

[0119] MODE FOR CARRYING OUT THE INVENTION

[0120] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video, thereby enhancing the security of the monitored object. This system is composed of surveillance cameras, a server, and a terminal, and each device operates in cooperation with each other.

[0121] System configuration

[0122] Surveillance cameras: These cameras are installed inside and outside buildings and facilities and capture video footage in real time. They are connected to devices via a network.

[0123] Terminal: Receives images sent from the surveillance camera and periodically sends them to the server. It also receives alarm signals from the server and activates alarms using light and sound.

[0124] Server: Analyzes the captured video using a generative AI model to recognize people in the video. It compares the characteristics of the recognized people with a database to identify unregistered people. If an unregistered person is identified, it sends an alarm signal to the device and automatically reports the incident to an external reporting point.

[0125] Hardware and Software

[0126] Surveillance cameras: Network cameras (e.g. IP cameras) are used.

[0127] Terminal: A computing device (e.g., PC, Raspberry Pi) for processing and transmitting video.

[0128] Server: Use a high-performance server (e.g., AWS EC2, on-premise server) and database (e.g., MySQL).

[0129] Generative AI model: Uses libraries for AI analysis (e.g., TENSORFLOW (registered trademark), OpenCV).

[0130] Video capture and transmission

[0131] The device captures video from the surveillance camera in real time. The captured video is sent to the server frame by frame at regular intervals (for example, 30 frames per second). This process ensures that the latest surveillance video is always provided to the server in a format that can be analyzed.

[0132] Video analysis and matching

[0133] The server analyzes the received video frames using a generative AI model. This AI model is based on facial recognition technology and recognizes the faces of people in the video and extracts their features. The extracted features are compared with a pre-registered database, which contains facial data of people involved in the building or facility.

[0134] Identifying suspicious people

[0135] If the result of the check is that there is no matching person in the database, the server identifies the person as an unregistered person (suspicious person), and it is determined that the identified unregistered person requires further action.

[0136] Activation of an alarm

[0137] The server sends an alarm signal to the terminal for any identified unregistered person. When the terminal receives the alarm signal, it immediately activates a light and sound alarm within the building to alert those in the vicinity. This alarm increases the likelihood that the suspicious person will leave the area, and allows those in the vicinity to become more vigilant.

[0138] Automatic reporting

[0139] At the same time, the server automatically sends information about the suspicious person to an external reporting organization (e.g., a security company or the police), which can then immediately respond by sending a report containing the video frame and the suspicious person's characteristics.

[0140] Log storage

[0141] The server records the results of the series of processes as a log. This log includes the results of video analysis, the identification of suspicious individuals, the issuance of alarms, and the details of reports. The log can be accessed and checked later by an administrator, helping to understand the situation and prevent recurrence.

[0142] Specific examples

[0143] Example 1: If the candidate is eligible, they will pass.

[0144] 1. Terminal: Captures video from surveillance cameras installed in the building and sends it to the server.

[0145] 2. Server: Receives the video and analyzes the person using a generative AI model.

[0146] 3. Server: The analyzed person's characteristics are compared with the database to confirm that the person is a registered person (for example, an employee).

[0147] 4. Server: No alarm signal is generated, so no action is taken by the system. As a result, the user can enter the building without any problems.

[0148] Example 2: When a suspicious person passes through

[0149] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[0150] 2. Server: Receives the video and analyzes the person using a generative AI model.

[0151] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people (suspicious individuals).

[0152] 4. Server: Sends an alarm signal to the device.

[0153] 5. Terminal: Receives alarm signals and activates light and sound alarms within the building.

[0154] 6. Server: Automatically reports information about suspicious individuals to security companies. Specifically, the report is made by sending an HTTP request to the security company's API.

[0155] 7. Server: This process is logged so that an administrator can review it later.

[0156] Prompt Sentence Examples

[0157] "Please explain how a system can capture video in real time from security cameras installed in an office building, analyze the video using a generative AI model (e.g., OpenCV), identify suspicious individuals, and then automatically report this information to a security company."

[0158] By using this prompt sentence, the generative AI model can be made to explain the specific processing details of the system.

[0159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0160] Step 1:

[0161] The terminal captures video from a surveillance camera in real time. The input is the video stream from the surveillance camera, and the output is the captured video frames. Specifically, the terminal acquires the stream based on the IP address of the surveillance camera, captures the frames, and stores them in memory.

[0162] Step 2:

[0163] The device sends the captured video to the server at regular intervals (e.g., 30 frames per second). The input is the captured video frame, and the output is the transmission to the server. Specifically, the device uses HTTP or WebSocket to packetize the frame data and send it to the server.

[0164] Step 3:

[0165] The server receives video frames sent from the terminal. The input is the video frame from the terminal, and the output is the storage of the received frame. Specifically, the server receives packets through the network interface, reconstructs them into frames, and stores them in memory.

[0166] Step 4:

[0167] The server analyzes the received video frames using a generative AI model (e.g., TensorFlow or OpenCV). The input is the stored frame, and the output is the analyzed person's facial feature data. Specifically, the server applies a face detection algorithm to the frame, extracts facial feature points, and stores them as vectors.

[0168] Step 5:

[0169] The server compares the extracted feature data with the database. The input is the extracted vector data, and the output is the comparison result (registered or unregistered). Specifically, the server uses an SQL query to compare the person feature vectors in the database and calculate the similarity.

[0170] Step 6:

[0171] The server identifies an unregistered person as a result of the matching. The input is the matching result, and the output is the identification result of the unregistered person. Specifically, if the similarity threshold is not exceeded, the server identifies the person as unregistered and stores that information in a structure.

[0172] Step 7:

[0173] The server sends an alarm signal to the terminal for the identified unregistered person. The input is the identification result of the unregistered person, and the output is the alarm signal to the terminal. Specifically, the server sends an alarm signal data packet to the terminal using the TCP / IP protocol.

[0174] Step 8:

[0175] The terminal receives the alarm signal and activates the light and sound alarms in the building. The input is the alarm signal from the server, and the output is the actual activation of the alarm sound and light. Specifically, the terminal analyzes the received signal and sends a signal to the circuitry that controls the speaker and light.

[0176] Step 9:

[0177] The server automatically reports information about unregistered individuals to external reporting destinations. The input is the identification result of the unregistered individual and related data, and the output is a report to the external reporting destination. Specifically, the server uses an API request or email sending function to send the information to the specified reporting destination.

[0178] Step 10:

[0179] The server saves the results of a series of processes as a log. The input is the result data of each process, and the output is the saved log file. Specifically, the server records the results of the process in text format or database format so that they can be checked later.

[0180] (Application example 1)

[0181] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0182] While conventional security systems have the ability to detect suspicious individuals inside and outside a building in real time and issue an alarm, they face the challenge of making it difficult for administrators to monitor and control the system from a remote location. Furthermore, when an unregistered individual is identified, the response can be delayed, which is insufficient for modern security needs that require a rapid response. There is a demand for a security system that can solve these issues and provide a more effective and prompt response.

[0183] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0184] In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, AI means for analyzing the received video and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for identifying unregistered people based on the comparison result, means for outputting an alarm signal for the identified unregistered people, means for issuing an alarm with light and sound based on the alarm signal, means for automatically reporting information about the unregistered people to an external reporting destination, means for saving the series of processes as a log, means for viewing the real-time video on a smartphone, means for sending a push notification to the smartphone when an unregistered person is detected, and means for remotely issuing an alarm from the smartphone. This allows an administrator to monitor in real time even from a remote location and immediately issue an alarm.

[0185] A "surveillance camera" is a device that captures real-time footage of the inside and outside of a building or facility.

[0186] "Real-time video" refers to video footage captured by a surveillance camera.

[0187] "Capturing" means taking and recording video.

[0188] "Sending at regular intervals" means sending the captured video to the server periodically.

[0189] "Analyze" means to analyze the video and understand its content.

[0190] "AI means" refers to a method or device for recognizing people in video using artificial intelligence technology.

[0191] The "pre-registered database" refers to a database that holds pre-registered face image data.

[0192] "Matching" means comparing the characteristics of a recognized person with data in a database.

[0193] "Identify" refers to identifying a person based on specific criteria.

[0194] An "alarm signal" is a signal that is transmitted when an abnormality is detected.

[0195] "Activating an alarm with light and sound" refers to using light and sound to notify of an abnormality.

[0196] "Automatically report to the reporting destination" means automatically reporting an abnormality to a designated external agency.

[0197] "Saving as a log" refers to recording and saving the results of processing and the history of events.

[0198] A "smartphone" is a mobile device that can connect to the Internet and run applications.

[0199] "Push notification" refers to the function of sending notifications from a server to a smartphone in real time.

[0200] "Remotely activating an alarm" means remotely operating and activating an alarm from a distance.

[0201] System configuration

[0202] The system for implementing the present invention comprises the following main hardware and software components:

[0203] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time, which is then transmitted to a server over a network.

[0204] Server: Receives the captured video and analyzes it using an AI model. The server compares the feature data with a database to identify unregistered individuals.

[0205] Smartphone: Runs an application that allows administrators to view video footage in real time and remotely control alarms.

[0206] Cloud services: Utilize cloud services such as AWS and Google® Cloud for video analysis and data management.

[0207] Program processing explanation

[0208] Video capture and transmission

[0209] Surveillance cameras capture images of the inside and outside of buildings in real time. The captured images are periodically sent to a server, frame by frame. Because the image data is sent over a network, the communication environment between the camera and the server is important.

[0210] Video analysis and matching

[0211] The server analyzes the received video using an AI model. Specifically, it detects people in the video frame and extracts their facial features. This analysis uses a facial recognition algorithm using OpenCV and Dlib. The extracted facial features are compared with a pre-registered database, and if there is no match, the person is identified as unregistered.

[0212] Alerts and notifications

[0213] When an unregistered person is identified, the server generates an alarm signal, triggering light and sound alarms within the building, and sends a push notification to the administrator's smartphone, immediately informing them of the abnormality. The push notification is sent using Firebase cloud messaging.

[0214] Log saving and reporting

[0215] The server saves the results of the series of processes as a log. The saved log includes the results of video analysis, the identification of suspicious individuals, and the history of alarm activation. This allows administrators to check the details later. In addition, information on unregistered individuals is automatically sent to external reporting points.

[0216] Specific examples

[0217] Example 1: If the candidate is eligible, they will pass.

[0218] 1. The user (surveillance camera) captures video from inside the facility and sends it to the server.

[0219] 2. The server uses an AI model to analyze the person and match them with a database.

[0220] 3. If the match is with a registered person, no alarm will be triggered and the system will operate normally.

[0221] Example 2: When a suspicious person passes through

[0222] 1. The user (surveillance camera) captures video from inside the facility and sends it to the server.

[0223] 2. The server uses an AI model to analyze the person and match them with a database.

[0224] 3. If an unregistered person is identified, the server generates an alarm signal and triggers an alarm.

[0225] 4. At the same time, the server sends a push notification to the administrator's smartphone and also sends the information to external reporting points such as security companies.

[0226] Prompt Sentence Examples

[0227] For example, by inputting the following prompt sentence into the generative AI model, you can get advice and adjustments to improve the accuracy of surveillance camera video analysis.

[0228] "We are developing an AI system that analyzes surveillance camera footage and performs person recognition. Could you please tell us some techniques to improve accuracy in identifying unregistered people by accurately capturing the characteristics of people in the footage?"

[0229] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0230] Step 1:

[0231] A user (surveillance camera) captures video of the inside and outside of a building in real time. The captured video is converted into frames within the camera. At that time, the video frames are given a time stamp.

[0232] Input: Real-time video

[0233] Output: Captured video frames

[0234] Step 2:

[0235] The device periodically transmits captured video frames to the server, where they are encoded and split into packets over the network.

[0236] Input: Captured video frames

[0237] Output: Video packets sent to the server

[0238] Step 3:

[0239] The server decodes the received video packets and reconstructs them into the original video frames, which are then stored in a buffer.

[0240] Input: Video packets sent to the server

[0241] Output: Decoded and buffered video frames

[0242] Step 4:

[0243] The server uses an AI model to analyze the video frames in the buffer and detect faces in the video, extracting facial feature points using OpenCV and the Dlib library.

[0244] Input: Buffered video frames

[0245] Output: Detected facial feature points

[0246] Step 5:

[0247] The server compares the extracted facial feature points with pre-registered face data in a database, and calculates the similarity between the feature vectors.

[0248] Input: Detected facial feature points, registered face data in the database

[0249] Output: Matching result (registered or unregistered)

[0250] Step 6:

[0251] If the matched person is an unregistered person, the server generates an alarm signal, which includes facial feature information and a video frame of the unregistered person.

[0252] Input: Matching result is unregistered person

[0253] Output: Alarm signal

[0254] Step 7:

[0255] The terminal receives the alarm signal and activates a light and sound alarm within the building. The alarm is visually and audibly activated using a speaker and a light.

[0256] Input: Alarm signal

[0257] Output: Light and sound alarm activation

[0258] Step 8:

[0259] The server simultaneously sends an alarm signal to the administrator's smartphone as a push notification, using Firebase cloud messaging to deliver the notification in real time.

[0260] Input: Alarm signal

[0261] Output: Push notification to smartphone

[0262] Step 9:

[0263] The server automatically reports information about unregistered individuals to an external reporting point, including video frames and details of the alert.

[0264] Input: Unregistered person information, alarm signal

[0265] Output: Automatic notification to external reporting destination

[0266] Step 10:

[0267] The server saves the results of the series of processes as a log, which includes video analysis results, alarm activation history, suspicious person information, and report history, and can be checked later by the administrator.

[0268] Input: A set of processing results

[0269] Output: Saved logs

[0270] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0271] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video, thereby enhancing the security of the monitored object. In addition, by combining it with an emotion engine using AI technology, it is possible to identify the emotional state of suspicious individuals, realizing more accurate crime prevention measures. This system is composed of surveillance cameras, a server, a terminal, and an emotion engine, which operate in cooperation with each other.

[0272] System configuration

[0273] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time. They are connected to devices via a network.

[0274] Terminal: Receives images sent from the surveillance cameras and periodically sends them to the server. It also receives alarm signals from the server and activates light and sound alarms within the building.

[0275] Server: Analyzes the captured video using an AI model to recognize people in the video. It compares the features of the recognized people with a database to identify unregistered people. It also analyzes the emotional state of the people using an emotion engine.

[0276] Emotion Engine: Analyzes facial expressions and movements of people in the video to determine their emotional state in real time, and issues additional alerts or notifications as needed based on the detected emotional state.

[0277] Specific Modes for Carrying Out the Invention

[0278] Video capture and transmission

[0279] The device captures video from the surveillance camera in real time. The captured video is sent to the server frame by frame at regular intervals. This process ensures that the latest surveillance video is always provided to the server in a format that can be analyzed.

[0280] Video analysis and matching

[0281] The server analyzes the received video frames using an AI model. This AI model is based on facial recognition technology and recognizes the faces of people in the video and extracts their features. The extracted features are compared with a pre-registered database, which contains facial data of people involved in the building or facility.

[0282] Emotion analysis using an emotion engine

[0283] In parallel with the facial recognition results, the server uses an emotion engine to analyze the emotional state of people in the video, which determines specific emotional states such as tension, anxiety, or anger based on the person's facial expressions and movements.

[0284] Identifying suspicious people and emotional states

[0285] By combining the matching results with the results of emotion analysis, the server identifies unregistered individuals and evaluates their emotional state. If abnormal emotional states, such as tension, anxiety, or anger, are detected, the server determines that more immediate action is required.

[0286] Activation of an alarm

[0287] The server generates an alarm signal according to the identified unregistered person and their emotional state and transmits it to the terminal. For example, if an unregistered person in a very tense state is identified, the terminal will issue a stronger alarm than usual.

[0288] Automatic reporting

[0289] At the same time, the server automatically reports the suspicious person's information and emotional state to an external reporting point (e.g., a security company or administrator). The report includes video frames, characteristic information, and the person's emotional state, allowing the security company to respond immediately.

[0290] Log storage

[0291] The server records a detailed log of the entire process, including the results of video analysis, the identification of suspicious individuals, their emotional state, whether an alarm was issued, and the details of the report, and the log is saved so that administrators can review the situation later.

[0292] Specific examples

[0293] Example 1: If the candidate is eligible, they will pass.

[0294] 1. User: The device captures video from a surveillance camera installed in the building and sends it to the server.

[0295] 2. Server: Receives the video and analyzes the person using the AI ​​model. The emotion engine detects the normal state.

[0296] 3. Server: Compares the analyzed person's characteristics with the database and verifies that the person is registered.

[0297] 4. Server: No alarm signal is generated, so no action is taken by the system. The user can enter the building without any problems.

[0298] Example 2: When a suspicious person with an abnormal emotional state passes by

[0299] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[0300] 2. Server: Receives the video and analyzes the person using an AI model. The emotion engine detects when the person is extremely nervous.

[0301] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people.

[0302] 4. Server: Having identified an unregistered person in a tense state, the server sends an alarm signal to the terminal. The server then sends the information to an external reporting point.

[0303] 5. Terminal: Receives the alarm signal and activates stronger than normal light and sound alarms within the building.

[0304] 6. Server: Reports information about suspicious individuals and the state of tension to the security company.

[0305] 7. Server: This process is logged so that an administrator can review it later.

[0306] In this way, the present invention automatically and effectively strengthens the security of buildings and facilities, and by taking into account emotional states in particular, more advanced crime prevention measures become possible.

[0307] The processing flow will be explained below.

[0308] Step 1:

[0309] The device captures video from the surveillance camera in real time, and the captured video is temporarily stored in the device, waiting for the next transmission timing.

[0310] Step 2:

[0311] The device transmits captured video frames to the server at regular intervals, and the transmitted data is compressed to reduce network load.

[0312] Step 3:

[0313] The server processes video frames received from the device in real time. The video frames are first passed to the AI ​​analysis module.

[0314] Step 4:

[0315] The server's AI analysis module identifies people in the video frame and extracts their facial features, using facial recognition algorithms.

[0316] Step 5:

[0317] The server then matches the extracted person's characteristics with existing data in a database that contains building personnel information to identify matches.

[0318] Step 6:

[0319] The server identifies unregistered individuals based on the matching results. If there is no matching person in the database, the individual is recognized as an unregistered (suspicious) individual.

[0320] Step 7:

[0321] In parallel, the server runs an emotion engine that analyzes the facial expressions and movements of people in the video, using AI to identify emotional states such as tension, anxiety, or anger.

[0322] Step 8:

[0323] The server integrates the analysis results of the emotion engine with the registered information of the person and checks for the presence of an unregistered person with an abnormal emotional state.

[0324] Step 9:

[0325] The server generates an alarm signal for an unregistered person with an abnormal emotional state and sends it to the terminal, which includes the analyzed feature information and the emotional state.

[0326] Step 10:

[0327] The terminal receives the alarm signal and immediately activates a light and sound alarm within the building, which is stronger than normal and alerts those around.

[0328] Step 11:

[0329] At the same time, the server automatically sends the information about the unregistered person and the emotion analysis results to an external reporting point (e.g., a security company or facility manager). The report includes detailed characteristic information and the person's emotional state.

[0330] Step 12:

[0331] The server saves all processing results as logs, which contain detailed records of timestamps, processing details, alarm activation history, and report details. Administrators can later check and analyze these logs.

[0332] In this way, the present invention effectively links surveillance cameras, terminals, servers, and emotion engines to automatically achieve real-time surveillance and anomaly detection with high accuracy.

[0333] Example 2

[0334] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0335] Conventional security systems are equipped with video surveillance using surveillance cameras and identification functions using facial recognition, but lack the ability to detect abnormalities based on a person's emotional state. As a result, it is difficult to detect abnormal emotional states, such as when a suspicious person is nervous or anxious, which reduces the accuracy of security measures. In addition, there is also the issue of not only simply detecting unregistered individuals, but also needing to respond appropriately according to the person's emotional state.

[0336] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, AI means for analyzing the received video and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for analyzing the emotional state of people in the video in real time using emotion analysis means and detecting abnormal emotional states, and means for generating an additional alarm signal in response to the detected abnormal emotional state. This makes it possible not only to detect unregistered people but also to detect abnormalities based on emotional states and respond quickly.

[0337] A "surveillance camera" is a device installed inside or outside a building or facility to capture video in real time.

[0338] The "terminal" is a device or software that receives images sent from the surveillance camera and periodically sends them to the server. The terminal also receives alarm signals from the server and activates light and sound alarms within the building.

[0339] The "server" is a computer system that analyzes the captured video and recognizes people in the video. The server also performs emotion analysis to assess the emotional state of people and issue alerts if necessary.

[0340] The "Emotion Engine" is a system that uses AI technology to analyze the facial expressions and movements of people in video footage and determine their emotional state in real time.

[0341] An "AI model" is an algorithm or software that uses artificial intelligence technology to recognize the faces of people in video and extract their features.

[0342] A "database" is a data storage that stores characteristic information (face data) of people who have been registered in advance.

[0343] An "alarm signal" is a signal sent to a terminal to trigger an alarm in response to an identified unregistered person and their abnormal emotional state.

[0344] "Light and sound alarms" are visual and audible warnings that are activated within a building based on an alarm signal, such as flashing lights and audible alarms.

[0345] "Automatic reporting" is a process in which the server notifies an external contact point (such as a security company or administrator) of information about a suspicious person and their emotional state. Notifications are sent via email, text message, or API communication.

[0346] A "log" is a detailed record of a series of processing results, and is data that is saved for later review by an administrator. Logs include analysis results, identification of suspicious individuals, emotional states, triggered alarms, and report contents.

[0347] MODE FOR CARRYING OUT THE INVENTION

[0348] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video. Furthermore, by combining it with an emotion engine using AI technology, it can identify the emotional state of suspicious individuals and achieve highly accurate crime prevention measures. This system is composed of surveillance cameras, a server, a terminal, and an emotion engine, which work in cooperation with each other.

[0349] System configuration

[0350] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time. They are connected to devices via a network.

[0351] Terminal: Receives images sent from the surveillance cameras and periodically sends them to the server. It also receives alarm signals from the server and activates light and sound alarms within the building.

[0352] Server: Analyzes the captured video using an AI model to recognize people in the video. It compares the features of the recognized people with a database to identify unregistered people. It also analyzes the emotional state of the people using an emotion engine.

[0353] Emotion Engine: Analyzes facial expressions and movements of people in the video to determine their emotional state in real time, and issues additional alerts or notifications as needed based on the detected emotional state.

[0354] Specific Modes for Carrying Out the Invention

[0355] The server has a means to capture video from the surveillance cameras in real time. Specifically, the surveillance cameras continuously capture video inside and outside the building or facility and send the video data to the terminal. The terminal converts this data into capture frames at regular intervals and sends them to the server via the network.

[0356] The server analyzes the received video frames using an AI model. This AI model uses libraries such as OpenCV and Dlib to recognize people's faces and extract their features. The extracted features are compared with a pre-registered database containing facial data of people associated with the building or facility. If the comparison identifies an unregistered person, the server generates an alarm signal.

[0357] Additionally, the server uses an emotion engine to analyze the emotional state of people in the video in real time. This emotion engine uses services such as Microsoft® Azure® Cognitive Services and Google Cloud Vision to analyze facial expressions and movements to identify emotions such as tension, anxiety, and anger. If an abnormal emotional state is detected, the server generates an additional alarm signal, enabling a rapid response.

[0358] The generated alarm signal is sent to a terminal, which activates the building's alarm system (e.g., audio alarm or flashing light), and the server automatically reports information about the suspicious person and their emotional state to an external reporting point via email, text message, or API communication.

[0359] Finally, the server records a detailed log of the entire process, including the results of video analysis, the identification of suspicious individuals, their emotional state, whether an alarm was activated, and the details of the report, which is then saved for later review by an administrator.

[0360] Specific examples

[0361] Example 1: If the candidate is eligible, they will pass.

[0362] 1. User: The device captures video from a surveillance camera installed in the building and sends it to the server.

[0363] 2. Server: Receives the video and analyzes the person using an AI model. The emotion engine detects the normal state.

[0364] 3. Server: Compares the analyzed person's characteristics with the database and verifies that the person is registered.

[0365] 4. Server: No alarm signal is generated, so no action is taken by the system. The user can enter the building without any problems.

[0366] Example 2: When a suspicious person with an abnormal emotional state passes by

[0367] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[0368] 2. Server: Receives the video and analyzes the person using an AI model. The emotion engine detects when the person is extremely nervous.

[0369] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people.

[0370] 4. Server: Having identified an unregistered person in a tense state, the server sends an alarm signal to the terminal. The server then sends the information to an external reporting point.

[0371] 5. Terminal: Receives the alarm signal and activates stronger than normal light and sound alarms within the building.

[0372] 6. Server: Reports information about suspicious individuals and the state of tension to the security company.

[0373] 7. Server: This process is logged so that an administrator can review it later.

[0374] Prompt Sentence Examples

[0375] Example prompt sentence:

[0376] "Analyze the emotional state of a person in a video captured by a surveillance camera late at night. Explain how the system would respond if it detected an abnormal level of tension."

[0377] In this way, the present invention automatically and effectively strengthens the security of buildings and facilities, and by taking into account emotional states in particular, more advanced crime prevention measures become possible.

[0378] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0379] Step 1:

[0380] The device captures video from a surveillance camera in real time and sends it to the server frame by frame at regular intervals. The input to this process is the video data from the surveillance camera, and the output is the captured frames sent to the server. This process requires a network connection to transfer the video data to the server via the Internet.

[0381] Step 2:

[0382] The server receives video frames sent from the surveillance camera. The input to this process is the video frames sent from the device, and the output is video data in an analyzable format. The received video data is temporarily stored in the server's storage and then proceeds to the next analysis step.

[0383] Step 3:

[0384] The server uses an AI model to analyze the received video frames and recognize the faces of people in the video. The input to this process is the received video frames, and the output is identified facial feature data. Specifically, an AI model (e.g., OpenCV or Dlib library) is used to run a face detection algorithm and extract the location and feature points of the face.

[0385] Step 4:

[0386] The server compares the feature data extracted by facial recognition with a pre-registered database. The input of this process is the identified facial feature data, and the output is the matching result. For matching, the server checks whether the face data matches the face data in the database, and if it does, it determines that the person is registered.

[0387] Step 5:

[0388] The server uses an emotion analysis engine to analyze the emotional state from the extracted facial feature data. The input for this process is the facial feature data after matching, and the output is the emotional state determination result. The emotion analysis engine uses APIs such as Microsoft Azure Cognitive Services and Google Cloud Vision to identify emotions such as tension, anxiety, and anger from facial expressions.

[0389] Step 6:

[0390] The server identifies unregistered individuals and abnormal emotional states based on the analysis results. The input to this process is the results of facial recognition and emotional analysis, and the output is alarm data containing suspicious individuals and their emotional states. If an abnormal emotional state is detected, the server determines that the individual is suspicious and requires immediate action.

[0391] Step 7:

[0392] The server generates an alarm signal according to the identified unregistered person and their emotional state, and transmits it to the terminal. The input of this process is the alarm data, and the output is the alarm signal. Once the alarm signal is generated, it is transmitted to the terminal.

[0393] Step 8:

[0394] The terminal receives the alarm signal and activates optical and audible alarms in the building. The input of this process is the received alarm signal and the output is the activated alarm. The terminal activates the alarm system and provides visual and audible warnings.

[0395] Step 9:

[0396] The server automatically reports information about suspicious individuals and their emotional state to an external reporting destination. The input to this process is the suspicious individual's information and emotional state, and the output is the sent report message. Reports are sent via email, text message, or API communication.

[0397] Step 10:

[0398] The server records a detailed log of the results of each process, with the input being the results of each process step and the output being a saved log file, allowing administrators to review and respond to the situation at a later time.

[0399] (Application example 2)

[0400] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0401] Conventional surveillance systems have the ability to recognize people in real time using video transmitted from surveillance cameras and issue alarms, but this alone is insufficient to determine the emotional state of a suspicious individual, limiting the accuracy of security responses. Conventional systems cannot respond when a suspicious individual has a calm expression or when preventive measures must be taken before the individual engages in suspicious behavior. To solve this problem, a system is needed that can analyze emotional states in real time and respond immediately. Therefore, the present invention aims to achieve more accurate crime prevention measures by instantly identifying the emotional state of a suspicious individual and issuing an alarm.

[0402] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0403] In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, AI means for analyzing the received video and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for identifying unregistered people based on the comparison result, means for outputting an alarm signal to the identified unregistered people, means for issuing an alarm with light and sound based on the alarm signal, means for automatically reporting information about the unregistered people to an external reporting destination, means for saving the series of processes as a log, means for displaying the faces of people in the video on a display mounted on the smart glasses, and means for analyzing the emotional state of the displayed people and issuing an alarm signal based on the emotional state. This enables the emotional state of people to be identified in real time, enabling prompt and appropriate crime prevention measures.

[0404] - A "surveillance camera" is a device installed to detect suspicious activity and for crime prevention purposes, and is used to capture real-time images of specific locations and acquire the footage.

[0405] "Real-time" refers to processing and responding to an event the moment it occurs, without delay.

[0406] "Video" means visual information captured by a camera or other photographic device, and refers to moving images displayed on a monitor or the like.

[0407] "Capture" refers to the operation or process of acquiring an image using a camera or other photographic device.

[0408] "Transmit" refers to the act of transferring acquired data or information to another device or system.

[0409] "Analysis" refers to the process of analyzing video and data in detail using computers and specific algorithms to extract the necessary information.

[0410] "AI means" refers to systems or devices that use artificial intelligence technology to perform processing or make decisions according to specific purposes.

[0411] "Features" refer to the distinctive attributes or patterns necessary to identify a person or object.

[0412] A "database" is a collection of information that systematically organizes related data and is constructed to enable efficient searching and updating.

[0413] "Verification" refers to the process of checking whether newly entered data matches the data and information acquired.

[0414] An "unregistered person" refers to a person whose information has not been registered in the database in advance.

[0415] "Distinguishing" means recognizing a particular object as distinct from others.

[0416] An "alarm signal" is a signal that the system emits when it detects an abnormality, and includes audio and light warnings.

[0417] "Light" refers to visible light, a type of electromagnetic wave, which is used as a visually recognizable warning.

[0418] "Sound" is a physical phenomenon that is transmitted through vibrations in the air and is used as an alarm that can be recognized through the ears.

[0419] "Reporting" refers to the act of informing an external agency or person in charge of an abnormality or emergency.

[0420] A "log" is data that records all the processes performed and events that occur while the system is running.

[0421] "Smart glasses" are glasses-type devices that have built-in displays, cameras, and communication functions, and have the ability to display information and acquire images.

[0422] A "display" is a display device for displaying images and text information.

[0423] "Emotional state" refers to the type and intensity of emotions a person is experiencing, such as tension, anxiety, anger, or other psychological conditions.

[0424] "Analysis means" refers to a method or device for analyzing specific data, and in this system includes emotion analysis using AI.

[0425] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video, thereby enhancing the security of the monitored object. In addition, by combining it with an emotion engine using AI technology, it is possible to identify the emotional state of suspicious individuals, realizing more accurate crime prevention measures. This system is composed of surveillance cameras, a server, a terminal, and an emotion engine, which operate in cooperation with each other.

[0426] System configuration

[0427] Surveillance cameras: These are devices installed inside and outside buildings and facilities that capture video in real time. Surveillance cameras are connected to servers and terminals via a network.

[0428] Server: This is the central device that analyzes the captured video using an AI model and recognizes people in the video. It compares the characteristics of the recognized people with a database to identify unregistered people. It also analyzes the person's emotional state using an emotion engine.

[0429] Terminal: A device that receives images sent from surveillance cameras and periodically sends them to a server, and also receives alarm signals from the server and activates light and sound alarms within the building.

[0430] Emotion Engine: This engine analyzes the facial expressions and movements of people in the video to determine their emotional state in real time. Based on the detected emotional state, it issues additional alerts or reports as needed.

[0431] Specific program description

[0432] The server performs video analysis using software such as Python, OpenCV, dlib, and Keras. Images captured by surveillance cameras are first captured in real time and then sent to the server at regular intervals. The server analyzes the received images, recognizes people's faces using an AI model, and determines their emotional state using an emotion engine. The analysis results are displayed on the smart glasses or a display.

[0433] Hardware used:

[0434] 1. Surveillance cameras

[0435] 2. Server

[0436] 3. Terminal

[0437] 4. Smart Glasses

[0438] Software used:

[0439] 1. Python

[0440] 2. OpenCV

[0441] 3. dlib

[0442] 4. Keras

[0443] Specific examples

[0444] Example 1: If the candidate is eligible, they will pass.

[0445] When a user enters a building, the device captures video from the security cameras installed in the building and sends it to the server. The server receives the video and analyzes the person using an AI model, and an emotion engine detects their normal state. The analyzed person's characteristics are compared with a database to confirm that they are a registered person. No alarm signal is issued, so the system takes no action. The user can enter the building without any problems.

[0446] Example 2: When a suspicious person with an abnormal emotional state passes by

[0447] Late at night, the device captures footage sent from the surveillance camera and sends it to the server. The server receives the footage and analyzes the person using an AI model, and the emotion engine detects that the person is in a very tense state. The analyzed person's characteristics are compared with a database to identify unregistered people. Having identified an unregistered person in a tense state, an alarm signal is sent to the device. The server then transmits the information to an external reporting point. The device receives the alarm signal and activates a stronger-than-normal light and sound alarm within the building. The server then reports information about the suspicious person and their state of tension to the security company. This entire process is recorded in a log, allowing an administrator to review the situation later.

[0448] Prompt Sentence Examples

[0449] 1. "Load an image dataset and crop it to center on a person's face."

[0450] 2. "Train a CNN model with emotion labels (e.g., anger, neutral, happy)."

[0451] 3. "Save the emotion recognition model and implement it so that it can analyze video input in real time."

[0452] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0453] System program processing flow

[0454] Step 1:

[0455] Capture and Send

[0456] Description: The device captures video from a surveillance camera in real time and transmits it to a server.

[0457] Input: Real-time video captured from a surveillance camera.

[0458] Output: Sending video data to the server.

[0459] How it works: A surveillance camera monitors a specific area and captures the video in real time, then periodically sends each frame of the captured video to a server.

[0460] Step 2:

[0461] Video reception and pre-processing

[0462] Description: The server receives the video sent from the terminal and performs preprocessing.

[0463] Input: Video data sent from the device.

[0464] Output: Preprocessed video data.

[0465] How it works: The server temporarily stores the received video data and performs preprocessing such as noise removal and resolution adjustment. This preprocessing ensures that subsequent AI analysis can be performed with high accuracy.

[0466] Step 3:

[0467] person recognition

[0468] Description: The server analyzes the pre-processed footage and uses AI means to recognize people within the footage.

[0469] Input: Preprocessed video data.

[0470] Output: Information about the recognized person.

[0471] Operation: For video analysis, a face detection algorithm (e.g., Haar-cascade or dlib) is used to recognize human faces in the video. Feature data of the recognized faces is extracted.

[0472] Step 4:

[0473] Feature Matching

[0474] Description: The server checks the characteristics of the recognized person against a pre-registered database.

[0475] Input: Recognized person feature data,database.

[0476] Output: Matching result (enrolled or not enrolled).

[0477] Operation: The feature data of a person obtained through analysis is compared with the feature data of a person previously registered in a database. This comparison identifies whether the person is registered or not.

[0478] Step 5:

[0479] Emotion analysis

[0480] Description: In parallel with the facial recognition results, the server uses an emotion engine to analyze the emotional state of people in the video.

[0481] Input: Recognized person's facial features, emotion engine.

[0482] Output: Emotional state (tension, anxiety, anger, etc.).

[0483] Movement: The emotion engine analyzes facial expressions and movements from facial feature data, labels the analyzed emotional states, and identifies specific emotional states such as tension, anxiety, and anger.

[0484] Step 6:

[0485] Determining and issuing an alarm

[0486] Description: The server generates an alarm signal according to the identified unregistered person and their emotional state and sends it to the terminal.

[0487] Input: Matching results, sentiment analysis results.

[0488] Output: Alarm signal.

[0489] Operation: If an unregistered person is detected and an abnormal emotional state (e.g., tension or anger) is detected, the server generates an alarm signal and sends it to the device. The device receives this signal and activates a light and sound alarm.

[0490] Step 7:

[0491] Reporting and logging

[0492] Description: The server automatically reports information about unregistered people and their emotional state to an external reporting destination, and saves the results of the process as a log.

[0493] Input: Information about unregistered persons, contact information.

[0494] Output: Report data, log data.

[0495] How it works: The server reports information about suspicious individuals and their emotional state in real time to security companies, administrators, and other reporting points. It also saves the results of this process in detail as a log so that administrators can check the situation later.

[0496] In this way, through each processing step, advanced crime prevention measures based on video data are realized.

[0497] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0498] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0499] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0500] [Second embodiment]

[0501] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0502] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0503] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0504] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0505] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0506] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0507] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0508] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0509] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0510] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0511] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0512] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0513] This invention is a system that enhances the security of monitored objects by capturing video in real time using a surveillance camera and recognizing and identifying people in the video. This system is composed of a surveillance camera, a server, and a terminal, which operate in cooperation with each other.

[0514] System configuration

[0515] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time. They are connected to devices via a network.

[0516] Terminal: Receives images sent from the surveillance cameras and periodically sends them to the server. It also receives alarm signals from the server and activates light and sound alarms within the building.

[0517] Server: Analyzes the captured video using an AI model and recognizes people in the video. The characteristics of the recognized people are compared with a database to identify unregistered people. If an unregistered person is identified, an alarm signal is sent to the terminal and an automatic report is made to an external reporting point.

[0518] Specific Modes for Carrying Out the Invention

[0519] Video capture and transmission

[0520] The device captures video from the surveillance camera in real time. The captured video is sent to the server frame by frame at regular intervals. This process ensures that the latest surveillance video is always provided to the server in a format that can be analyzed.

[0521] Video analysis and matching

[0522] The server analyzes the received video frames using an AI model. This AI model is based on facial recognition technology and recognizes the faces of people in the video and extracts their features. The extracted features are compared with a pre-registered database, which contains facial data of people involved in the building or facility.

[0523] Identifying suspicious people

[0524] If the result of the check is that there is no matching person in the database, the server identifies the person as an unregistered person (suspicious person), and it is determined that the identified unregistered person requires further action.

[0525] Activation of an alarm

[0526] The server sends an alarm signal to the terminal for any identified unregistered person. When the terminal receives the alarm signal, it immediately activates a light and sound alarm within the building to alert those in the vicinity. This alarm increases the likelihood that the suspicious person will leave the area, and allows those in the vicinity to become more vigilant.

[0527] Automatic reporting

[0528] At the same time, the server automatically sends information about the suspicious person to an external reporting point, including the video frame and the suspicious person's characteristics, so that the security company or relevant security department can respond immediately.

[0529] Log storage

[0530] The server records the results of the series of processes as a log. This log includes the results of video analysis, the identification of suspicious individuals, the issuance of alarms, and the details of reports. The log can be accessed and checked later by an administrator, helping to understand the situation and prevent recurrence.

[0531] Specific examples

[0532] Example 1: If the candidate is eligible, they will pass.

[0533] 1. User: The device captures video from a surveillance camera installed in the building and sends it to the server.

[0534] 2. Server: Receives the video and analyzes the person using an AI model.

[0535] 3. Server: Compares the analyzed person's characteristics with a database and confirms that the person is a registered person (e.g., an employee).

[0536] 4. Server: No alarm signal is generated, so no action is taken by the system. The user can enter the building without any problems.

[0537] Example 2: When a suspicious person passes through

[0538] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[0539] 2. Server: Receives the video and analyzes the person using an AI model.

[0540] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people.

[0541] 4. Server: Sends an alarm signal to the device.

[0542] 5. Terminal: Receives alarm signals and activates light and sound alarms within the building.

[0543] 6. Server: Automatically reports information about suspicious individuals to security companies.

[0544] 7. Server: This process is logged so that an administrator can review it later.

[0545] In this way, the present invention makes it possible to automatically and effectively strengthen the security of buildings and facilities.

[0546] The processing flow will be explained below.

[0547] Step 1:

[0548] The device captures video from the surveillance camera in real time, and the captured video is buffered in the device and awaits the next transmission timing.

[0549] Step 2:

[0550] The device transmits the captured video frames to the server at regular intervals. This transmission is carried out over a network, and the data is efficiently compressed to ensure real-time performance.

[0551] Step 3:

[0552] The server receives the video frames sent from the device, which are then immediately passed to the AI ​​analysis module.

[0553] Step 4:

[0554] The server's AI analysis module identifies people in the video frame and extracts their facial features, using a facial recognition algorithm.

[0555] Step 5:

[0556] The server compares the extracted person's features with a pre-registered facial database of building personnel. During the matching process, it calculates the degree of match between the features and identifies those with a high degree of match.

[0557] Step 6:

[0558] The server uses the results of the comparison to determine whether an unregistered person exists. If there is no match in the database, the person is identified as suspicious.

[0559] Step 7:

[0560] When the server identifies a suspicious person, it generates an alarm signal and sends it to the terminal. The alarm signal includes detailed information about the identified person (e.g., video frames and features).

[0561] Step 8:

[0562] The terminal receives the alarm signal and immediately activates a light and sound alarm within the building, with preset light and sound intensities and patterns.

[0563] Step 9:

[0564] At the same time, the server automatically reports the suspicious person's information to an external reporting point (e.g., a security company or administrator). The report includes important information (such as the suspicious person's characteristics, video frames, and the time of the incident).

[0565] Step 10:

[0566] The server records a detailed log of the process results, including the details of each step and a timestamp, and saves it so that an administrator can review the situation later.

[0567] Example 1

[0568] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0569] Conventional security systems often rely on manual monitoring, which can lead to incomplete monitoring and human error. Furthermore, existing video surveillance systems have difficulty recognizing and identifying people in real time, which can result in delays in detecting suspicious individuals. A system that can resolve these issues and perform monitoring more accurately and quickly is needed.

[0570] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0571] In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, means for analyzing the received video using a generative AI model and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for identifying unregistered people based on the results of the comparison, means for outputting an alarm signal for the identified unregistered people, means for issuing an alarm with light and sound based on the alarm signal, means for automatically reporting information about the unregistered person to an external reporting destination, and means for saving the series of processes as a log. This enables real-time recognition and identification of people, enabling quick and accurate detection of suspicious people and appropriate response.

[0572] A "surveillance camera" is a device that captures video footage in real time and transmits the video data over a network.

[0573] A "capturing means" is a method or device by which a surveillance camera generates a video stream and captures that video footage.

[0574] "Transmitting means" refers to a method or device for transmitting captured video data to a server via a network at regular intervals.

[0575] A "generative AI model" is an algorithm or program that uses machine learning techniques to recognize and identify people.

[0576] The "analyzing means" refers to a method or device for analyzing received video data and recognizing people in the video.

[0577] A "means for extracting features" is a method or device for extracting identifiable features such as the face or posture of a recognized person.

[0578] A "database" is a storage device or system that stores facial data and feature data of registered people and uses them for matching purposes.

[0579] The "matching means" refers to a method or device for comparing extracted feature data with registered data in a database to find a matching person.

[0580] The "means for identifying" refers to a method or device for detecting an unregistered person identified as a result of the matching and recognizing him or her as a suspicious person.

[0581] The "means for outputting an alert signal" refers to a method or device for generating and transmitting a signal to alert an identified unregistered person.

[0582] The "means for activating an alarm" refers to a method or device for actually activating an alarm using light and sound within the building based on the alarm signal received by the terminal.

[0583] "Means for reporting" refers to a method or device for automatically sending information about a suspicious person to an external reporting destination.

[0584] "Means for saving as a log" refers to a method or device for recording the results of a series of processes and saving them so that they can be checked later.

[0585] "Real-time" refers to the state in which video is processed immediately after it is captured, with almost no delay.

[0586] MODE FOR CARRYING OUT THE INVENTION

[0587] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video, thereby enhancing the security of the monitored object. This system is composed of surveillance cameras, a server, and a terminal, and each device operates in cooperation with each other.

[0588] System configuration

[0589] Surveillance cameras: These cameras are installed inside and outside buildings and facilities and capture video footage in real time. They are connected to devices via a network.

[0590] Terminal: Receives images sent from the surveillance camera and periodically sends them to the server. It also receives alarm signals from the server and activates alarms using light and sound.

[0591] Server: Analyzes the captured video using a generative AI model to recognize people in the video. It compares the characteristics of the recognized people with a database to identify unregistered people. If an unregistered person is identified, it sends an alarm signal to the device and automatically reports the incident to an external reporting point.

[0592] Hardware and Software

[0593] Surveillance cameras: Network cameras (e.g. IP cameras) are used.

[0594] Terminal: A computing device (e.g., PC, Raspberry Pi) for processing and transmitting video.

[0595] Server: Use a high-performance server (e.g., AWS EC2, on-premise server) and database (e.g., MySQL).

[0596] Generative AI models: Use libraries for AI analysis (e.g., TensorFlow, OpenCV).

[0597] Video capture and transmission

[0598] The device captures video from the surveillance camera in real time. The captured video is sent to the server frame by frame at regular intervals (for example, 30 frames per second). This process ensures that the latest surveillance video is always provided to the server in a format that can be analyzed.

[0599] Video analysis and matching

[0600] The server analyzes the received video frames using a generative AI model. This AI model is based on facial recognition technology and recognizes the faces of people in the video and extracts their features. The extracted features are compared with a pre-registered database, which contains facial data of people involved in the building or facility.

[0601] Identifying suspicious people

[0602] If the result of the check is that there is no matching person in the database, the server identifies the person as an unregistered person (suspicious person), and it is determined that the identified unregistered person requires further action.

[0603] Activation of an alarm

[0604] The server sends an alarm signal to the terminal for any identified unregistered person. When the terminal receives the alarm signal, it immediately activates a light and sound alarm within the building to alert those in the vicinity. This alarm increases the likelihood that the suspicious person will leave the area, and allows those in the vicinity to become more vigilant.

[0605] Automatic reporting

[0606] At the same time, the server automatically sends information about the suspicious person to an external reporting organization (e.g., a security company or the police), which can then immediately respond by sending a report containing the video frame and the suspicious person's characteristics.

[0607] Log storage

[0608] The server records the results of the series of processes as a log. This log includes the results of video analysis, the identification of suspicious individuals, the issuance of alarms, and the details of reports. The log can be accessed and checked later by an administrator, helping to understand the situation and prevent recurrence.

[0609] Specific examples

[0610] Example 1: If the candidate is eligible, they will pass.

[0611] 1. Terminal: Captures video from surveillance cameras installed in the building and sends it to the server.

[0612] 2. Server: Receives the video and analyzes the person using a generative AI model.

[0613] 3. Server: The analyzed person's characteristics are compared with the database to confirm that the person is a registered person (for example, an employee).

[0614] 4. Server: No alarm signal is generated, so no action is taken by the system. As a result, the user can enter the building without any problems.

[0615] Example 2: When a suspicious person passes through

[0616] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[0617] 2. Server: Receives the video and analyzes the person using a generative AI model.

[0618] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people (suspicious individuals).

[0619] 4. Server: Sends an alarm signal to the device.

[0620] 5. Terminal: Receives alarm signals and activates light and sound alarms within the building.

[0621] 6. Server: Automatically reports information about suspicious individuals to security companies. Specifically, the report is made by sending an HTTP request to the security company's API.

[0622] 7. Server: This process is logged so that an administrator can review it later.

[0623] Prompt Sentence Examples

[0624] "Please explain how a system can capture video in real time from security cameras installed in an office building, analyze the video using a generative AI model (e.g., OpenCV), identify suspicious individuals, and then automatically report this information to a security company."

[0625] By using this prompt sentence, the generative AI model can be made to explain the specific processing details of the system.

[0626] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0627] Step 1:

[0628] The terminal captures video from a surveillance camera in real time. The input is the video stream from the surveillance camera, and the output is the captured video frames. Specifically, the terminal acquires the stream based on the IP address of the surveillance camera, captures the frames, and stores them in memory.

[0629] Step 2:

[0630] The device sends the captured video to the server at regular intervals (e.g., 30 frames per second). The input is the captured video frame, and the output is the transmission to the server. Specifically, the device uses HTTP or WebSocket to packetize the frame data and send it to the server.

[0631] Step 3:

[0632] The server receives video frames sent from the terminal. The input is the video frame from the terminal, and the output is the storage of the received frame. Specifically, the server receives packets through the network interface, reconstructs them into frames, and stores them in memory.

[0633] Step 4:

[0634] The server analyzes the received video frames using a generative AI model (e.g., TensorFlow or OpenCV). The input is the stored frame, and the output is the analyzed person's facial feature data. Specifically, the server applies a face detection algorithm to the frame, extracts facial feature points, and stores them as vectors.

[0635] Step 5:

[0636] The server compares the extracted feature data with the database. The input is the extracted vector data, and the output is the comparison result (registered or unregistered). Specifically, the server uses an SQL query to compare the person feature vectors in the database and calculate the similarity.

[0637] Step 6:

[0638] The server identifies an unregistered person as a result of the matching. The input is the matching result, and the output is the identification result of the unregistered person. Specifically, if the similarity threshold is not exceeded, the server identifies the person as unregistered and stores that information in a structure.

[0639] Step 7:

[0640] The server sends an alarm signal to the terminal for the identified unregistered person. The input is the identification result of the unregistered person, and the output is the alarm signal to the terminal. Specifically, the server sends an alarm signal data packet to the terminal using the TCP / IP protocol.

[0641] Step 8:

[0642] The terminal receives the alarm signal and activates the light and sound alarms in the building. The input is the alarm signal from the server, and the output is the actual activation of the alarm sound and light. Specifically, the terminal analyzes the received signal and sends a signal to the circuitry that controls the speaker and light.

[0643] Step 9:

[0644] The server automatically reports information about unregistered individuals to external reporting destinations. The input is the identification result of the unregistered individual and related data, and the output is a report to the external reporting destination. Specifically, the server uses an API request or email sending function to send the information to the specified reporting destination.

[0645] Step 10:

[0646] The server saves the results of a series of processes as a log. The input is the result data of each process, and the output is the saved log file. Specifically, the server records the results of the process in text format or database format so that they can be checked later.

[0647] (Application example 1)

[0648] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0649] While conventional security systems have the ability to detect suspicious individuals inside and outside a building in real time and issue an alarm, they face the challenge of making it difficult for administrators to monitor and control the system from a remote location. Furthermore, when an unregistered individual is identified, the response can be delayed, which is insufficient for modern security needs that require a rapid response. There is a demand for a security system that can solve these issues and provide a more effective and prompt response.

[0650] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0651] In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, AI means for analyzing the received video and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for identifying unregistered people based on the comparison result, means for outputting an alarm signal for the identified unregistered people, means for issuing an alarm with light and sound based on the alarm signal, means for automatically reporting information about the unregistered people to an external reporting destination, means for saving the series of processes as a log, means for viewing the real-time video on a smartphone, means for sending a push notification to the smartphone when an unregistered person is detected, and means for remotely issuing an alarm from the smartphone. This allows an administrator to monitor in real time even from a remote location and immediately issue an alarm.

[0652] A "surveillance camera" is a device that captures real-time footage of the inside and outside of a building or facility.

[0653] "Real-time video" refers to video footage captured by a surveillance camera.

[0654] "Capturing" means taking and recording video.

[0655] "Sending at regular intervals" means sending the captured video to the server periodically.

[0656] "Analyze" means to analyze the video and understand its content.

[0657] "AI means" refers to a method or device for recognizing people in video using artificial intelligence technology.

[0658] The "pre-registered database" refers to a database that holds pre-registered face image data.

[0659] "Matching" means comparing the characteristics of a recognized person with data in a database.

[0660] "Identify" refers to identifying a person based on specific criteria.

[0661] An "alarm signal" is a signal that is transmitted when an abnormality is detected.

[0662] "Activating an alarm with light and sound" refers to using light and sound to notify of an abnormality.

[0663] "Automatically report to the reporting destination" means automatically reporting an abnormality to a designated external agency.

[0664] "Saving as a log" refers to recording and saving the results of processing and the history of events.

[0665] A "smartphone" is a mobile device that can connect to the Internet and run applications.

[0666] "Push notification" refers to the function of sending notifications from a server to a smartphone in real time.

[0667] "Remotely activating an alarm" means remotely operating and activating an alarm from a distance.

[0668] System configuration

[0669] The system for implementing the present invention comprises the following main hardware and software components:

[0670] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time, which is then transmitted to a server over a network.

[0671] Server: Receives the captured video and analyzes it using an AI model. The server compares the feature data with a database to identify unregistered individuals.

[0672] Smartphone: Runs an application that allows administrators to view video footage in real time and remotely control alarms.

[0673] Cloud services: Utilize cloud services such as AWS and Google Cloud for video analysis and data management.

[0674] Program processing explanation

[0675] Video capture and transmission

[0676] Surveillance cameras capture images of the inside and outside of buildings in real time. The captured images are periodically sent to a server, frame by frame. Because the image data is sent over a network, the communication environment between the camera and the server is important.

[0677] Video analysis and matching

[0678] The server analyzes the received video using an AI model. Specifically, it detects people in the video frame and extracts their facial features. This analysis uses a facial recognition algorithm using OpenCV and Dlib. The extracted facial features are compared with a pre-registered database, and if there is no match, the person is identified as unregistered.

[0679] Alerts and notifications

[0680] When an unregistered person is identified, the server generates an alarm signal, triggering light and sound alarms within the building, and sends a push notification to the administrator's smartphone, immediately informing them of the abnormality. The push notification is sent using Firebase cloud messaging.

[0681] Log saving and reporting

[0682] The server saves the results of the series of processes as a log. The saved log includes the results of video analysis, the identification of suspicious individuals, and the history of alarm activation. This allows administrators to check the details later. In addition, information on unregistered individuals is automatically sent to external reporting points.

[0683] Specific examples

[0684] Example 1: If the candidate is eligible, they will pass.

[0685] 1. The user (surveillance camera) captures video from inside the facility and sends it to the server.

[0686] 2. The server uses an AI model to analyze the person and match them with a database.

[0687] 3. If the match is with a registered person, no alarm will be triggered and the system will operate normally.

[0688] Example 2: When a suspicious person passes through

[0689] 1. The user (surveillance camera) captures video from inside the facility and sends it to the server.

[0690] 2. The server uses an AI model to analyze the person and match them with a database.

[0691] 3. If an unregistered person is identified, the server generates an alarm signal and triggers an alarm.

[0692] 4. At the same time, the server sends a push notification to the administrator's smartphone and also sends the information to external reporting points such as security companies.

[0693] Prompt Sentence Examples

[0694] For example, by inputting the following prompt sentence into the generative AI model, you can get advice and adjustments to improve the accuracy of surveillance camera video analysis.

[0695] "We are developing an AI system that analyzes surveillance camera footage and performs person recognition. Could you please tell us some techniques to improve accuracy in identifying unregistered people by accurately capturing the characteristics of people in the footage?"

[0696] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0697] Step 1:

[0698] A user (surveillance camera) captures video of the inside and outside of a building in real time. The captured video is converted into frames within the camera. At that time, the video frames are given a time stamp.

[0699] Input: Real-time video

[0700] Output: Captured video frames

[0701] Step 2:

[0702] The device periodically transmits captured video frames to the server, where they are encoded and split into packets over the network.

[0703] Input: Captured video frames

[0704] Output: Video packets sent to the server

[0705] Step 3:

[0706] The server decodes the received video packets and reconstructs them into the original video frames, which are then stored in a buffer.

[0707] Input: Video packets sent to the server

[0708] Output: Decoded and buffered video frames

[0709] Step 4:

[0710] The server uses an AI model to analyze the video frames in the buffer and detect faces in the video, extracting facial feature points using OpenCV and the Dlib library.

[0711] Input: Buffered video frames

[0712] Output: Detected facial feature points

[0713] Step 5:

[0714] The server compares the extracted facial feature points with pre-registered face data in a database, and calculates the similarity between the feature vectors.

[0715] Input: Detected facial feature points, registered face data in the database

[0716] Output: Matching result (registered or unregistered)

[0717] Step 6:

[0718] If the matched person is an unregistered person, the server generates an alarm signal, which includes facial feature information and a video frame of the unregistered person.

[0719] Input: Matching result is unregistered person

[0720] Output: Alarm signal

[0721] Step 7:

[0722] The terminal receives the alarm signal and activates a light and sound alarm within the building. The alarm is visually and audibly activated using a speaker and a light.

[0723] Input: Alarm signal

[0724] Output: Light and sound alarm activation

[0725] Step 8:

[0726] The server simultaneously sends an alarm signal to the administrator's smartphone as a push notification, using Firebase cloud messaging to deliver the notification in real time.

[0727] Input: Alarm signal

[0728] Output: Push notification to smartphone

[0729] Step 9:

[0730] The server automatically reports information about unregistered individuals to an external reporting point, including video frames and details of the alert.

[0731] Input: Unregistered person information, alarm signal

[0732] Output: Automatic notification to external reporting destination

[0733] Step 10:

[0734] The server saves the results of the series of processes as a log, which includes video analysis results, alarm activation history, suspicious person information, and report history, and can be checked later by the administrator.

[0735] Input: A set of processing results

[0736] Output: Saved logs

[0737] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0738] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video, thereby enhancing the security of the monitored object. In addition, by combining it with an emotion engine using AI technology, it is possible to identify the emotional state of suspicious individuals, realizing more accurate crime prevention measures. This system is composed of surveillance cameras, a server, a terminal, and an emotion engine, which operate in cooperation with each other.

[0739] System configuration

[0740] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time. They are connected to devices via a network.

[0741] Terminal: Receives images sent from the surveillance cameras and periodically sends them to the server. It also receives alarm signals from the server and activates light and sound alarms within the building.

[0742] Server: Analyzes the captured video using an AI model to recognize people in the video. It compares the features of the recognized people with a database to identify unregistered people. It also analyzes the emotional state of the people using an emotion engine.

[0743] Emotion Engine: Analyzes facial expressions and movements of people in the video to determine their emotional state in real time, and issues additional alerts or notifications as needed based on the detected emotional state.

[0744] Specific Modes for Carrying Out the Invention

[0745] Video capture and transmission

[0746] The device captures video from the surveillance camera in real time. The captured video is sent to the server frame by frame at regular intervals. This process ensures that the latest surveillance video is always provided to the server in a format that can be analyzed.

[0747] Video analysis and matching

[0748] The server analyzes the received video frames using an AI model. This AI model is based on facial recognition technology and recognizes the faces of people in the video and extracts their features. The extracted features are compared with a pre-registered database, which contains facial data of people involved in the building or facility.

[0749] Emotion analysis using an emotion engine

[0750] In parallel with the facial recognition results, the server uses an emotion engine to analyze the emotional state of people in the video, which determines specific emotional states such as tension, anxiety, or anger based on the person's facial expressions and movements.

[0751] Identifying suspicious people and emotional states

[0752] By combining the matching results with the results of emotion analysis, the server identifies unregistered individuals and evaluates their emotional state. If abnormal emotional states, such as tension, anxiety, or anger, are detected, the server determines that more immediate action is required.

[0753] Activation of an alarm

[0754] The server generates an alarm signal according to the identified unregistered person and their emotional state and transmits it to the terminal. For example, if an unregistered person in a very tense state is identified, the terminal will issue a stronger alarm than usual.

[0755] Automatic reporting

[0756] At the same time, the server automatically reports the suspicious person's information and emotional state to an external reporting point (e.g., a security company or administrator). The report includes video frames, characteristic information, and the person's emotional state, allowing the security company to respond immediately.

[0757] Log storage

[0758] The server records a detailed log of the entire process, including the results of video analysis, the identification of suspicious individuals, their emotional state, whether an alarm was issued, and the details of the report, and the log is saved so that administrators can review the situation later.

[0759] Specific examples

[0760] Example 1: If the candidate is eligible, they will pass.

[0761] 1. User: The device captures video from a surveillance camera installed in the building and sends it to the server.

[0762] 2. Server: Receives the video and analyzes the person using the AI ​​model. The emotion engine detects the normal state.

[0763] 3. Server: Compares the analyzed person's characteristics with the database and verifies that the person is registered.

[0764] 4. Server: No alarm signal is generated, so no action is taken by the system. The user can enter the building without any problems.

[0765] Example 2: When a suspicious person with an abnormal emotional state passes by

[0766] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[0767] 2. Server: Receives the video and analyzes the person using an AI model. The emotion engine detects when the person is extremely nervous.

[0768] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people.

[0769] 4. Server: Having identified an unregistered person in a tense state, the server sends an alarm signal to the terminal. The server then sends the information to an external reporting point.

[0770] 5. Terminal: Receives the alarm signal and activates stronger than normal light and sound alarms within the building.

[0771] 6. Server: Reports information about suspicious individuals and the state of tension to the security company.

[0772] 7. Server: This process is logged so that an administrator can review it later.

[0773] In this way, the present invention automatically and effectively strengthens the security of buildings and facilities, and by taking into account emotional states in particular, more advanced crime prevention measures become possible.

[0774] The processing flow will be explained below.

[0775] Step 1:

[0776] The device captures video from the surveillance camera in real time, and the captured video is temporarily stored in the device, waiting for the next transmission timing.

[0777] Step 2:

[0778] The device transmits captured video frames to the server at regular intervals, and the transmitted data is compressed to reduce network load.

[0779] Step 3:

[0780] The server processes video frames received from the device in real time. The video frames are first passed to the AI ​​analysis module.

[0781] Step 4:

[0782] The server's AI analysis module identifies people in the video frame and extracts their facial features, using facial recognition algorithms.

[0783] Step 5:

[0784] The server then matches the extracted person's characteristics with existing data in a database that contains building personnel information to identify matches.

[0785] Step 6:

[0786] The server identifies unregistered individuals based on the matching results. If there is no matching person in the database, the individual is recognized as an unregistered (suspicious) individual.

[0787] Step 7:

[0788] In parallel, the server runs an emotion engine that analyzes the facial expressions and movements of people in the video, using AI to identify emotional states such as tension, anxiety, or anger.

[0789] Step 8:

[0790] The server integrates the analysis results of the emotion engine with the registered information of the person and checks for the presence of an unregistered person with an abnormal emotional state.

[0791] Step 9:

[0792] The server generates an alarm signal for an unregistered person with an abnormal emotional state and sends it to the terminal, which includes the analyzed feature information and the emotional state.

[0793] Step 10:

[0794] The terminal receives the alarm signal and immediately activates a light and sound alarm within the building, which is stronger than normal and alerts those around.

[0795] Step 11:

[0796] At the same time, the server automatically sends the information about the unregistered person and the emotion analysis results to an external reporting point (e.g., a security company or facility manager). The report includes detailed characteristic information and the person's emotional state.

[0797] Step 12:

[0798] The server saves all processing results as logs, which contain detailed records of timestamps, processing details, alarm activation history, and report details. Administrators can later check and analyze these logs.

[0799] In this way, the present invention effectively links surveillance cameras, terminals, servers, and emotion engines to automatically achieve real-time surveillance and anomaly detection with high accuracy.

[0800] Example 2

[0801] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0802] Conventional security systems are equipped with video surveillance using surveillance cameras and identification functions using facial recognition, but lack the ability to detect abnormalities based on a person's emotional state. As a result, it is difficult to detect abnormal emotional states, such as when a suspicious person is nervous or anxious, which reduces the accuracy of security measures. In addition, there is also the issue of not only simply detecting unregistered individuals, but also needing to respond appropriately according to the person's emotional state.

[0803] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, AI means for analyzing the received video and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for analyzing the emotional state of people in the video in real time using emotion analysis means and detecting abnormal emotional states, and means for generating an additional alarm signal in response to the detected abnormal emotional state. This makes it possible not only to detect unregistered people but also to detect abnormalities based on emotional states and respond quickly.

[0804] A "surveillance camera" is a device installed inside or outside a building or facility to capture video in real time.

[0805] The "terminal" is a device or software that receives images sent from the surveillance camera and periodically sends them to the server. The terminal also receives alarm signals from the server and activates light and sound alarms within the building.

[0806] The "server" is a computer system that analyzes the captured video and recognizes people in the video. The server also performs emotion analysis to assess the emotional state of people and issue alerts if necessary.

[0807] The "Emotion Engine" is a system that uses AI technology to analyze the facial expressions and movements of people in video footage and determine their emotional state in real time.

[0808] An "AI model" is an algorithm or software that uses artificial intelligence technology to recognize the faces of people in video and extract their features.

[0809] A "database" is a data storage that stores characteristic information (face data) of people who have been registered in advance.

[0810] An "alarm signal" is a signal sent to a terminal to trigger an alarm in response to an identified unregistered person and their abnormal emotional state.

[0811] "Light and sound alarms" are visual and audible warnings that are activated within a building based on an alarm signal, such as flashing lights and audible alarms.

[0812] "Automatic reporting" is a process in which the server notifies an external contact point (such as a security company or administrator) of information about a suspicious person and their emotional state. Notifications are sent via email, text message, or API communication.

[0813] A "log" is a detailed record of a series of processing results, and is data that is saved for later review by an administrator. Logs include analysis results, identification of suspicious individuals, emotional states, triggered alarms, and report contents.

[0814] MODE FOR CARRYING OUT THE INVENTION

[0815] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video. Furthermore, by combining it with an emotion engine using AI technology, it can identify the emotional state of suspicious individuals and achieve highly accurate crime prevention measures. This system is composed of surveillance cameras, a server, a terminal, and an emotion engine, which work in cooperation with each other.

[0816] System configuration

[0817] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time. They are connected to devices via a network.

[0818] Terminal: Receives images sent from the surveillance cameras and periodically sends them to the server. It also receives alarm signals from the server and activates light and sound alarms within the building.

[0819] Server: Analyzes the captured video using an AI model to recognize people in the video. It compares the features of the recognized people with a database to identify unregistered people. It also analyzes the emotional state of the people using an emotion engine.

[0820] Emotion Engine: Analyzes facial expressions and movements of people in the video to determine their emotional state in real time, and issues additional alerts or notifications as needed based on the detected emotional state.

[0821] Specific Modes for Carrying Out the Invention

[0822] The server has a means to capture video from the surveillance cameras in real time. Specifically, the surveillance cameras continuously capture video inside and outside the building or facility and send the video data to the terminal. The terminal converts this data into capture frames at regular intervals and sends them to the server via the network.

[0823] The server analyzes the received video frames using an AI model. This AI model uses libraries such as OpenCV and Dlib to recognize people's faces and extract their features. The extracted features are compared with a pre-registered database containing facial data of people associated with the building or facility. If the comparison identifies an unregistered person, the server generates an alarm signal.

[0824] Additionally, the server uses an emotion engine to analyze the emotional state of people in the video in real time. This emotion engine, powered by services such as Microsoft Azure Cognitive Services and Google Cloud Vision, analyzes facial expressions and movements to identify emotions such as tension, anxiety, and anger. If an abnormal emotional state is detected, the server generates an additional alarm signal, enabling a rapid response.

[0825] The generated alarm signal is sent to a terminal, which activates the building's alarm system (e.g., audio alarm or flashing light), and the server automatically reports information about the suspicious person and their emotional state to an external reporting point via email, text message, or API communication.

[0826] Finally, the server records a detailed log of the entire process, including the results of video analysis, the identification of suspicious individuals, their emotional state, whether an alarm was activated, and the details of the report, which is then saved for later review by an administrator.

[0827] Specific examples

[0828] Example 1: If the candidate is eligible, they will pass.

[0829] 1. User: The device captures video from a surveillance camera installed in the building and sends it to the server.

[0830] 2. Server: Receives the video and analyzes the person using an AI model. The emotion engine detects the normal state.

[0831] 3. Server: Compares the analyzed person's characteristics with the database and verifies that the person is registered.

[0832] 4. Server: No alarm signal is generated, so no action is taken by the system. The user can enter the building without any problems.

[0833] Example 2: When a suspicious person with an abnormal emotional state passes by

[0834] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[0835] 2. Server: Receives the video and analyzes the person using an AI model. The emotion engine detects when the person is extremely nervous.

[0836] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people.

[0837] 4. Server: Having identified an unregistered person in a tense state, the server sends an alarm signal to the terminal. The server then sends the information to an external reporting point.

[0838] 5. Terminal: Receives the alarm signal and activates stronger than normal light and sound alarms within the building.

[0839] 6. Server: Reports information about suspicious individuals and the state of tension to the security company.

[0840] 7. Server: This process is logged so that an administrator can review it later.

[0841] Prompt Sentence Examples

[0842] Example prompt sentence:

[0843] "Analyze the emotional state of a person in a video captured by a surveillance camera late at night. Explain how the system would respond if it detected an abnormal level of tension."

[0844] In this way, the present invention automatically and effectively strengthens the security of buildings and facilities, and by taking into account emotional states in particular, more advanced crime prevention measures become possible.

[0845] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0846] Step 1:

[0847] The device captures video from a surveillance camera in real time and sends it to the server frame by frame at regular intervals. The input to this process is the video data from the surveillance camera, and the output is the captured frames sent to the server. This process requires a network connection to transfer the video data to the server via the Internet.

[0848] Step 2:

[0849] The server receives video frames sent from the surveillance camera. The input to this process is the video frames sent from the device, and the output is video data in an analyzable format. The received video data is temporarily stored in the server's storage and then proceeds to the next analysis step.

[0850] Step 3:

[0851] The server uses an AI model to analyze the received video frames and recognize the faces of people in the video. The input to this process is the received video frames, and the output is identified facial feature data. Specifically, an AI model (e.g., OpenCV or Dlib library) is used to run a face detection algorithm and extract the location and feature points of the face.

[0852] Step 4:

[0853] The server compares the feature data extracted by facial recognition with a pre-registered database. The input of this process is the identified facial feature data, and the output is the matching result. For matching, the server checks whether the face data matches the face data in the database, and if it does, it determines that the person is registered.

[0854] Step 5:

[0855] The server uses an emotion analysis engine to analyze the emotional state from the extracted facial feature data. The input for this process is the facial feature data after matching, and the output is the emotional state determination result. The emotion analysis engine uses APIs such as Microsoft Azure Cognitive Services and Google Cloud Vision to identify emotions such as tension, anxiety, and anger from facial expressions.

[0856] Step 6:

[0857] The server identifies unregistered individuals and abnormal emotional states based on the analysis results. The input to this process is the results of facial recognition and emotional analysis, and the output is alarm data containing suspicious individuals and their emotional states. If an abnormal emotional state is detected, the server determines that the individual is suspicious and requires immediate action.

[0858] Step 7:

[0859] The server generates an alarm signal according to the identified unregistered person and their emotional state, and transmits it to the terminal. The input of this process is the alarm data, and the output is the alarm signal. Once the alarm signal is generated, it is transmitted to the terminal.

[0860] Step 8:

[0861] The terminal receives the alarm signal and activates optical and audible alarms in the building. The input of this process is the received alarm signal and the output is the activated alarm. The terminal activates the alarm system and provides visual and audible warnings.

[0862] Step 9:

[0863] The server automatically reports information about suspicious individuals and their emotional state to an external reporting destination. The input to this process is the suspicious individual's information and emotional state, and the output is the sent report message. Reports are sent via email, text message, or API communication.

[0864] Step 10:

[0865] The server records a detailed log of the results of each process, with the input being the results of each process step and the output being a saved log file, allowing administrators to review and respond to the situation at a later time.

[0866] (Application example 2)

[0867] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0868] Conventional surveillance systems have the ability to recognize people in real time using video transmitted from surveillance cameras and issue alarms, but this alone is insufficient to determine the emotional state of a suspicious individual, limiting the accuracy of security responses. Conventional systems cannot respond when a suspicious individual has a calm expression or when preventive measures must be taken before the individual engages in suspicious behavior. To solve this problem, a system is needed that can analyze emotional states in real time and respond immediately. Therefore, the present invention aims to achieve more accurate crime prevention measures by instantly identifying the emotional state of a suspicious individual and issuing an alarm.

[0869] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0870] In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, AI means for analyzing the received video and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for identifying unregistered people based on the comparison result, means for outputting an alarm signal to the identified unregistered people, means for issuing an alarm with light and sound based on the alarm signal, means for automatically reporting information about the unregistered people to an external reporting destination, means for saving the series of processes as a log, means for displaying the faces of people in the video on a display mounted on the smart glasses, and means for analyzing the emotional state of the displayed people and issuing an alarm signal based on the emotional state. This enables the emotional state of people to be identified in real time, enabling prompt and appropriate crime prevention measures.

[0871] - A "surveillance camera" is a device installed to detect suspicious activity and for crime prevention purposes, and is used to capture real-time images of specific locations and acquire the footage.

[0872] "Real-time" refers to processing and responding to an event the moment it occurs, without delay.

[0873] "Video" means visual information captured by a camera or other photographic device, and refers to moving images displayed on a monitor or the like.

[0874] "Capture" refers to the operation or process of acquiring an image using a camera or other photographic device.

[0875] "Transmit" refers to the act of transferring acquired data or information to another device or system.

[0876] "Analysis" refers to the process of analyzing video and data in detail using computers and specific algorithms to extract the necessary information.

[0877] "AI means" refers to systems or devices that use artificial intelligence technology to perform processing or make decisions according to specific purposes.

[0878] "Features" refer to the distinctive attributes or patterns necessary to identify a person or object.

[0879] A "database" is a collection of information that systematically organizes related data and is constructed to enable efficient searching and updating.

[0880] "Verification" refers to the process of checking whether newly entered data matches the data and information acquired.

[0881] An "unregistered person" refers to a person whose information has not been registered in the database in advance.

[0882] "Distinguishing" means recognizing a particular object as distinct from others.

[0883] An "alarm signal" is a signal that the system emits when it detects an abnormality, and includes audio and light warnings.

[0884] "Light" refers to visible light, a type of electromagnetic wave, which is used as a visually recognizable warning.

[0885] "Sound" is a physical phenomenon that is transmitted through vibrations in the air and is used as an alarm that can be recognized through the ears.

[0886] "Reporting" refers to the act of informing an external agency or person in charge of an abnormality or emergency.

[0887] A "log" is data that records all the processes performed and events that occur while the system is running.

[0888] "Smart glasses" are glasses-type devices that have built-in displays, cameras, and communication functions, and have the ability to display information and acquire images.

[0889] A "display" is a display device for displaying images and text information.

[0890] "Emotional state" refers to the type and intensity of emotions a person is experiencing, such as tension, anxiety, anger, or other psychological conditions.

[0891] "Analysis means" refers to a method or device for analyzing specific data, and in this system includes emotion analysis using AI.

[0892] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video, thereby enhancing the security of the monitored object. In addition, by combining it with an emotion engine using AI technology, it is possible to identify the emotional state of suspicious individuals, realizing more accurate crime prevention measures. This system is composed of surveillance cameras, a server, a terminal, and an emotion engine, which operate in cooperation with each other.

[0893] System configuration

[0894] Surveillance cameras: These are devices installed inside and outside buildings and facilities that capture video in real time. Surveillance cameras are connected to servers and terminals via a network.

[0895] Server: This is the central device that analyzes the captured video using an AI model and recognizes people in the video. It compares the characteristics of the recognized people with a database to identify unregistered people. It also analyzes the person's emotional state using an emotion engine.

[0896] Terminal: A device that receives images sent from surveillance cameras and periodically sends them to a server, and also receives alarm signals from the server and activates light and sound alarms within the building.

[0897] Emotion Engine: This engine analyzes the facial expressions and movements of people in the video to determine their emotional state in real time. Based on the detected emotional state, it issues additional alerts or reports as needed.

[0898] Specific program description

[0899] The server performs video analysis using software such as Python, OpenCV, dlib, and Keras. Images captured by surveillance cameras are first captured in real time and then sent to the server at regular intervals. The server analyzes the received images, recognizes people's faces using an AI model, and determines their emotional state using an emotion engine. The analysis results are displayed on the smart glasses or a display.

[0900] Hardware used:

[0901] 1. Surveillance cameras

[0902] 2. Server

[0903] 3. Terminal

[0904] 4. Smart Glasses

[0905] Software used:

[0906] 1. Python

[0907] 2. OpenCV

[0908] 3. dlib

[0909] 4. Keras

[0910] Specific examples

[0911] Example 1: If the candidate is eligible, they will pass.

[0912] When a user enters a building, the device captures video from the security cameras installed in the building and sends it to the server. The server receives the video and analyzes the person using an AI model, and an emotion engine detects their normal state. The analyzed person's characteristics are compared with a database to confirm that they are a registered person. No alarm signal is issued, so the system takes no action. The user can enter the building without any problems.

[0913] Example 2: When a suspicious person with an abnormal emotional state passes by

[0914] Late at night, the device captures footage sent from the surveillance camera and sends it to the server. The server receives the footage and analyzes the person using an AI model, and the emotion engine detects that the person is in a very tense state. The analyzed person's characteristics are compared with a database to identify unregistered people. Having identified an unregistered person in a tense state, an alarm signal is sent to the device. The server then transmits the information to an external reporting point. The device receives the alarm signal and activates a stronger-than-normal light and sound alarm within the building. The server then reports information about the suspicious person and their state of tension to the security company. This entire process is recorded in a log, allowing an administrator to review the situation later.

[0915] Prompt Sentence Examples

[0916] 1. "Load an image dataset and crop it to center on a person's face."

[0917] 2. "Train a CNN model with emotion labels (e.g., anger, neutral, happy)."

[0918] 3. "Save the emotion recognition model and implement it so that it can analyze video input in real time."

[0919] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0920] System program processing flow

[0921] Step 1:

[0922] Capture and Send

[0923] Description: The device captures video from a surveillance camera in real time and transmits it to a server.

[0924] Input: Real-time video captured from a surveillance camera.

[0925] Output: Sending video data to the server.

[0926] How it works: A surveillance camera monitors a specific area and captures the video in real time, then periodically sends each frame of the captured video to a server.

[0927] Step 2:

[0928] Video reception and pre-processing

[0929] Description: The server receives the video sent from the terminal and performs preprocessing.

[0930] Input: Video data sent from the device.

[0931] Output: Preprocessed video data.

[0932] How it works: The server temporarily stores the received video data and performs preprocessing such as noise removal and resolution adjustment. This preprocessing ensures that subsequent AI analysis can be performed with high accuracy.

[0933] Step 3:

[0934] person recognition

[0935] Description: The server analyzes the pre-processed footage and uses AI means to recognize people within the footage.

[0936] Input: Preprocessed video data.

[0937] Output: Information about the recognized person.

[0938] Operation: For video analysis, a face detection algorithm (e.g., Haar-cascade or dlib) is used to recognize human faces in the video. Feature data of the recognized faces is extracted.

[0939] Step 4:

[0940] Feature Matching

[0941] Description: The server checks the characteristics of the recognized person against a pre-registered database.

[0942] Input: Recognized person feature data,database.

[0943] Output: Matching result (enrolled or not enrolled).

[0944] Operation: The feature data of a person obtained through analysis is compared with the feature data of a person previously registered in a database. This comparison identifies whether the person is registered or not.

[0945] Step 5:

[0946] Emotion analysis

[0947] Description: In parallel with the facial recognition results, the server uses an emotion engine to analyze the emotional state of people in the video.

[0948] Input: Recognized person's facial features, emotion engine.

[0949] Output: Emotional state (tension, anxiety, anger, etc.).

[0950] Movement: The emotion engine analyzes facial expressions and movements from facial feature data, labels the analyzed emotional states, and identifies specific emotional states such as tension, anxiety, and anger.

[0951] Step 6:

[0952] Determining and issuing an alarm

[0953] Description: The server generates an alarm signal according to the identified unregistered person and their emotional state and sends it to the terminal.

[0954] Input: Matching results, sentiment analysis results.

[0955] Output: Alarm signal.

[0956] Operation: If an unregistered person is detected and an abnormal emotional state (e.g., tension or anger) is detected, the server generates an alarm signal and sends it to the device. The device receives this signal and activates a light and sound alarm.

[0957] Step 7:

[0958] Reporting and logging

[0959] Description: The server automatically reports information about unregistered people and their emotional state to an external reporting destination, and saves the results of the process as a log.

[0960] Input: Information about unregistered persons, contact information.

[0961] Output: Report data, log data.

[0962] How it works: The server reports information about suspicious individuals and their emotional state in real time to security companies, administrators, and other reporting points. It also saves the results of this process in detail as a log so that administrators can check the situation later.

[0963] In this way, through each processing step, advanced crime prevention measures based on video data are realized.

[0964] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0965] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0966] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0967] [Third embodiment]

[0968] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0969] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0970] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0971] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0972] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0973] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0974] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0975] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0976] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0977] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0978] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0979] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0980] This invention is a system that enhances the security of monitored objects by capturing video in real time using a surveillance camera and recognizing and identifying people in the video. This system is composed of a surveillance camera, a server, and a terminal, which operate in cooperation with each other.

[0981] System configuration

[0982] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time. They are connected to devices via a network.

[0983] Terminal: Receives images sent from the surveillance cameras and periodically sends them to the server. It also receives alarm signals from the server and activates light and sound alarms within the building.

[0984] Server: Analyzes the captured video using an AI model and recognizes people in the video. The characteristics of the recognized people are compared with a database to identify unregistered people. If an unregistered person is identified, an alarm signal is sent to the terminal and an automatic report is made to an external reporting point.

[0985] Specific Modes for Carrying Out the Invention

[0986] Video capture and transmission

[0987] The device captures video from the surveillance camera in real time. The captured video is sent to the server frame by frame at regular intervals. This process ensures that the latest surveillance video is always provided to the server in a format that can be analyzed.

[0988] Video analysis and matching

[0989] The server analyzes the received video frames using an AI model. This AI model is based on facial recognition technology and recognizes the faces of people in the video and extracts their features. The extracted features are compared with a pre-registered database, which contains facial data of people involved in the building or facility.

[0990] Identifying suspicious people

[0991] If the result of the check is that there is no matching person in the database, the server identifies the person as an unregistered person (suspicious person), and it is determined that the identified unregistered person requires further action.

[0992] Activation of an alarm

[0993] The server sends an alarm signal to the terminal for any identified unregistered person. When the terminal receives the alarm signal, it immediately activates a light and sound alarm within the building to alert those in the vicinity. This alarm increases the likelihood that the suspicious person will leave the area, and allows those in the vicinity to become more vigilant.

[0994] Automatic reporting

[0995] At the same time, the server automatically sends information about the suspicious person to an external reporting point, including the video frame and the suspicious person's characteristics, so that the security company or relevant security department can respond immediately.

[0996] Log storage

[0997] The server records the results of the series of processes as a log. This log includes the results of video analysis, the identification of suspicious individuals, the issuance of alarms, and the details of reports. The log can be accessed and checked later by an administrator, helping to understand the situation and prevent recurrence.

[0998] Specific examples

[0999] Example 1: If the candidate is eligible, they will pass.

[1000] 1. User: The device captures video from a surveillance camera installed in the building and sends it to the server.

[1001] 2. Server: Receives the video and analyzes the person using an AI model.

[1002] 3. Server: Compares the analyzed person's characteristics with a database and confirms that the person is a registered person (e.g., an employee).

[1003] 4. Server: No alarm signal is generated, so no action is taken by the system. The user can enter the building without any problems.

[1004] Example 2: When a suspicious person passes through

[1005] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[1006] 2. Server: Receives the video and analyzes the person using an AI model.

[1007] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people.

[1008] 4. Server: Sends an alarm signal to the device.

[1009] 5. Terminal: Receives alarm signals and activates light and sound alarms within the building.

[1010] 6. Server: Automatically reports information about suspicious individuals to security companies.

[1011] 7. Server: This process is logged so that an administrator can review it later.

[1012] In this way, the present invention makes it possible to automatically and effectively strengthen the security of buildings and facilities.

[1013] The processing flow will be explained below.

[1014] Step 1:

[1015] The device captures video from the surveillance camera in real time, and the captured video is buffered in the device and awaits the next transmission timing.

[1016] Step 2:

[1017] The device transmits the captured video frames to the server at regular intervals. This transmission is carried out over a network, and the data is efficiently compressed to ensure real-time performance.

[1018] Step 3:

[1019] The server receives the video frames sent from the device, which are then immediately passed to the AI ​​analysis module.

[1020] Step 4:

[1021] The server's AI analysis module identifies people in the video frame and extracts their facial features, using a facial recognition algorithm.

[1022] Step 5:

[1023] The server compares the extracted person's features with a pre-registered facial database of building personnel. During the matching process, it calculates the degree of match between the features and identifies those with a high degree of match.

[1024] Step 6:

[1025] The server uses the results of the comparison to determine whether an unregistered person exists. If there is no match in the database, the person is identified as suspicious.

[1026] Step 7:

[1027] When the server identifies a suspicious person, it generates an alarm signal and sends it to the terminal. The alarm signal includes detailed information about the identified person (e.g., video frames and features).

[1028] Step 8:

[1029] The terminal receives the alarm signal and immediately activates a light and sound alarm within the building, with preset light and sound intensities and patterns.

[1030] Step 9:

[1031] At the same time, the server automatically reports the suspicious person's information to an external reporting point (e.g., a security company or administrator). The report includes important information (such as the suspicious person's characteristics, video frames, and the time of the incident).

[1032] Step 10:

[1033] The server records a detailed log of the process results, including the details of each step and a timestamp, and saves it so that an administrator can review the situation later.

[1034] Example 1

[1035] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1036] Conventional security systems often rely on manual monitoring, which can lead to incomplete monitoring and human error. Furthermore, existing video surveillance systems have difficulty recognizing and identifying people in real time, which can result in delays in detecting suspicious individuals. A system that can resolve these issues and perform monitoring more accurately and quickly is needed.

[1037] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1038] In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, means for analyzing the received video using a generative AI model and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for identifying unregistered people based on the results of the comparison, means for outputting an alarm signal for the identified unregistered people, means for issuing an alarm with light and sound based on the alarm signal, means for automatically reporting information about the unregistered person to an external reporting destination, and means for saving the series of processes as a log. This enables real-time recognition and identification of people, enabling quick and accurate detection of suspicious people and appropriate response.

[1039] A "surveillance camera" is a device that captures video footage in real time and transmits the video data over a network.

[1040] A "capturing means" is a method or device by which a surveillance camera generates a video stream and captures that video footage.

[1041] "Transmitting means" refers to a method or device for transmitting captured video data to a server via a network at regular intervals.

[1042] A "generative AI model" is an algorithm or program that uses machine learning techniques to recognize and identify people.

[1043] The "analyzing means" refers to a method or device for analyzing received video data and recognizing people in the video.

[1044] A "means for extracting features" is a method or device for extracting identifiable features such as the face or posture of a recognized person.

[1045] A "database" is a storage device or system that stores facial data and feature data of registered people and uses them for matching purposes.

[1046] The "matching means" refers to a method or device for comparing extracted feature data with registered data in a database to find a matching person.

[1047] The "means for identifying" refers to a method or device for detecting an unregistered person identified as a result of the matching and recognizing him or her as a suspicious person.

[1048] The "means for outputting an alert signal" refers to a method or device for generating and transmitting a signal to alert an identified unregistered person.

[1049] The "means for activating an alarm" refers to a method or device for actually activating an alarm using light and sound within the building based on the alarm signal received by the terminal.

[1050] "Means for reporting" refers to a method or device for automatically sending information about a suspicious person to an external reporting destination.

[1051] "Means for saving as a log" refers to a method or device for recording the results of a series of processes and saving them so that they can be checked later.

[1052] "Real-time" refers to the state in which video is processed immediately after it is captured, with almost no delay.

[1053] MODE FOR CARRYING OUT THE INVENTION

[1054] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video, thereby enhancing the security of the monitored object. This system is composed of surveillance cameras, a server, and a terminal, and each device operates in cooperation with each other.

[1055] System configuration

[1056] Surveillance cameras: These cameras are installed inside and outside buildings and facilities and capture video footage in real time. They are connected to devices via a network.

[1057] Terminal: Receives images sent from the surveillance camera and periodically sends them to the server. It also receives alarm signals from the server and activates alarms using light and sound.

[1058] Server: Analyzes the captured video using a generative AI model to recognize people in the video. It compares the characteristics of the recognized people with a database to identify unregistered people. If an unregistered person is identified, it sends an alarm signal to the device and automatically reports the incident to an external reporting point.

[1059] Hardware and Software

[1060] Surveillance cameras: Network cameras (e.g. IP cameras) are used.

[1061] Terminal: A computing device (e.g., PC, Raspberry Pi) for processing and transmitting video.

[1062] Server: Use a high-performance server (e.g., AWS EC2, on-premise server) and database (e.g., MySQL).

[1063] Generative AI models: Use libraries for AI analysis (e.g., TensorFlow, OpenCV).

[1064] Video capture and transmission

[1065] The device captures video from the surveillance camera in real time. The captured video is sent to the server frame by frame at regular intervals (for example, 30 frames per second). This process ensures that the latest surveillance video is always provided to the server in a format that can be analyzed.

[1066] Video analysis and matching

[1067] The server analyzes the received video frames using a generative AI model. This AI model is based on facial recognition technology and recognizes the faces of people in the video and extracts their features. The extracted features are compared with a pre-registered database, which contains facial data of people involved in the building or facility.

[1068] Identifying suspicious people

[1069] If the result of the check is that there is no matching person in the database, the server identifies the person as an unregistered person (suspicious person), and it is determined that the identified unregistered person requires further action.

[1070] Activation of an alarm

[1071] The server sends an alarm signal to the terminal for any identified unregistered person. When the terminal receives the alarm signal, it immediately activates a light and sound alarm within the building to alert those in the vicinity. This alarm increases the likelihood that the suspicious person will leave the area, and allows those in the vicinity to become more vigilant.

[1072] Automatic reporting

[1073] At the same time, the server automatically sends information about the suspicious person to an external reporting organization (e.g., a security company or the police), which can then immediately respond by sending a report containing the video frame and the suspicious person's characteristics.

[1074] Log storage

[1075] The server records the results of the series of processes as a log. This log includes the results of video analysis, the identification of suspicious individuals, the issuance of alarms, and the details of reports. The log can be accessed and checked later by an administrator, helping to understand the situation and prevent recurrence.

[1076] Specific examples

[1077] Example 1: If the candidate is eligible, they will pass.

[1078] 1. Terminal: Captures video from surveillance cameras installed in the building and sends it to the server.

[1079] 2. Server: Receives the video and analyzes the person using a generative AI model.

[1080] 3. Server: The analyzed person's characteristics are compared with the database to confirm that the person is a registered person (for example, an employee).

[1081] 4. Server: No alarm signal is generated, so no action is taken by the system. As a result, the user can enter the building without any problems.

[1082] Example 2: When a suspicious person passes through

[1083] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[1084] 2. Server: Receives the video and analyzes the person using a generative AI model.

[1085] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people (suspicious individuals).

[1086] 4. Server: Sends an alarm signal to the device.

[1087] 5. Terminal: Receives alarm signals and activates light and sound alarms within the building.

[1088] 6. Server: Automatically reports information about suspicious individuals to security companies. Specifically, the report is made by sending an HTTP request to the security company's API.

[1089] 7. Server: This process is logged so that an administrator can review it later.

[1090] Prompt Sentence Examples

[1091] "Please explain how a system can capture video in real time from security cameras installed in an office building, analyze the video using a generative AI model (e.g., OpenCV), identify suspicious individuals, and then automatically report this information to a security company."

[1092] By using this prompt sentence, the generative AI model can be made to explain the specific processing details of the system.

[1093] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1094] Step 1:

[1095] The terminal captures video from a surveillance camera in real time. The input is the video stream from the surveillance camera, and the output is the captured video frames. Specifically, the terminal acquires the stream based on the IP address of the surveillance camera, captures the frames, and stores them in memory.

[1096] Step 2:

[1097] The device sends the captured video to the server at regular intervals (e.g., 30 frames per second). The input is the captured video frame, and the output is the transmission to the server. Specifically, the device uses HTTP or WebSocket to packetize the frame data and send it to the server.

[1098] Step 3:

[1099] The server receives video frames sent from the terminal. The input is the video frame from the terminal, and the output is the storage of the received frame. Specifically, the server receives packets through the network interface, reconstructs them into frames, and stores them in memory.

[1100] Step 4:

[1101] The server analyzes the received video frames using a generative AI model (e.g., TensorFlow or OpenCV). The input is the stored frame, and the output is the analyzed person's facial feature data. Specifically, the server applies a face detection algorithm to the frame, extracts facial feature points, and stores them as vectors.

[1102] Step 5:

[1103] The server compares the extracted feature data with the database. The input is the extracted vector data, and the output is the comparison result (registered or unregistered). Specifically, the server uses an SQL query to compare the person feature vectors in the database and calculate the similarity.

[1104] Step 6:

[1105] The server identifies an unregistered person as a result of the matching. The input is the matching result, and the output is the identification result of the unregistered person. Specifically, if the similarity threshold is not exceeded, the server identifies the person as unregistered and stores that information in a structure.

[1106] Step 7:

[1107] The server sends an alarm signal to the terminal for the identified unregistered person. The input is the identification result of the unregistered person, and the output is the alarm signal to the terminal. Specifically, the server sends an alarm signal data packet to the terminal using the TCP / IP protocol.

[1108] Step 8:

[1109] The terminal receives the alarm signal and activates the light and sound alarms in the building. The input is the alarm signal from the server, and the output is the actual activation of the alarm sound and light. Specifically, the terminal analyzes the received signal and sends a signal to the circuitry that controls the speaker and light.

[1110] Step 9:

[1111] The server automatically reports information about unregistered individuals to external reporting destinations. The input is the identification result of the unregistered individual and related data, and the output is a report to the external reporting destination. Specifically, the server uses an API request or email sending function to send the information to the specified reporting destination.

[1112] Step 10:

[1113] The server saves the results of a series of processes as a log. The input is the result data of each process, and the output is the saved log file. Specifically, the server records the results of the process in text format or database format so that they can be checked later.

[1114] (Application example 1)

[1115] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1116] While conventional security systems have the ability to detect suspicious individuals inside and outside a building in real time and issue an alarm, they face the challenge of making it difficult for administrators to monitor and control the system from a remote location. Furthermore, when an unregistered individual is identified, the response can be delayed, which is insufficient for modern security needs that require a rapid response. There is a demand for a security system that can solve these issues and provide a more effective and prompt response.

[1117] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1118] In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, AI means for analyzing the received video and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for identifying unregistered people based on the comparison result, means for outputting an alarm signal for the identified unregistered people, means for issuing an alarm with light and sound based on the alarm signal, means for automatically reporting information about the unregistered people to an external reporting destination, means for saving the series of processes as a log, means for viewing the real-time video on a smartphone, means for sending a push notification to the smartphone when an unregistered person is detected, and means for remotely issuing an alarm from the smartphone. This allows an administrator to monitor in real time even from a remote location and immediately issue an alarm.

[1119] A "surveillance camera" is a device that captures real-time footage of the inside and outside of a building or facility.

[1120] "Real-time video" refers to video footage captured by a surveillance camera.

[1121] "Capturing" means taking and recording video.

[1122] "Sending at regular intervals" means sending the captured video to the server periodically.

[1123] "Analyze" means to analyze the video and understand its content.

[1124] "AI means" refers to a method or device for recognizing people in video using artificial intelligence technology.

[1125] The "pre-registered database" refers to a database that holds pre-registered face image data.

[1126] "Matching" means comparing the characteristics of a recognized person with data in a database.

[1127] "Identify" refers to identifying a person based on specific criteria.

[1128] An "alarm signal" is a signal that is transmitted when an abnormality is detected.

[1129] "Activating an alarm with light and sound" refers to using light and sound to notify of an abnormality.

[1130] "Automatically report to the reporting destination" means automatically reporting an abnormality to a designated external agency.

[1131] "Saving as a log" refers to recording and saving the results of processing and the history of events.

[1132] A "smartphone" is a mobile device that can connect to the Internet and run applications.

[1133] "Push notification" refers to the function of sending notifications from a server to a smartphone in real time.

[1134] "Remotely activating an alarm" means remotely operating and activating an alarm from a distance.

[1135] System configuration

[1136] The system for implementing the present invention comprises the following main hardware and software components:

[1137] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time, which is then transmitted to a server over a network.

[1138] Server: Receives the captured video and analyzes it using an AI model. The server compares the feature data with a database to identify unregistered individuals.

[1139] Smartphone: Runs an application that allows administrators to view video footage in real time and remotely control alarms.

[1140] Cloud services: Utilize cloud services such as AWS and Google Cloud for video analysis and data management.

[1141] Program processing explanation

[1142] Video capture and transmission

[1143] Surveillance cameras capture images of the inside and outside of buildings in real time. The captured images are periodically sent to a server, frame by frame. Because the image data is sent over a network, the communication environment between the camera and the server is important.

[1144] Video analysis and matching

[1145] The server analyzes the received video using an AI model. Specifically, it detects people in the video frame and extracts their facial features. This analysis uses a facial recognition algorithm using OpenCV and Dlib. The extracted facial features are compared with a pre-registered database, and if there is no match, the person is identified as unregistered.

[1146] Alerts and notifications

[1147] When an unregistered person is identified, the server generates an alarm signal, triggering light and sound alarms within the building, and sends a push notification to the administrator's smartphone, immediately informing them of the abnormality. The push notification is sent using Firebase cloud messaging.

[1148] Log saving and reporting

[1149] The server saves the results of the series of processes as a log. The saved log includes the results of video analysis, the identification of suspicious individuals, and the history of alarm activation. This allows administrators to check the details later. In addition, information on unregistered individuals is automatically sent to external reporting points.

[1150] Specific examples

[1151] Example 1: If the candidate is eligible, they will pass.

[1152] 1. The user (surveillance camera) captures video from inside the facility and sends it to the server.

[1153] 2. The server uses an AI model to analyze the person and match them with a database.

[1154] 3. If the match is with a registered person, no alarm will be triggered and the system will operate normally.

[1155] Example 2: When a suspicious person passes through

[1156] 1. The user (surveillance camera) captures video from inside the facility and sends it to the server.

[1157] 2. The server uses an AI model to analyze the person and match them with a database.

[1158] 3. If an unregistered person is identified, the server generates an alarm signal and triggers an alarm.

[1159] 4. At the same time, the server sends a push notification to the administrator's smartphone and also sends the information to external reporting points such as security companies.

[1160] Prompt Sentence Examples

[1161] For example, by inputting the following prompt sentence into the generative AI model, you can get advice and adjustments to improve the accuracy of surveillance camera video analysis.

[1162] "We are developing an AI system that analyzes surveillance camera footage and performs person recognition. Could you please tell us some techniques to improve accuracy in identifying unregistered people by accurately capturing the characteristics of people in the footage?"

[1163] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1164] Step 1:

[1165] A user (surveillance camera) captures video of the inside and outside of a building in real time. The captured video is converted into frames within the camera. At that time, the video frames are given a time stamp.

[1166] Input: Real-time video

[1167] Output: Captured video frames

[1168] Step 2:

[1169] The device periodically transmits captured video frames to the server, where they are encoded and split into packets over the network.

[1170] Input: Captured video frames

[1171] Output: Video packets sent to the server

[1172] Step 3:

[1173] The server decodes the received video packets and reconstructs them into the original video frames, which are then stored in a buffer.

[1174] Input: Video packets sent to the server

[1175] Output: Decoded and buffered video frames

[1176] Step 4:

[1177] The server uses an AI model to analyze the video frames in the buffer and detect faces in the video, extracting facial feature points using OpenCV and the Dlib library.

[1178] Input: Buffered video frames

[1179] Output: Detected facial feature points

[1180] Step 5:

[1181] The server compares the extracted facial feature points with pre-registered face data in a database, and calculates the similarity between the feature vectors.

[1182] Input: Detected facial feature points, registered face data in the database

[1183] Output: Matching result (registered or unregistered)

[1184] Step 6:

[1185] If the matched person is an unregistered person, the server generates an alarm signal, which includes facial feature information and a video frame of the unregistered person.

[1186] Input: Matching result is unregistered person

[1187] Output: Alarm signal

[1188] Step 7:

[1189] The terminal receives the alarm signal and activates a light and sound alarm within the building. The alarm is visually and audibly activated using a speaker and a light.

[1190] Input: Alarm signal

[1191] Output: Light and sound alarm activation

[1192] Step 8:

[1193] The server simultaneously sends an alarm signal to the administrator's smartphone as a push notification, using Firebase cloud messaging to deliver the notification in real time.

[1194] Input: Alarm signal

[1195] Output: Push notification to smartphone

[1196] Step 9:

[1197] The server automatically reports information about unregistered individuals to an external reporting point, including video frames and details of the alert.

[1198] Input: Unregistered person information, alarm signal

[1199] Output: Automatic notification to external reporting destination

[1200] Step 10:

[1201] The server saves the results of the series of processes as a log, which includes video analysis results, alarm activation history, suspicious person information, and report history, and can be checked later by the administrator.

[1202] Input: A set of processing results

[1203] Output: Saved logs

[1204] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1205] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video, thereby enhancing the security of the monitored object. In addition, by combining it with an emotion engine using AI technology, it is possible to identify the emotional state of suspicious individuals, realizing more accurate crime prevention measures. This system is composed of surveillance cameras, a server, a terminal, and an emotion engine, which operate in cooperation with each other.

[1206] System configuration

[1207] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time. They are connected to devices via a network.

[1208] Terminal: Receives images sent from the surveillance cameras and periodically sends them to the server. It also receives alarm signals from the server and activates light and sound alarms within the building.

[1209] Server: Analyzes the captured video using an AI model to recognize people in the video. It compares the features of the recognized people with a database to identify unregistered people. It also analyzes the emotional state of the people using an emotion engine.

[1210] Emotion Engine: Analyzes facial expressions and movements of people in the video to determine their emotional state in real time, and issues additional alerts or notifications as needed based on the detected emotional state.

[1211] Specific Modes for Carrying Out the Invention

[1212] Video capture and transmission

[1213] The device captures video from the surveillance camera in real time. The captured video is sent to the server frame by frame at regular intervals. This process ensures that the latest surveillance video is always provided to the server in a format that can be analyzed.

[1214] Video analysis and matching

[1215] The server analyzes the received video frames using an AI model. This AI model is based on facial recognition technology and recognizes the faces of people in the video and extracts their features. The extracted features are compared with a pre-registered database, which contains facial data of people involved in the building or facility.

[1216] Emotion analysis using an emotion engine

[1217] In parallel with the facial recognition results, the server uses an emotion engine to analyze the emotional state of people in the video, which determines specific emotional states such as tension, anxiety, or anger based on the person's facial expressions and movements.

[1218] Identifying suspicious people and emotional states

[1219] By combining the matching results with the results of emotion analysis, the server identifies unregistered individuals and evaluates their emotional state. If abnormal emotional states, such as tension, anxiety, or anger, are detected, the server determines that more immediate action is required.

[1220] Activation of an alarm

[1221] The server generates an alarm signal according to the identified unregistered person and their emotional state and transmits it to the terminal. For example, if an unregistered person in a very tense state is identified, the terminal will issue a stronger alarm than usual.

[1222] Automatic reporting

[1223] At the same time, the server automatically reports the suspicious person's information and emotional state to an external reporting point (e.g., a security company or administrator). The report includes video frames, characteristic information, and the person's emotional state, allowing the security company to respond immediately.

[1224] Log storage

[1225] The server records a detailed log of the entire process, including the results of video analysis, the identification of suspicious individuals, their emotional state, whether an alarm was issued, and the details of the report, and the log is saved so that administrators can review the situation later.

[1226] Specific examples

[1227] Example 1: If the candidate is eligible, they will pass.

[1228] 1. User: The device captures video from a surveillance camera installed in the building and sends it to the server.

[1229] 2. Server: Receives the video and analyzes the person using the AI ​​model. The emotion engine detects the normal state.

[1230] 3. Server: Compares the analyzed person's characteristics with the database and verifies that the person is registered.

[1231] 4. Server: No alarm signal is generated, so no action is taken by the system. The user can enter the building without any problems.

[1232] Example 2: When a suspicious person with an abnormal emotional state passes by

[1233] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[1234] 2. Server: Receives the video and analyzes the person using an AI model. The emotion engine detects when the person is extremely nervous.

[1235] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people.

[1236] 4. Server: Having identified an unregistered person in a tense state, the server sends an alarm signal to the terminal. The server then sends the information to an external reporting point.

[1237] 5. Terminal: Receives the alarm signal and activates stronger than normal light and sound alarms within the building.

[1238] 6. Server: Reports information about suspicious individuals and the state of tension to the security company.

[1239] 7. Server: This process is logged so that an administrator can review it later.

[1240] In this way, the present invention automatically and effectively strengthens the security of buildings and facilities, and by taking into account emotional states in particular, more advanced crime prevention measures become possible.

[1241] The processing flow will be explained below.

[1242] Step 1:

[1243] The device captures video from the surveillance camera in real time, and the captured video is temporarily stored in the device, waiting for the next transmission timing.

[1244] Step 2:

[1245] The device transmits captured video frames to the server at regular intervals, and the transmitted data is compressed to reduce network load.

[1246] Step 3:

[1247] The server processes video frames received from the device in real time. The video frames are first passed to the AI ​​analysis module.

[1248] Step 4:

[1249] The server's AI analysis module identifies people in the video frame and extracts their facial features, using facial recognition algorithms.

[1250] Step 5:

[1251] The server then matches the extracted person's characteristics with existing data in a database that contains building personnel information to identify matches.

[1252] Step 6:

[1253] The server identifies unregistered individuals based on the matching results. If there is no matching person in the database, the individual is recognized as an unregistered (suspicious) individual.

[1254] Step 7:

[1255] In parallel, the server runs an emotion engine that analyzes the facial expressions and movements of people in the video, using AI to identify emotional states such as tension, anxiety, or anger.

[1256] Step 8:

[1257] The server integrates the analysis results of the emotion engine with the registered information of the person and checks for the presence of an unregistered person with an abnormal emotional state.

[1258] Step 9:

[1259] The server generates an alarm signal for an unregistered person with an abnormal emotional state and sends it to the terminal, which includes the analyzed feature information and the emotional state.

[1260] Step 10:

[1261] The terminal receives the alarm signal and immediately activates a light and sound alarm within the building, which is stronger than normal and alerts those around.

[1262] Step 11:

[1263] At the same time, the server automatically sends the information about the unregistered person and the emotion analysis results to an external reporting point (e.g., a security company or facility manager). The report includes detailed characteristic information and the person's emotional state.

[1264] Step 12:

[1265] The server saves all processing results as logs, which contain detailed records of timestamps, processing details, alarm activation history, and report details. Administrators can later check and analyze these logs.

[1266] In this way, the present invention effectively links surveillance cameras, terminals, servers, and emotion engines to automatically achieve real-time surveillance and anomaly detection with high accuracy.

[1267] Example 2

[1268] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1269] Conventional security systems are equipped with video surveillance using surveillance cameras and identification functions using facial recognition, but lack the ability to detect abnormalities based on a person's emotional state. As a result, it is difficult to detect abnormal emotional states, such as when a suspicious person is nervous or anxious, which reduces the accuracy of security measures. In addition, there is also the issue of not only simply detecting unregistered individuals, but also needing to respond appropriately according to the person's emotional state.

[1270] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, AI means for analyzing the received video and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for analyzing the emotional state of people in the video in real time using emotion analysis means and detecting abnormal emotional states, and means for generating an additional alarm signal in response to the detected abnormal emotional state. This makes it possible not only to detect unregistered people but also to detect abnormalities based on emotional states and respond quickly.

[1271] A "surveillance camera" is a device installed inside or outside a building or facility to capture video in real time.

[1272] The "terminal" is a device or software that receives images sent from the surveillance camera and periodically sends them to the server. The terminal also receives alarm signals from the server and activates light and sound alarms within the building.

[1273] The "server" is a computer system that analyzes the captured video and recognizes people in the video. The server also performs emotion analysis to assess the emotional state of people and issue alerts if necessary.

[1274] The "Emotion Engine" is a system that uses AI technology to analyze the facial expressions and movements of people in video footage and determine their emotional state in real time.

[1275] An "AI model" is an algorithm or software that uses artificial intelligence technology to recognize the faces of people in video and extract their features.

[1276] A "database" is a data storage that stores characteristic information (face data) of people who have been registered in advance.

[1277] An "alarm signal" is a signal sent to a terminal to trigger an alarm in response to an identified unregistered person and their abnormal emotional state.

[1278] "Light and sound alarms" are visual and audible warnings that are activated within a building based on an alarm signal, such as flashing lights and audible alarms.

[1279] "Automatic reporting" is a process in which the server notifies an external contact point (such as a security company or administrator) of information about a suspicious person and their emotional state. Notifications are sent via email, text message, or API communication.

[1280] A "log" is a detailed record of a series of processing results, and is data that is saved for later review by an administrator. Logs include analysis results, identification of suspicious individuals, emotional states, triggered alarms, and report contents.

[1281] MODE FOR CARRYING OUT THE INVENTION

[1282] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video. Furthermore, by combining it with an emotion engine using AI technology, it can identify the emotional state of suspicious individuals and achieve highly accurate crime prevention measures. This system is composed of surveillance cameras, a server, a terminal, and an emotion engine, which work in cooperation with each other.

[1283] System configuration

[1284] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time. They are connected to devices via a network.

[1285] Terminal: Receives images sent from the surveillance cameras and periodically sends them to the server. It also receives alarm signals from the server and activates light and sound alarms within the building.

[1286] Server: Analyzes the captured video using an AI model to recognize people in the video. It compares the features of the recognized people with a database to identify unregistered people. It also analyzes the emotional state of the people using an emotion engine.

[1287] Emotion Engine: Analyzes facial expressions and movements of people in the video to determine their emotional state in real time, and issues additional alerts or notifications as needed based on the detected emotional state.

[1288] Specific Modes for Carrying Out the Invention

[1289] The server has a means to capture video from the surveillance cameras in real time. Specifically, the surveillance cameras continuously capture video inside and outside the building or facility and send the video data to the terminal. The terminal converts this data into capture frames at regular intervals and sends them to the server via the network.

[1290] The server analyzes the received video frames using an AI model. This AI model uses libraries such as OpenCV and Dlib to recognize people's faces and extract their features. The extracted features are compared with a pre-registered database containing facial data of people associated with the building or facility. If the comparison identifies an unregistered person, the server generates an alarm signal.

[1291] Additionally, the server uses an emotion engine to analyze the emotional state of people in the video in real time. This emotion engine, powered by services such as Microsoft Azure Cognitive Services and Google Cloud Vision, analyzes facial expressions and movements to identify emotions such as tension, anxiety, and anger. If an abnormal emotional state is detected, the server generates an additional alarm signal, enabling a rapid response.

[1292] The generated alarm signal is sent to a terminal, which activates the building's alarm system (e.g., audio alarm or flashing light), and the server automatically reports information about the suspicious person and their emotional state to an external reporting point via email, text message, or API communication.

[1293] Finally, the server records a detailed log of the entire process, including the results of video analysis, the identification of suspicious individuals, their emotional state, whether an alarm was activated, and the details of the report, which is then saved for later review by an administrator.

[1294] Specific examples

[1295] Example 1: If the candidate is eligible, they will pass.

[1296] 1. User: The device captures video from a surveillance camera installed in the building and sends it to the server.

[1297] 2. Server: Receives the video and analyzes the person using an AI model. The emotion engine detects the normal state.

[1298] 3. Server: Compares the analyzed person's characteristics with the database and verifies that the person is registered.

[1299] 4. Server: No alarm signal is generated, so no action is taken by the system. The user can enter the building without any problems.

[1300] Example 2: When a suspicious person with an abnormal emotional state passes by

[1301] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[1302] 2. Server: Receives the video and analyzes the person using an AI model. The emotion engine detects when the person is extremely nervous.

[1303] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people.

[1304] 4. Server: Having identified an unregistered person in a tense state, the server sends an alarm signal to the terminal. The server then sends the information to an external reporting point.

[1305] 5. Terminal: Receives the alarm signal and activates stronger than normal light and sound alarms within the building.

[1306] 6. Server: Reports information about suspicious individuals and the state of tension to the security company.

[1307] 7. Server: This process is logged so that an administrator can review it later.

[1308] Prompt Sentence Examples

[1309] Example prompt sentence:

[1310] "Analyze the emotional state of a person in a video captured by a surveillance camera late at night. Explain how the system would respond if it detected an abnormal level of tension."

[1311] In this way, the present invention automatically and effectively strengthens the security of buildings and facilities, and by taking into account emotional states in particular, more advanced crime prevention measures become possible.

[1312] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1313] Step 1:

[1314] The device captures video from a surveillance camera in real time and sends it to the server frame by frame at regular intervals. The input to this process is the video data from the surveillance camera, and the output is the captured frames sent to the server. This process requires a network connection to transfer the video data to the server via the Internet.

[1315] Step 2:

[1316] The server receives video frames sent from the surveillance camera. The input to this process is the video frames sent from the device, and the output is video data in an analyzable format. The received video data is temporarily stored in the server's storage and then proceeds to the next analysis step.

[1317] Step 3:

[1318] The server uses an AI model to analyze the received video frames and recognize the faces of people in the video. The input to this process is the received video frames, and the output is identified facial feature data. Specifically, an AI model (e.g., OpenCV or Dlib library) is used to run a face detection algorithm and extract the location and feature points of the face.

[1319] Step 4:

[1320] The server compares the feature data extracted by facial recognition with a pre-registered database. The input of this process is the identified facial feature data, and the output is the matching result. For matching, the server checks whether the face data matches the face data in the database, and if it does, it determines that the person is registered.

[1321] Step 5:

[1322] The server uses an emotion analysis engine to analyze the emotional state from the extracted facial feature data. The input for this process is the facial feature data after matching, and the output is the emotional state determination result. The emotion analysis engine uses APIs such as Microsoft Azure Cognitive Services and Google Cloud Vision to identify emotions such as tension, anxiety, and anger from facial expressions.

[1323] Step 6:

[1324] The server identifies unregistered individuals and abnormal emotional states based on the analysis results. The input to this process is the results of facial recognition and emotional analysis, and the output is alarm data containing suspicious individuals and their emotional states. If an abnormal emotional state is detected, the server determines that the individual is suspicious and requires immediate action.

[1325] Step 7:

[1326] The server generates an alarm signal according to the identified unregistered person and their emotional state, and transmits it to the terminal. The input of this process is the alarm data, and the output is the alarm signal. Once the alarm signal is generated, it is transmitted to the terminal.

[1327] Step 8:

[1328] The terminal receives the alarm signal and activates optical and audible alarms in the building. The input of this process is the received alarm signal and the output is the activated alarm. The terminal activates the alarm system and provides visual and audible warnings.

[1329] Step 9:

[1330] The server automatically reports information about suspicious individuals and their emotional state to an external reporting destination. The input to this process is the suspicious individual's information and emotional state, and the output is the sent report message. Reports are sent via email, text message, or API communication.

[1331] Step 10:

[1332] The server records a detailed log of the results of each process, with the input being the results of each process step and the output being a saved log file, allowing administrators to review and respond to the situation at a later time.

[1333] (Application example 2)

[1334] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1335] Conventional surveillance systems have the ability to recognize people in real time using video transmitted from surveillance cameras and issue alarms, but this alone is insufficient to determine the emotional state of a suspicious individual, limiting the accuracy of security responses. Conventional systems cannot respond when a suspicious individual has a calm expression or when preventive measures must be taken before the individual engages in suspicious behavior. To solve this problem, a system is needed that can analyze emotional states in real time and respond immediately. Therefore, the present invention aims to achieve more accurate crime prevention measures by instantly identifying the emotional state of a suspicious individual and issuing an alarm.

[1336] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1337] In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, AI means for analyzing the received video and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for identifying unregistered people based on the comparison result, means for outputting an alarm signal to the identified unregistered people, means for issuing an alarm with light and sound based on the alarm signal, means for automatically reporting information about the unregistered people to an external reporting destination, means for saving the series of processes as a log, means for displaying the faces of people in the video on a display mounted on the smart glasses, and means for analyzing the emotional state of the displayed people and issuing an alarm signal based on the emotional state. This enables the emotional state of people to be identified in real time, enabling prompt and appropriate crime prevention measures.

[1338] - A "surveillance camera" is a device installed to detect suspicious activity and for crime prevention purposes, and is used to capture real-time images of specific locations and acquire the footage.

[1339] "Real-time" refers to processing and responding to an event the moment it occurs, without delay.

[1340] "Video" means visual information captured by a camera or other photographic device, and refers to moving images displayed on a monitor or the like.

[1341] "Capture" refers to the operation or process of acquiring an image using a camera or other photographic device.

[1342] "Transmit" refers to the act of transferring acquired data or information to another device or system.

[1343] "Analysis" refers to the process of analyzing video and data in detail using computers and specific algorithms to extract the necessary information.

[1344] "AI means" refers to systems or devices that use artificial intelligence technology to perform processing or make decisions according to specific purposes.

[1345] "Features" refer to the distinctive attributes or patterns necessary to identify a person or object.

[1346] A "database" is a collection of information that systematically organizes related data and is constructed to enable efficient searching and updating.

[1347] "Verification" refers to the process of checking whether newly entered data matches the data and information acquired.

[1348] An "unregistered person" refers to a person whose information has not been registered in the database in advance.

[1349] "Distinguishing" means recognizing a particular object as distinct from others.

[1350] An "alarm signal" is a signal that the system emits when it detects an abnormality, and includes audio and light warnings.

[1351] "Light" refers to visible light, a type of electromagnetic wave, which is used as a visually recognizable warning.

[1352] "Sound" is a physical phenomenon that is transmitted through vibrations in the air and is used as an alarm that can be recognized through the ears.

[1353] "Reporting" refers to the act of informing an external agency or person in charge of an abnormality or emergency.

[1354] A "log" is data that records all the processes performed and events that occur while the system is running.

[1355] "Smart glasses" are glasses-type devices that have built-in displays, cameras, and communication functions, and have the ability to display information and acquire images.

[1356] A "display" is a display device for displaying images and text information.

[1357] "Emotional state" refers to the type and intensity of emotions a person is experiencing, such as tension, anxiety, anger, or other psychological conditions.

[1358] "Analysis means" refers to a method or device for analyzing specific data, and in this system includes emotion analysis using AI.

[1359] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video, thereby enhancing the security of the monitored object. In addition, by combining it with an emotion engine using AI technology, it is possible to identify the emotional state of suspicious individuals, realizing more accurate crime prevention measures. This system is composed of surveillance cameras, a server, a terminal, and an emotion engine, which operate in cooperation with each other.

[1360] System configuration

[1361] Surveillance cameras: These are devices installed inside and outside buildings and facilities that capture video in real time. Surveillance cameras are connected to servers and terminals via a network.

[1362] Server: This is the central device that analyzes the captured video using an AI model and recognizes people in the video. It compares the characteristics of the recognized people with a database to identify unregistered people. It also analyzes the person's emotional state using an emotion engine.

[1363] Terminal: A device that receives images sent from surveillance cameras and periodically sends them to a server, and also receives alarm signals from the server and activates light and sound alarms within the building.

[1364] Emotion Engine: This engine analyzes the facial expressions and movements of people in the video to determine their emotional state in real time. Based on the detected emotional state, it issues additional alerts or reports as needed.

[1365] Specific program description

[1366] The server performs video analysis using software such as Python, OpenCV, dlib, and Keras. Images captured by surveillance cameras are first captured in real time and then sent to the server at regular intervals. The server analyzes the received images, recognizes people's faces using an AI model, and determines their emotional state using an emotion engine. The analysis results are displayed on the smart glasses or a display.

[1367] Hardware used:

[1368] 1. Surveillance cameras

[1369] 2. Server

[1370] 3. Terminal

[1371] 4. Smart Glasses

[1372] Software used:

[1373] 1. Python

[1374] 2. OpenCV

[1375] 3. dlib

[1376] 4. Keras

[1377] Specific examples

[1378] Example 1: If the candidate is eligible, they will pass.

[1379] When a user enters a building, the device captures video from the security cameras installed in the building and sends it to the server. The server receives the video and analyzes the person using an AI model, and an emotion engine detects their normal state. The analyzed person's characteristics are compared with a database to confirm that they are a registered person. No alarm signal is issued, so the system takes no action. The user can enter the building without any problems.

[1380] Example 2: When a suspicious person with an abnormal emotional state passes by

[1381] Late at night, the device captures footage sent from the surveillance camera and sends it to the server. The server receives the footage and analyzes the person using an AI model, and the emotion engine detects that the person is in a very tense state. The analyzed person's characteristics are compared with a database to identify unregistered people. Having identified an unregistered person in a tense state, an alarm signal is sent to the device. The server then transmits the information to an external reporting point. The device receives the alarm signal and activates a stronger-than-normal light and sound alarm within the building. The server then reports information about the suspicious person and their state of tension to the security company. This entire process is recorded in a log, allowing an administrator to review the situation later.

[1382] Prompt Sentence Examples

[1383] 1. "Load an image dataset and crop it to center on a person's face."

[1384] 2. "Train a CNN model with emotion labels (e.g., anger, neutral, happy)."

[1385] 3. "Save the emotion recognition model and implement it so that it can analyze video input in real time."

[1386] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1387] System program processing flow

[1388] Step 1:

[1389] Capture and Send

[1390] Description: The device captures video from a surveillance camera in real time and transmits it to a server.

[1391] Input: Real-time video captured from a surveillance camera.

[1392] Output: Sending video data to the server.

[1393] How it works: A surveillance camera monitors a specific area and captures the video in real time, then periodically sends each frame of the captured video to a server.

[1394] Step 2:

[1395] Video reception and pre-processing

[1396] Description: The server receives the video sent from the terminal and performs preprocessing.

[1397] Input: Video data sent from the device.

[1398] Output: Preprocessed video data.

[1399] How it works: The server temporarily stores the received video data and performs preprocessing such as noise removal and resolution adjustment. This preprocessing ensures that subsequent AI analysis can be performed with high accuracy.

[1400] Step 3:

[1401] person recognition

[1402] Description: The server analyzes the pre-processed footage and uses AI means to recognize people within the footage.

[1403] Input: Preprocessed video data.

[1404] Output: Information about the recognized person.

[1405] Operation: For video analysis, a face detection algorithm (e.g., Haar-cascade or dlib) is used to recognize human faces in the video. Feature data of the recognized faces is extracted.

[1406] Step 4:

[1407] Feature Matching

[1408] Description: The server checks the characteristics of the recognized person against a pre-registered database.

[1409] Input: Recognized person feature data,database.

[1410] Output: Matching result (enrolled or not enrolled).

[1411] Operation: The feature data of a person obtained through analysis is compared with the feature data of a person previously registered in a database. This comparison identifies whether the person is registered or not.

[1412] Step 5:

[1413] Emotion analysis

[1414] Description: In parallel with the facial recognition results, the server uses an emotion engine to analyze the emotional state of people in the video.

[1415] Input: Recognized person's facial features, emotion engine.

[1416] Output: Emotional state (tension, anxiety, anger, etc.).

[1417] Movement: The emotion engine analyzes facial expressions and movements from facial feature data, labels the analyzed emotional states, and identifies specific emotional states such as tension, anxiety, and anger.

[1418] Step 6:

[1419] Determining and issuing an alarm

[1420] Description: The server generates an alarm signal according to the identified unregistered person and their emotional state and sends it to the terminal.

[1421] Input: Matching results, sentiment analysis results.

[1422] Output: Alarm signal.

[1423] Operation: If an unregistered person is detected and an abnormal emotional state (e.g., tension or anger) is detected, the server generates an alarm signal and sends it to the device. The device receives this signal and activates a light and sound alarm.

[1424] Step 7:

[1425] Reporting and logging

[1426] Description: The server automatically reports information about unregistered people and their emotional state to an external reporting destination, and saves the results of the process as a log.

[1427] Input: Information about unregistered persons, contact information.

[1428] Output: Report data, log data.

[1429] How it works: The server reports information about suspicious individuals and their emotional state in real time to security companies, administrators, and other reporting points. It also saves the results of this process in detail as a log so that administrators can check the situation later.

[1430] In this way, through each processing step, advanced crime prevention measures based on video data are realized.

[1431] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1432] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1433] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1434] [Fourth embodiment]

[1435] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1436] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1437] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1438] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1439] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1440] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1441] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1442] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1443] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1444] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1445] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1446] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1447] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1448] This invention is a system that enhances the security of monitored objects by capturing video in real time using a surveillance camera and recognizing and identifying people in the video. This system is composed of a surveillance camera, a server, and a terminal, which operate in cooperation with each other.

[1449] System configuration

[1450] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time. They are connected to devices via a network.

[1451] Terminal: Receives images sent from the surveillance cameras and periodically sends them to the server. It also receives alarm signals from the server and activates light and sound alarms within the building.

[1452] Server: Analyzes the captured video using an AI model and recognizes people in the video. The characteristics of the recognized people are compared with a database to identify unregistered people. If an unregistered person is identified, an alarm signal is sent to the terminal and an automatic report is made to an external reporting point.

[1453] Specific Modes for Carrying Out the Invention

[1454] Video capture and transmission

[1455] The device captures video from the surveillance camera in real time. The captured video is sent to the server frame by frame at regular intervals. This process ensures that the latest surveillance video is always provided to the server in a format that can be analyzed.

[1456] Video analysis and matching

[1457] The server analyzes the received video frames using an AI model. This AI model is based on facial recognition technology and recognizes the faces of people in the video and extracts their features. The extracted features are compared with a pre-registered database, which contains facial data of people involved in the building or facility.

[1458] Identifying suspicious people

[1459] If the result of the check is that there is no matching person in the database, the server identifies the person as an unregistered person (suspicious person), and it is determined that the identified unregistered person requires further action.

[1460] Activation of an alarm

[1461] The server sends an alarm signal to the terminal for any identified unregistered person. When the terminal receives the alarm signal, it immediately activates a light and sound alarm within the building to alert those in the vicinity. This alarm increases the likelihood that the suspicious person will leave the area, and allows those in the vicinity to become more vigilant.

[1462] Automatic reporting

[1463] At the same time, the server automatically sends information about the suspicious person to an external reporting point, including the video frame and the suspicious person's characteristics, so that the security company or relevant security department can respond immediately.

[1464] Log storage

[1465] The server records the results of the series of processes as a log. This log includes the results of video analysis, the identification of suspicious individuals, the issuance of alarms, and the details of reports. The log can be accessed and checked later by an administrator, helping to understand the situation and prevent recurrence.

[1466] Specific examples

[1467] Example 1: If the candidate is eligible, they will pass.

[1468] 1. User: The device captures video from a surveillance camera installed in the building and sends it to the server.

[1469] 2. Server: Receives the video and analyzes the person using an AI model.

[1470] 3. Server: Compares the analyzed person's characteristics with a database and confirms that the person is a registered person (e.g., an employee).

[1471] 4. Server: No alarm signal is generated, so no action is taken by the system. The user can enter the building without any problems.

[1472] Example 2: When a suspicious person passes through

[1473] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[1474] 2. Server: Receives the video and analyzes the person using an AI model.

[1475] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people.

[1476] 4. Server: Sends an alarm signal to the device.

[1477] 5. Terminal: Receives alarm signals and activates light and sound alarms within the building.

[1478] 6. Server: Automatically reports information about suspicious individuals to security companies.

[1479] 7. Server: This process is logged so that an administrator can review it later.

[1480] In this way, the present invention makes it possible to automatically and effectively strengthen the security of buildings and facilities.

[1481] The processing flow will be explained below.

[1482] Step 1:

[1483] The device captures video from the surveillance camera in real time, and the captured video is buffered in the device and awaits the next transmission timing.

[1484] Step 2:

[1485] The device transmits the captured video frames to the server at regular intervals. This transmission is carried out over a network, and the data is efficiently compressed to ensure real-time performance.

[1486] Step 3:

[1487] The server receives the video frames sent from the device, which are then immediately passed to the AI ​​analysis module.

[1488] Step 4:

[1489] The server's AI analysis module identifies people in the video frame and extracts their facial features, using a facial recognition algorithm.

[1490] Step 5:

[1491] The server compares the extracted person's features with a pre-registered facial database of building personnel. During the matching process, it calculates the degree of match between the features and identifies those with a high degree of match.

[1492] Step 6:

[1493] The server uses the results of the comparison to determine whether an unregistered person exists. If there is no match in the database, the person is identified as suspicious.

[1494] Step 7:

[1495] When the server identifies a suspicious person, it generates an alarm signal and sends it to the terminal. The alarm signal includes detailed information about the identified person (e.g., video frames and features).

[1496] Step 8:

[1497] The terminal receives the alarm signal and immediately activates a light and sound alarm within the building, with preset light and sound intensities and patterns.

[1498] Step 9:

[1499] At the same time, the server automatically reports the suspicious person's information to an external reporting point (e.g., a security company or administrator). The report includes important information (such as the suspicious person's characteristics, video frames, and the time of the incident).

[1500] Step 10:

[1501] The server records a detailed log of the process results, including the details of each step and a timestamp, and saves it so that an administrator can review the situation later.

[1502] Example 1

[1503] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1504] Conventional security systems often rely on manual monitoring, which can lead to incomplete monitoring and human error. Furthermore, existing video surveillance systems have difficulty recognizing and identifying people in real time, which can result in delays in detecting suspicious individuals. A system that can resolve these issues and perform monitoring more accurately and quickly is needed.

[1505] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1506] In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, means for analyzing the received video using a generative AI model and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for identifying unregistered people based on the results of the comparison, means for outputting an alarm signal for the identified unregistered people, means for issuing an alarm with light and sound based on the alarm signal, means for automatically reporting information about the unregistered person to an external reporting destination, and means for saving the series of processes as a log. This enables real-time recognition and identification of people, enabling quick and accurate detection of suspicious people and appropriate response.

[1507] A "surveillance camera" is a device that captures video footage in real time and transmits the video data over a network.

[1508] A "capturing means" is a method or device by which a surveillance camera generates a video stream and captures that video footage.

[1509] "Transmitting means" refers to a method or device for transmitting captured video data to a server via a network at regular intervals.

[1510] A "generative AI model" is an algorithm or program that uses machine learning techniques to recognize and identify people.

[1511] The "analyzing means" refers to a method or device for analyzing received video data and recognizing people in the video.

[1512] A "means for extracting features" is a method or device for extracting identifiable features such as the face or posture of a recognized person.

[1513] A "database" is a storage device or system that stores facial data and feature data of registered people and uses them for matching purposes.

[1514] The "matching means" refers to a method or device for comparing extracted feature data with registered data in a database to find a matching person.

[1515] The "means for identifying" refers to a method or device for detecting an unregistered person identified as a result of the matching and recognizing him or her as a suspicious person.

[1516] The "means for outputting an alert signal" refers to a method or device for generating and transmitting a signal to alert an identified unregistered person.

[1517] The "means for activating an alarm" refers to a method or device for actually activating an alarm using light and sound within the building based on the alarm signal received by the terminal.

[1518] "Means for reporting" refers to a method or device for automatically sending information about a suspicious person to an external reporting destination.

[1519] "Means for saving as a log" refers to a method or device for recording the results of a series of processes and saving them so that they can be checked later.

[1520] "Real-time" refers to the state in which video is processed immediately after it is captured, with almost no delay.

[1521] MODE FOR CARRYING OUT THE INVENTION

[1522] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video, thereby enhancing the security of the monitored object. This system is composed of surveillance cameras, a server, and a terminal, and each device operates in cooperation with each other.

[1523] System configuration

[1524] Surveillance cameras: These cameras are installed inside and outside buildings and facilities and capture video footage in real time. They are connected to devices via a network.

[1525] Terminal: Receives images sent from the surveillance camera and periodically sends them to the server. It also receives alarm signals from the server and activates alarms using light and sound.

[1526] Server: Analyzes the captured video using a generative AI model to recognize people in the video. It compares the characteristics of the recognized people with a database to identify unregistered people. If an unregistered person is identified, it sends an alarm signal to the device and automatically reports the incident to an external reporting point.

[1527] Hardware and Software

[1528] Surveillance cameras: Network cameras (e.g. IP cameras) are used.

[1529] Terminal: A computing device (e.g., PC, Raspberry Pi) for processing and transmitting video.

[1530] Server: Use a high-performance server (e.g., AWS EC2, on-premise server) and database (e.g., MySQL).

[1531] Generative AI models: Use libraries for AI analysis (e.g., TensorFlow, OpenCV).

[1532] Video capture and transmission

[1533] The device captures video from the surveillance camera in real time. The captured video is sent to the server frame by frame at regular intervals (for example, 30 frames per second). This process ensures that the latest surveillance video is always provided to the server in a format that can be analyzed.

[1534] Video analysis and matching

[1535] The server analyzes the received video frames using a generative AI model. This AI model is based on facial recognition technology and recognizes the faces of people in the video and extracts their features. The extracted features are compared with a pre-registered database, which contains facial data of people involved in the building or facility.

[1536] Identifying suspicious people

[1537] If the result of the check is that there is no matching person in the database, the server identifies the person as an unregistered person (suspicious person), and it is determined that the identified unregistered person requires further action.

[1538] Activation of an alarm

[1539] The server sends an alarm signal to the terminal for any identified unregistered person. When the terminal receives the alarm signal, it immediately activates a light and sound alarm within the building to alert those in the vicinity. This alarm increases the likelihood that the suspicious person will leave the area, and allows those in the vicinity to become more vigilant.

[1540] Automatic reporting

[1541] At the same time, the server automatically sends information about the suspicious person to an external reporting organization (e.g., a security company or the police), which can then immediately respond by sending a report containing the video frame and the suspicious person's characteristics.

[1542] Log storage

[1543] The server records the results of the series of processes as a log. This log includes the results of video analysis, the identification of suspicious individuals, the issuance of alarms, and the details of reports. The log can be accessed and checked later by an administrator, helping to understand the situation and prevent recurrence.

[1544] Specific examples

[1545] Example 1: If the candidate is eligible, they will pass.

[1546] 1. Terminal: Captures video from surveillance cameras installed in the building and sends it to the server.

[1547] 2. Server: Receives the video and analyzes the person using a generative AI model.

[1548] 3. Server: The analyzed person's characteristics are compared with the database to confirm that the person is a registered person (for example, an employee).

[1549] 4. Server: No alarm signal is generated, so no action is taken by the system. As a result, the user can enter the building without any problems.

[1550] Example 2: When a suspicious person passes through

[1551] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[1552] 2. Server: Receives the video and analyzes the person using a generative AI model.

[1553] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people (suspicious individuals).

[1554] 4. Server: Sends an alarm signal to the device.

[1555] 5. Terminal: Receives alarm signals and activates light and sound alarms within the building.

[1556] 6. Server: Automatically reports information about suspicious individuals to security companies. Specifically, the report is made by sending an HTTP request to the security company's API.

[1557] 7. Server: This process is logged so that an administrator can review it later.

[1558] Prompt Sentence Examples

[1559] "Please explain how a system can capture video in real time from security cameras installed in an office building, analyze the video using a generative AI model (e.g., OpenCV), identify suspicious individuals, and then automatically report this information to a security company."

[1560] By using this prompt sentence, the generative AI model can be made to explain the specific processing details of the system.

[1561] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1562] Step 1:

[1563] The terminal captures video from a surveillance camera in real time. The input is the video stream from the surveillance camera, and the output is the captured video frames. Specifically, the terminal acquires the stream based on the IP address of the surveillance camera, captures the frames, and stores them in memory.

[1564] Step 2:

[1565] The device sends the captured video to the server at regular intervals (e.g., 30 frames per second). The input is the captured video frame, and the output is the transmission to the server. Specifically, the device uses HTTP or WebSocket to packetize the frame data and send it to the server.

[1566] Step 3:

[1567] The server receives video frames sent from the terminal. The input is the video frame from the terminal, and the output is the storage of the received frame. Specifically, the server receives packets through the network interface, reconstructs them into frames, and stores them in memory.

[1568] Step 4:

[1569] The server analyzes the received video frames using a generative AI model (e.g., TensorFlow or OpenCV). The input is the stored frame, and the output is the analyzed person's facial feature data. Specifically, the server applies a face detection algorithm to the frame, extracts facial feature points, and stores them as vectors.

[1570] Step 5:

[1571] The server compares the extracted feature data with the database. The input is the extracted vector data, and the output is the comparison result (registered or unregistered). Specifically, the server uses an SQL query to compare the person feature vectors in the database and calculate the similarity.

[1572] Step 6:

[1573] The server identifies an unregistered person as a result of the matching. The input is the matching result, and the output is the identification result of the unregistered person. Specifically, if the similarity threshold is not exceeded, the server identifies the person as unregistered and stores that information in a structure.

[1574] Step 7:

[1575] The server sends an alarm signal to the terminal for the identified unregistered person. The input is the identification result of the unregistered person, and the output is the alarm signal to the terminal. Specifically, the server sends an alarm signal data packet to the terminal using the TCP / IP protocol.

[1576] Step 8:

[1577] The terminal receives the alarm signal and activates the light and sound alarms in the building. The input is the alarm signal from the server, and the output is the actual activation of the alarm sound and light. Specifically, the terminal analyzes the received signal and sends a signal to the circuitry that controls the speaker and light.

[1578] Step 9:

[1579] The server automatically reports information about unregistered individuals to external reporting destinations. The input is the identification result of the unregistered individual and related data, and the output is a report to the external reporting destination. Specifically, the server uses an API request or email sending function to send the information to the specified reporting destination.

[1580] Step 10:

[1581] The server saves the results of a series of processes as a log. The input is the result data of each process, and the output is the saved log file. Specifically, the server records the results of the process in text format or database format so that they can be checked later.

[1582] (Application example 1)

[1583] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1584] While conventional security systems have the ability to detect suspicious individuals inside and outside a building in real time and issue an alarm, they face the challenge of making it difficult for administrators to monitor and control the system from a remote location. Furthermore, when an unregistered individual is identified, the response can be delayed, which is insufficient for modern security needs that require a rapid response. There is a demand for a security system that can solve these issues and provide a more effective and prompt response.

[1585] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1586] In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, AI means for analyzing the received video and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for identifying unregistered people based on the comparison result, means for outputting an alarm signal for the identified unregistered people, means for issuing an alarm with light and sound based on the alarm signal, means for automatically reporting information about the unregistered people to an external reporting destination, means for saving the series of processes as a log, means for viewing the real-time video on a smartphone, means for sending a push notification to the smartphone when an unregistered person is detected, and means for remotely issuing an alarm from the smartphone. This allows an administrator to monitor in real time even from a remote location and immediately issue an alarm.

[1587] A "surveillance camera" is a device that captures real-time footage of the inside and outside of a building or facility.

[1588] "Real-time video" refers to video footage captured by a surveillance camera.

[1589] "Capturing" means taking and recording video.

[1590] "Sending at regular intervals" means sending the captured video to the server periodically.

[1591] "Analyze" means to analyze the video and understand its content.

[1592] "AI means" refers to a method or device for recognizing people in video using artificial intelligence technology.

[1593] The "pre-registered database" refers to a database that holds pre-registered face image data.

[1594] "Matching" means comparing the characteristics of a recognized person with data in a database.

[1595] "Identify" refers to identifying a person based on specific criteria.

[1596] An "alarm signal" is a signal that is transmitted when an abnormality is detected.

[1597] "Activating an alarm with light and sound" refers to using light and sound to notify of an abnormality.

[1598] "Automatically report to the reporting destination" means automatically reporting an abnormality to a designated external agency.

[1599] "Saving as a log" refers to recording and saving the results of processing and the history of events.

[1600] A "smartphone" is a mobile device that can connect to the Internet and run applications.

[1601] "Push notification" refers to the function of sending notifications from a server to a smartphone in real time.

[1602] "Remotely activating an alarm" means remotely operating and activating an alarm from a distance.

[1603] System configuration

[1604] The system for implementing the present invention comprises the following main hardware and software components:

[1605] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time, which is then transmitted to a server over a network.

[1606] Server: Receives the captured video and analyzes it using an AI model. The server compares the feature data with a database to identify unregistered individuals.

[1607] Smartphone: Runs an application that allows administrators to view video footage in real time and remotely control alarms.

[1608] Cloud services: Utilize cloud services such as AWS and Google Cloud for video analysis and data management.

[1609] Program processing explanation

[1610] Video capture and transmission

[1611] Surveillance cameras capture images of the inside and outside of buildings in real time. The captured images are periodically sent to a server, frame by frame. Because the image data is sent over a network, the communication environment between the camera and the server is important.

[1612] Video analysis and matching

[1613] The server analyzes the received video using an AI model. Specifically, it detects people in the video frame and extracts their facial features. This analysis uses a facial recognition algorithm using OpenCV and Dlib. The extracted facial features are compared with a pre-registered database, and if there is no match, the person is identified as unregistered.

[1614] Alerts and notifications

[1615] When an unregistered person is identified, the server generates an alarm signal, triggering light and sound alarms within the building, and sends a push notification to the administrator's smartphone, immediately informing them of the abnormality. The push notification is sent using Firebase cloud messaging.

[1616] Log saving and reporting

[1617] The server saves the results of the series of processes as a log. The saved log includes the results of video analysis, the identification of suspicious individuals, and the history of alarm activation. This allows administrators to check the details later. In addition, information on unregistered individuals is automatically sent to external reporting points.

[1618] Specific examples

[1619] Example 1: If the candidate is eligible, they will pass.

[1620] 1. The user (surveillance camera) captures video from inside the facility and sends it to the server.

[1621] 2. The server uses an AI model to analyze the person and match them with a database.

[1622] 3. If the match is with a registered person, no alarm will be triggered and the system will operate normally.

[1623] Example 2: When a suspicious person passes through

[1624] 1. The user (surveillance camera) captures video from inside the facility and sends it to the server.

[1625] 2. The server uses an AI model to analyze the person and match them with a database.

[1626] 3. If an unregistered person is identified, the server generates an alarm signal and triggers an alarm.

[1627] 4. At the same time, the server sends a push notification to the administrator's smartphone and also sends the information to external reporting points such as security companies.

[1628] Prompt Sentence Examples

[1629] For example, by inputting the following prompt sentence into the generative AI model, you can get advice and adjustments to improve the accuracy of surveillance camera video analysis.

[1630] "We are developing an AI system that analyzes surveillance camera footage and performs person recognition. Could you please tell us some techniques to improve accuracy in identifying unregistered people by accurately capturing the characteristics of people in the footage?"

[1631] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1632] Step 1:

[1633] A user (surveillance camera) captures video of the inside and outside of a building in real time. The captured video is converted into frames within the camera. At that time, the video frames are given a time stamp.

[1634] Input: Real-time video

[1635] Output: Captured video frames

[1636] Step 2:

[1637] The device periodically transmits captured video frames to the server, where they are encoded and split into packets over the network.

[1638] Input: Captured video frames

[1639] Output: Video packets sent to the server

[1640] Step 3:

[1641] The server decodes the received video packets and reconstructs them into the original video frames, which are then stored in a buffer.

[1642] Input: Video packets sent to the server

[1643] Output: Decoded and buffered video frames

[1644] Step 4:

[1645] The server uses an AI model to analyze the video frames in the buffer and detect faces in the video, extracting facial feature points using OpenCV and the Dlib library.

[1646] Input: Buffered video frames

[1647] Output: Detected facial feature points

[1648] Step 5:

[1649] The server compares the extracted facial feature points with pre-registered face data in a database, and calculates the similarity between the feature vectors.

[1650] Input: Detected facial feature points, registered face data in the database

[1651] Output: Matching result (registered or unregistered)

[1652] Step 6:

[1653] If the matched person is an unregistered person, the server generates an alarm signal, which includes facial feature information and a video frame of the unregistered person.

[1654] Input: Matching result is unregistered person

[1655] Output: Alarm signal

[1656] Step 7:

[1657] The terminal receives the alarm signal and activates a light and sound alarm within the building. The alarm is visually and audibly activated using a speaker and a light.

[1658] Input: Alarm signal

[1659] Output: Light and sound alarm activation

[1660] Step 8:

[1661] The server simultaneously sends an alarm signal to the administrator's smartphone as a push notification, using Firebase cloud messaging to deliver the notification in real time.

[1662] Input: Alarm signal

[1663] Output: Push notification to smartphone

[1664] Step 9:

[1665] The server automatically reports information about unregistered individuals to an external reporting point, including video frames and details of the alert.

[1666] Input: Unregistered person information, alarm signal

[1667] Output: Automatic notification to external reporting destination

[1668] Step 10:

[1669] The server saves the results of the series of processes as a log, which includes video analysis results, alarm activation history, suspicious person information, and report history, and can be checked later by the administrator.

[1670] Input: A set of processing results

[1671] Output: Saved logs

[1672] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1673] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video, thereby enhancing the security of the monitored object. In addition, by combining it with an emotion engine using AI technology, it is possible to identify the emotional state of suspicious individuals, realizing more accurate crime prevention measures. This system is composed of surveillance cameras, a server, a terminal, and an emotion engine, which operate in cooperation with each other.

[1674] System configuration

[1675] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time. They are connected to devices via a network.

[1676] Terminal: Receives images sent from the surveillance cameras and periodically sends them to the server. It also receives alarm signals from the server and activates light and sound alarms within the building.

[1677] Server: Analyzes the captured video using an AI model to recognize people in the video. It compares the features of the recognized people with a database to identify unregistered people. It also analyzes the emotional state of the people using an emotion engine.

[1678] Emotion Engine: Analyzes facial expressions and movements of people in the video to determine their emotional state in real time, and issues additional alerts or notifications as needed based on the detected emotional state.

[1679] Specific Modes for Carrying Out the Invention

[1680] Video capture and transmission

[1681] The device captures video from the surveillance camera in real time. The captured video is sent to the server frame by frame at regular intervals. This process ensures that the latest surveillance video is always provided to the server in a format that can be analyzed.

[1682] Video analysis and matching

[1683] The server analyzes the received video frames using an AI model. This AI model is based on facial recognition technology and recognizes the faces of people in the video and extracts their features. The extracted features are compared with a pre-registered database, which contains facial data of people involved in the building or facility.

[1684] Emotion analysis using an emotion engine

[1685] In parallel with the facial recognition results, the server uses an emotion engine to analyze the emotional state of people in the video, which determines specific emotional states such as tension, anxiety, or anger based on the person's facial expressions and movements.

[1686] Identifying suspicious people and emotional states

[1687] By combining the matching results with the results of emotion analysis, the server identifies unregistered individuals and evaluates their emotional state. If abnormal emotional states, such as tension, anxiety, or anger, are detected, the server determines that more immediate action is required.

[1688] Activation of an alarm

[1689] The server generates an alarm signal according to the identified unregistered person and their emotional state and transmits it to the terminal. For example, if an unregistered person in a very tense state is identified, the terminal will issue a stronger alarm than usual.

[1690] Automatic reporting

[1691] At the same time, the server automatically reports the suspicious person's information and emotional state to an external reporting point (e.g., a security company or administrator). The report includes video frames, characteristic information, and the person's emotional state, allowing the security company to respond immediately.

[1692] Log storage

[1693] The server records a detailed log of the entire process, including the results of video analysis, the identification of suspicious individuals, their emotional state, whether an alarm was issued, and the details of the report, and the log is saved so that administrators can review the situation later.

[1694] Specific examples

[1695] Example 1: If the candidate is eligible, they will pass.

[1696] 1. User: The device captures video from a surveillance camera installed in the building and sends it to the server.

[1697] 2. Server: Receives the video and analyzes the person using the AI ​​model. The emotion engine detects the normal state.

[1698] 3. Server: Compares the analyzed person's characteristics with the database and verifies that the person is registered.

[1699] 4. Server: No alarm signal is generated, so no action is taken by the system. The user can enter the building without any problems.

[1700] Example 2: When a suspicious person with an abnormal emotional state passes by

[1701] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[1702] 2. Server: Receives the video and analyzes the person using an AI model. The emotion engine detects when the person is extremely nervous.

[1703] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people.

[1704] 4. Server: Having identified an unregistered person in a tense state, the server sends an alarm signal to the terminal. The server then sends the information to an external reporting point.

[1705] 5. Terminal: Receives the alarm signal and activates stronger than normal light and sound alarms within the building.

[1706] 6. Server: Reports information about suspicious individuals and the state of tension to the security company.

[1707] 7. Server: This process is logged so that an administrator can review it later.

[1708] In this way, the present invention automatically and effectively strengthens the security of buildings and facilities, and by taking into account emotional states in particular, more advanced crime prevention measures become possible.

[1709] The processing flow will be explained below.

[1710] Step 1:

[1711] The device captures video from the surveillance camera in real time, and the captured video is temporarily stored in the device, waiting for the next transmission timing.

[1712] Step 2:

[1713] The device transmits captured video frames to the server at regular intervals, and the transmitted data is compressed to reduce network load.

[1714] Step 3:

[1715] The server processes video frames received from the device in real time. The video frames are first passed to the AI ​​analysis module.

[1716] Step 4:

[1717] The server's AI analysis module identifies people in the video frame and extracts their facial features, using facial recognition algorithms.

[1718] Step 5:

[1719] The server then matches the extracted person's characteristics with existing data in a database that contains building personnel information to identify matches.

[1720] Step 6:

[1721] The server identifies unregistered individuals based on the matching results. If there is no matching person in the database, the individual is recognized as an unregistered (suspicious) individual.

[1722] Step 7:

[1723] In parallel, the server runs an emotion engine that analyzes the facial expressions and movements of people in the video, using AI to identify emotional states such as tension, anxiety, or anger.

[1724] Step 8:

[1725] The server integrates the analysis results of the emotion engine with the registered information of the person and checks for the presence of an unregistered person with an abnormal emotional state.

[1726] Step 9:

[1727] The server generates an alarm signal for an unregistered person with an abnormal emotional state and sends it to the terminal, which includes the analyzed feature information and the emotional state.

[1728] Step 10:

[1729] The terminal receives the alarm signal and immediately activates a light and sound alarm within the building, which is stronger than normal and alerts those around.

[1730] Step 11:

[1731] At the same time, the server automatically sends the information about the unregistered person and the emotion analysis results to an external reporting point (e.g., a security company or facility manager). The report includes detailed characteristic information and the person's emotional state.

[1732] Step 12:

[1733] The server saves all processing results as logs, which contain detailed records of timestamps, processing details, alarm activation history, and report details. Administrators can later check and analyze these logs.

[1734] In this way, the present invention effectively links surveillance cameras, terminals, servers, and emotion engines to automatically achieve real-time surveillance and anomaly detection with high accuracy.

[1735] Example 2

[1736] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1737] Conventional security systems are equipped with video surveillance using surveillance cameras and identification functions using facial recognition, but lack the ability to detect abnormalities based on a person's emotional state. As a result, it is difficult to detect abnormal emotional states, such as when a suspicious person is nervous or anxious, which reduces the accuracy of security measures. In addition, there is also the issue of not only simply detecting unregistered individuals, but also needing to respond appropriately according to the person's emotional state.

[1738] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, AI means for analyzing the received video and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for analyzing the emotional state of people in the video in real time using emotion analysis means and detecting abnormal emotional states, and means for generating an additional alarm signal in response to the detected abnormal emotional state. This makes it possible not only to detect unregistered people but also to detect abnormalities based on emotional states and respond quickly.

[1739] A "surveillance camera" is a device installed inside or outside a building or facility to capture video in real time.

[1740] The "terminal" is a device or software that receives images sent from the surveillance camera and periodically sends them to the server. The terminal also receives alarm signals from the server and activates light and sound alarms within the building.

[1741] The "server" is a computer system that analyzes the captured video and recognizes people in the video. The server also performs emotion analysis to assess the emotional state of people and issue alerts if necessary.

[1742] The "Emotion Engine" is a system that uses AI technology to analyze the facial expressions and movements of people in video footage and determine their emotional state in real time.

[1743] An "AI model" is an algorithm or software that uses artificial intelligence technology to recognize the faces of people in video and extract their features.

[1744] A "database" is a data storage that stores characteristic information (face data) of people who have been registered in advance.

[1745] An "alarm signal" is a signal sent to a terminal to trigger an alarm in response to an identified unregistered person and their abnormal emotional state.

[1746] "Light and sound alarms" are visual and audible warnings that are activated within a building based on an alarm signal, such as flashing lights and audible alarms.

[1747] "Automatic reporting" is a process in which the server notifies an external contact point (such as a security company or administrator) of information about a suspicious person and their emotional state. Notifications are sent via email, text message, or API communication.

[1748] A "log" is a detailed record of a series of processing results, and is data that is saved for later review by an administrator. Logs include analysis results, identification of suspicious individuals, emotional states, triggered alarms, and report contents.

[1749] MODE FOR CARRYING OUT THE INVENTION

[1750] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video. Furthermore, by combining it with an emotion engine using AI technology, it can identify the emotional state of suspicious individuals and achieve highly accurate crime prevention measures. This system is composed of surveillance cameras, a server, a terminal, and an emotion engine, which work in cooperation with each other.

[1751] System configuration

[1752] Surveillance cameras: These are installed inside and outside buildings and facilities and capture video in real time. They are connected to devices via a network.

[1753] Terminal: Receives images sent from the surveillance cameras and periodically sends them to the server. It also receives alarm signals from the server and activates light and sound alarms within the building.

[1754] Server: Analyzes the captured video using an AI model to recognize people in the video. It compares the features of the recognized people with a database to identify unregistered people. It also analyzes the emotional state of the people using an emotion engine.

[1755] Emotion Engine: Analyzes facial expressions and movements of people in the video to determine their emotional state in real time, and issues additional alerts or notifications as needed based on the detected emotional state.

[1756] Specific Modes for Carrying Out the Invention

[1757] The server has a means to capture video from the surveillance cameras in real time. Specifically, the surveillance cameras continuously capture video inside and outside the building or facility and send the video data to the terminal. The terminal converts this data into capture frames at regular intervals and sends them to the server via the network.

[1758] The server analyzes the received video frames using an AI model. This AI model uses libraries such as OpenCV and Dlib to recognize people's faces and extract their features. The extracted features are compared with a pre-registered database containing facial data of people associated with the building or facility. If the comparison identifies an unregistered person, the server generates an alarm signal.

[1759] Additionally, the server uses an emotion engine to analyze the emotional state of people in the video in real time. This emotion engine, powered by services such as Microsoft Azure Cognitive Services and Google Cloud Vision, analyzes facial expressions and movements to identify emotions such as tension, anxiety, and anger. If an abnormal emotional state is detected, the server generates an additional alarm signal, enabling a rapid response.

[1760] The generated alarm signal is sent to a terminal, which activates the building's alarm system (e.g., audio alarm or flashing light), and the server automatically reports information about the suspicious person and their emotional state to an external reporting point via email, text message, or API communication.

[1761] Finally, the server records a detailed log of the entire process, including the results of video analysis, the identification of suspicious individuals, their emotional state, whether an alarm was activated, and the details of the report, which is then saved for later review by an administrator.

[1762] Specific examples

[1763] Example 1: If the candidate is eligible, they will pass.

[1764] 1. User: The device captures video from a surveillance camera installed in the building and sends it to the server.

[1765] 2. Server: Receives the video and analyzes the person using an AI model. The emotion engine detects the normal state.

[1766] 3. Server: Compares the analyzed person's characteristics with the database and verifies that the person is registered.

[1767] 4. Server: No alarm signal is generated, so no action is taken by the system. The user can enter the building without any problems.

[1768] Example 2: When a suspicious person with an abnormal emotional state passes by

[1769] 1. Terminal: Captures video sent from a surveillance camera late at night and sends it to a server.

[1770] 2. Server: Receives the video and analyzes the person using an AI model. The emotion engine detects when the person is extremely nervous.

[1771] 3. Server: Compares the analyzed person's characteristics with the database and identifies unregistered people.

[1772] 4. Server: Having identified an unregistered person in a tense state, the server sends an alarm signal to the terminal. The server then sends the information to an external reporting point.

[1773] 5. Terminal: Receives the alarm signal and activates stronger than normal light and sound alarms within the building.

[1774] 6. Server: Reports information about suspicious individuals and the state of tension to the security company.

[1775] 7. Server: This process is logged so that an administrator can review it later.

[1776] Prompt Sentence Examples

[1777] Example prompt sentence:

[1778] "Analyze the emotional state of a person in a video captured by a surveillance camera late at night. Explain how the system would respond if it detected an abnormal level of tension."

[1779] In this way, the present invention automatically and effectively strengthens the security of buildings and facilities, and by taking into account emotional states in particular, more advanced crime prevention measures become possible.

[1780] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1781] Step 1:

[1782] The device captures video from a surveillance camera in real time and sends it to the server frame by frame at regular intervals. The input to this process is the video data from the surveillance camera, and the output is the captured frames sent to the server. This process requires a network connection to transfer the video data to the server via the Internet.

[1783] Step 2:

[1784] The server receives video frames sent from the surveillance camera. The input to this process is the video frames sent from the device, and the output is video data in an analyzable format. The received video data is temporarily stored in the server's storage and then proceeds to the next analysis step.

[1785] Step 3:

[1786] The server uses an AI model to analyze the received video frames and recognize the faces of people in the video. The input to this process is the received video frames, and the output is identified facial feature data. Specifically, an AI model (e.g., OpenCV or Dlib library) is used to run a face detection algorithm and extract the location and feature points of the face.

[1787] Step 4:

[1788] The server compares the feature data extracted by facial recognition with a pre-registered database. The input of this process is the identified facial feature data, and the output is the matching result. For matching, the server checks whether the face data matches the face data in the database, and if it does, it determines that the person is registered.

[1789] Step 5:

[1790] The server uses an emotion analysis engine to analyze the emotional state from the extracted facial feature data. The input for this process is the facial feature data after matching, and the output is the emotional state determination result. The emotion analysis engine uses APIs such as Microsoft Azure Cognitive Services and Google Cloud Vision to identify emotions such as tension, anxiety, and anger from facial expressions.

[1791] Step 6:

[1792] The server identifies unregistered individuals and abnormal emotional states based on the analysis results. The input to this process is the results of facial recognition and emotional analysis, and the output is alarm data containing suspicious individuals and their emotional states. If an abnormal emotional state is detected, the server determines that the individual is suspicious and requires immediate action.

[1793] Step 7:

[1794] The server generates an alarm signal according to the identified unregistered person and their emotional state, and transmits it to the terminal. The input of this process is the alarm data, and the output is the alarm signal. Once the alarm signal is generated, it is transmitted to the terminal.

[1795] Step 8:

[1796] The terminal receives the alarm signal and activates optical and audible alarms in the building. The input of this process is the received alarm signal and the output is the activated alarm. The terminal activates the alarm system and provides visual and audible warnings.

[1797] Step 9:

[1798] The server automatically reports information about suspicious individuals and their emotional state to an external reporting destination. The input to this process is the suspicious individual's information and emotional state, and the output is the sent report message. Reports are sent via email, text message, or API communication.

[1799] Step 10:

[1800] The server records a detailed log of the results of each process, with the input being the results of each process step and the output being a saved log file, allowing administrators to review and respond to the situation at a later time.

[1801] (Application example 2)

[1802] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1803] Conventional surveillance systems have the ability to recognize people in real time using video transmitted from surveillance cameras and issue alarms, but this alone is insufficient to determine the emotional state of a suspicious individual, limiting the accuracy of security responses. Conventional systems cannot respond when a suspicious individual has a calm expression or when preventive measures must be taken before the individual engages in suspicious behavior. To solve this problem, a system is needed that can analyze emotional states in real time and respond immediately. Therefore, the present invention aims to achieve more accurate crime prevention measures by instantly identifying the emotional state of a suspicious individual and issuing an alarm.

[1804] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1805] In this invention, the server includes means for capturing video from a surveillance camera in real time, means for transmitting the captured video at regular intervals, AI means for analyzing the received video and recognizing people in the video, means for comparing the characteristics of the recognized people with a pre-registered database, means for identifying unregistered people based on the comparison result, means for outputting an alarm signal to the identified unregistered people, means for issuing an alarm with light and sound based on the alarm signal, means for automatically reporting information about the unregistered people to an external reporting destination, means for saving the series of processes as a log, means for displaying the faces of people in the video on a display mounted on the smart glasses, and means for analyzing the emotional state of the displayed people and issuing an alarm signal based on the emotional state. This enables the emotional state of people to be identified in real time, enabling prompt and appropriate crime prevention measures.

[1806] - A "surveillance camera" is a device installed to detect suspicious activity and for crime prevention purposes, and is used to capture real-time images of specific locations and acquire the footage.

[1807] "Real-time" refers to processing and responding to an event the moment it occurs, without delay.

[1808] "Video" means visual information captured by a camera or other photographic device, and refers to moving images displayed on a monitor or the like.

[1809] "Capture" refers to the operation or process of acquiring an image using a camera or other photographic device.

[1810] "Transmit" refers to the act of transferring acquired data or information to another device or system.

[1811] "Analysis" refers to the process of analyzing video and data in detail using computers and specific algorithms to extract the necessary information.

[1812] "AI means" refers to systems or devices that use artificial intelligence technology to perform processing or make decisions according to specific purposes.

[1813] "Features" refer to the distinctive attributes or patterns necessary to identify a person or object.

[1814] A "database" is a collection of information that systematically organizes related data and is constructed to enable efficient searching and updating.

[1815] "Verification" refers to the process of checking whether newly entered data matches the data and information acquired.

[1816] An "unregistered person" refers to a person whose information has not been registered in the database in advance.

[1817] "Distinguishing" means recognizing a particular object as distinct from others.

[1818] An "alarm signal" is a signal that the system emits when it detects an abnormality, and includes audio and light warnings.

[1819] "Light" refers to visible light, a type of electromagnetic wave, which is used as a visually recognizable warning.

[1820] "Sound" is a physical phenomenon that is transmitted through vibrations in the air and is used as an alarm that can be recognized through the ears.

[1821] "Reporting" refers to the act of informing an external agency or person in charge of an abnormality or emergency.

[1822] A "log" is data that records all the processes performed and events that occur while the system is running.

[1823] "Smart glasses" are glasses-type devices that have built-in displays, cameras, and communication functions, and have the ability to display information and acquire images.

[1824] A "display" is a display device for displaying images and text information.

[1825] "Emotional state" refers to the type and intensity of emotions a person is experiencing, such as tension, anxiety, anger, or other psychological conditions.

[1826] "Analysis means" refers to a method or device for analyzing specific data, and in this system includes emotion analysis using AI.

[1827] This invention is a system that uses surveillance cameras to capture video in real time and recognizes and identifies people in the video, thereby enhancing the security of the monitored object. In addition, by combining it with an emotion engine using AI technology, it is possible to identify the emotional state of suspicious individuals, realizing more accurate crime prevention measures. This system is composed of surveillance cameras, a server, a terminal, and an emotion engine, which operate in cooperation with each other.

[1828] System configuration

[1829] Surveillance cameras: These are devices installed inside and outside buildings and facilities that capture video in real time. Surveillance cameras are connected to servers and terminals via a network.

[1830] Server: This is the central device that analyzes the captured video using an AI model and recognizes people in the video. It compares the characteristics of the recognized people with a database to identify unregistered people. It also analyzes the person's emotional state using an emotion engine.

[1831] Terminal: A device that receives images sent from surveillance cameras and periodically sends them to a server, and also receives alarm signals from the server and activates light and sound alarms within the building.

[1832] Emotion Engine: This engine analyzes the facial expressions and movements of people in the video to determine their emotional state in real time. Based on the detected emotional state, it issues additional alerts or reports as needed.

[1833] Specific program description

[1834] The server performs video analysis using software such as Python, OpenCV, dlib, and Keras. Images captured by surveillance cameras are first captured in real time and then sent to the server at regular intervals. The server analyzes the received images, recognizes people's faces using an AI model, and determines their emotional state using an emotion engine. The analysis results are displayed on the smart glasses or a display.

[1835] Hardware used:

[1836] 1. Surveillance cameras

[1837] 2. Server

[1838] 3. Terminal

[1839] 4. Smart Glasses

[1840] Software used:

[1841] 1. Python

[1842] 2. OpenCV

[1843] 3. dlib

[1844] 4. Keras

[1845] Specific examples

[1846] Example 1: If the candidate is eligible, they will pass.

[1847] When a user enters a building, the device captures video from the security cameras installed in the building and sends it to the server. The server receives the video and analyzes the person using an AI model, and an emotion engine detects their normal state. The analyzed person's characteristics are compared with a database to confirm that they are a registered person. No alarm signal is issued, so the system takes no action. The user can enter the building without any problems.

[1848] Example 2: When a suspicious person with an abnormal emotional state passes by

[1849] Late at night, the device captures footage sent from the surveillance camera and sends it to the server. The server receives the footage and analyzes the person using an AI model, and the emotion engine detects that the person is in a very tense state. The analyzed person's characteristics are compared with a database to identify unregistered people. Having identified an unregistered person in a tense state, an alarm signal is sent to the device. The server then transmits the information to an external reporting point. The device receives the alarm signal and activates a stronger-than-normal light and sound alarm within the building. The server then reports information about the suspicious person and their state of tension to the security company. This entire process is recorded in a log, allowing an administrator to review the situation later.

[1850] Prompt Sentence Examples

[1851] 1. "Load an image dataset and crop it to center on a person's face."

[1852] 2. "Train a CNN model with emotion labels (e.g., anger, neutral, happy)."

[1853] 3. "Save the emotion recognition model and implement it so that it can analyze video input in real time."

[1854] The flow of the specific processing in the application example 2 will be described with r...

Claims

1. A means of capturing video from surveillance cameras in real time; means for transmitting the captured video at regular intervals; AI means to analyze received video and recognize people in the video; means for matching the characteristics of the recognized person with a pre-registered database; A means for identifying an unregistered person as a result of the matching; means for outputting a warning signal to the identified unregistered person; means for issuing an alarm with light and sound based on the alarm signal; means for automatically reporting information about the unregistered person to an external reporting destination; means for saving the series of processes as a log; A system including:

2. 2. The system according to claim 1, wherein the personal information registered in the database is information about people involved in the building.

3. 10. The system of claim 1, wherein the video analysis uses a facial recognition algorithm.

Images