System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving video data from a camera carried by a police officer; means for storing the received video data; means for analyzing the stored video data; means for generating an alert based on the analysis result and notifying the police officer; and means for receiving a response result from the police officer and updating a database.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When police officers conduct routine questioning and patrols, they are required to accurately judge their targets and identify signs of crime, and this requires skill and experience. However, these tasks place a heavy burden on each individual officer, and there are cases where errors of judgment or unfair responses can become problematic. In addition, preventing and resolving crimes efficiently, securing evidence, and maintaining trust with citizens are important issues in police work. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides a system that receives, stores, and analyzes video data from cameras carried by police officers, generates an alert, and notifies the police officers. The system includes a means for receiving video data from cameras carried by police officers in real time, a means for storing the received video data, a means for analyzing the stored video data, a means for generating an alert based on the analysis results and notifying the police officers, and a means for receiving the response results from the police officers and updating the database. This system supports the judgment of police officers and improves the efficiency and reliability of crime prevention and resolution.

[0006] A "police officer" is a public servant whose duties are to maintain law and order, prevent and investigate crime, and ensure public safety.

[0007] A "camera" is a device for recording video and audio, and specifically refers to those carried by police officers on their uniforms.

[0008] "Video data" refers to visual information captured by a camera that can be stored and transferred in digital format.

[0009] "Means for receiving" refers to devices or software for acquiring video data transmitted from a camera via a network.

[0010] "Storage means" refers to a device or software that stores the received video data in a storage device and makes it possible to retrieve the data as needed.

[0011] "Means of analysis" refers to AI algorithms or software that analyzes stored video data and detects specific patterns or anomalies.

[0012] "Means for generating an alert" refers to devices or software that create information to warn or alert police officers based on the results of the analysis.

[0013] "Means for notifying" means a device or software that transmits the generated alarm to the officer's terminal and displays a visual, audible or vibration alert.

[0014] "Means for receiving response results" refers to devices or software for obtaining feedback from police officers and data on the results of the response.

[0015] The "means for updating the database" refers to a device or software for recording the received response results in the database and storing them in association with existing data.

[0016] "GPS information" means data about a current location obtained using the Global Positioning System.

[0017] "Suspicious behavior" refers to behavior that differs from normal patterns of behavior and may indicate criminal or illegal activity.

[0018] An "anomaly" is a deviation from normal operating conditions or patterns of behavior that indicates some kind of problem. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention relates to a system that analyzes video data in real time in conjunction with a constantly running camera carried by a police officer. This system is composed of the following elements:

[0041] 1. Server

[0042] The server includes hardware and software for receiving and storing video data sent from police officers' terminals. It also includes an AI module that analyzes the stored video data. The server generates alerts based on the analysis results and sends them to the police officers' terminals. It also receives response results from police officers and updates the database.

[0043] 2. Terminal

[0044] The officer's terminal communicates with the camera and captures video data. It is equipped with a communication module that transmits video data to the server in real time. It also notifies officers of received alarms and issues warnings through visual, audible, and vibration alerts. It also provides an interface for officers to input the results of their response and transmits the data to the server.

[0045] 3. User (police officer)

[0046] Police officers use this system when patrolling or questioning police officers. By wearing cameras and devices, video data is sent to a server in real time, and if suspicious behavior or abnormalities are detected, they are immediately notified. Upon receiving the notification, the police officer takes appropriate action according to the situation and updates the database by entering the results into the device.

[0047] As a specific example of the operation, the following scenario can be considered.

[0048] Example 1: Supporting police questioning

[0049] server:

[0050] A police officer's camera captures a person while patrolling a busy street at night.

[0051] The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals.

[0052] If the result of the comparison indicates that there is a high probability that the person is a wanted criminal, the server immediately generates an alert and notifies the terminal.

[0053] Device:

[0054] The device receives the alert and displays the message "Possible wanted criminal" on the screen. The device immediately notifies police officers using audio alerts and vibration functions.

[0055] User (police officer):

[0056] Police officers will check this notification and question the person in question. If they are able to confirm the person's identity and find that the person is in fact wanted, the police officers will immediately arrest the person.

[0057] Example 2: Detecting signs of crime

[0058] server:

[0059] A police officer's camera patrolling a park during the day captures a scene of several young people gathered together and behaving abnormally.

[0060] The server analyzes the acquired video data using an AI module, and if a suspicious behavior pattern is detected, it determines that there is a "sign of a crime," generates an alert, and notifies the device.

[0061] Device:

[0062] The device receives the alarm notification, displays "Suspicious Activity Detected," and notifies the police. The device immediately issues a visual and audio alert.

[0063] User (police officer):

[0064] Police officers will review the notification, proceed to the scene, approach the group of young people in question, and investigate their behavior. If they find any suspicious items, they will conduct further investigations and take necessary action on the spot.

[0065] The above specific example is intended to enable immediate response at the scene of a crime and to prevent and resolve it quickly. This system is expected to improve the accuracy of police officers' judgment, and to increase the efficiency and reliability of police activities.

[0066] The processing flow will be explained below.

[0067] Step 1:

[0068] Device: Camera activation and video capture

[0069] The device activates the camera worn by the officer and begins capturing video data. The camera continues to capture video at 30 frames per second and transmits it to the device.

[0070] Step 2:

[0071] Terminal: Compression and transmission of video data

[0072] The device compresses and encrypts the video data acquired from the camera in real time, and then sends the compressed and encrypted video data to the server.

[0073] Step 3:

[0074] Server: Receiving and temporarily storing video data

[0075] The server receives the video data sent from the device and temporarily stores it, adding GPS information and a timestamp to the received video data.

[0076] Step 4:

[0077] Server: AI video analysis

[0078] An AI module on the server analyzes the temporarily stored video data in real time, using facial recognition and motion analysis algorithms to identify suspicious behavior and wanted criminals.

[0079] Step 5:

[0080] Server: Generate alerts

[0081] Based on the analysis results of the AI ​​module, the server generates an alert if necessary, which includes the analysis results, location information, and a timestamp.

[0082] Step 6:

[0083] Server: Alert notification

[0084] The server sends the generated alert to the terminal, which receives the alert and prepares to notify the police officer.

[0085] Step 7:

[0086] Terminal: Display of alert notification

[0087] The device notifies officers of received alarms visually and audibly, displays detailed information on the alarm on the device's screen, and alerts officers with a warning sound and vibration.

[0088] Step 8:

[0089] User (police officer): Response to alarm

[0090] Police officers will check the alarm on the device and take action according to the situation at the scene, such as questioning the suspect, checking the scene, and in some cases, responding to an emergency.

[0091] Step 9:

[0092] Device: Enter response results

[0093] Police officers use the terminal interface to input the results of their response, including the details of the police stop, any suspicious objects found, and information about the person they have apprehended.

[0094] Step 10:

[0095] Device: Sending response results

[0096] The device sends the inputted response results to the server, where they are encrypted and sent securely to the server.

[0097] Step 11:

[0098] Server:Database Update

[0099] The server receives the correspondence results and updates the database, correlating them with existing video data and analysis results, and saving them for future analysis.

[0100] Through these steps, the system will support police officers in responding to crimes on the scene and improve the efficiency of crime prevention and resolution.

[0101] Example 1

[0102] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0103] In traditional police work, police officers are required to provide fast and accurate information so that they can grasp the situation at the scene and respond immediately. However, identifying suspicious individuals and detecting signs of crime at the scene relies heavily on the experience and intuition of police officers, which can lead to delayed responses and incorrect judgments. Furthermore, delayed feedback of information from the scene can affect database updates and subsequent investigative activities. To solve these issues, a system is needed that allows police officers to grasp the situation at the scene in real time and respond appropriately.

[0104] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0105] In this invention, the server includes means for receiving video data from a camera carried by a police officer, means for saving the received video data, means for using an AI module to analyze the saved video data, means for generating an alert based on the analysis results and notifying the police officer, and means for receiving response results from the police officer and updating the database, thereby enabling the police officer to visually grasp the situation at the scene in real time and respond quickly and accurately based on the analysis results of the AI.

[0106] A "police officer" is a public servant who performs duties to maintain law and order and ensure public safety.

[0107] "Photographic equipment" refers to equipment carried by police officers and used to capture video data at the scene, including, for example, cameras and body cams.

[0108] "Video Data" means information in the form of video captured by a camera that records police officers' activities and the surrounding environment.

[0109] "Means for receiving" refers to the technical mechanism by which the server captures the video data transmitted from the imaging device.

[0110] "Storage means" means the technical mechanism used to safely and efficiently store received video data for a certain period of time, including, for example, a database or cloud storage.

[0111] "Analysing means" means technical mechanisms, such as artificial intelligence modules, used to process stored video data and extract specific information.

[0112] An "artificial intelligence module" is a software or hardware component used to analyze or recognize data, including, for example, machine learning algorithms and deep learning models.

[0113] "Means for generating an alert" means the technical mechanism that generates an alert to notify police officers when an abnormality or suspicious activity is detected based on the analysis results.

[0114] "Means for notifying" refers to the technical mechanism for transmitting the generated alert to the police officer's terminal and conveying the information to the police officer.

[0115] "Means for receiving response results" refers to the technical mechanism by which police officers input the results of their response at the scene and import the results into a server.

[0116] "Means for updating the database" refers to the technical mechanisms for amending and adding to the database in order to keep the data in the entire system up to date based on the results of police officers' responses.

[0117] "Suspicious behavior" refers to behavior that deviates from typical patterns of behavior and is considered to be an indication of a potential crime.

[0118] "Anomaly" means a condition or occurrence that is not observed or expected under normal circumstances.

[0119] "Location information" refers to geographical data about a police officer's current location, such as GPS data.

[0120] The present invention relates to a system that analyzes video data in real time in conjunction with a constantly running camera carried by a police officer. This system is composed of the following elements:

[0121] server

[0122] The server includes hardware and software for receiving and storing video data sent from police officers' devices. The received video data is securely stored using cloud storage (e.g., Amazon S3). The server then analyzes the stored video data using an AI module (e.g., TensorFlow or OpenCV). Specifically, the AI ​​module analyzes video frames and recognizes suspicious behavior or specific individuals. The server also generates alerts based on the analysis results and notifies the police officers' devices of these alerts. It also receives response results from police officers and updates a database (e.g., MySQL or PostgreSQL).

[0123] Terminal

[0124] The officer's terminal is responsible for capturing video data in conjunction with the always-on camera. The terminal is equipped with a high-performance camera module (e.g., Logitech C920). It also has a communication module (e.g., 4G / 5G modem) that transmits video data to the server in real time. Received alerts are notified to the officer using a visual display, an audible voice alert, and a vibration function. An interface (e.g., touchscreen, keyboard) is also provided for the officer to input the response results. This data is then sent to the server.

[0125] User (police officer)

[0126] Police officers use this system when patrolling or questioning police officers. By using a constantly running camera and wearing a terminal, video data is sent to a server in real time, and if suspicious behavior or abnormalities are detected, they are immediately notified. Upon receiving the notification, the police officer responds according to the situation. The results are entered into the terminal and sent to and recorded on the server.

[0127] Specific examples

[0128] Example 1: Supporting police questioning

[0129] server:

[0130] A police officer's camera patrolling a busy downtown area at night captures a person. The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals. If the comparison results in a high probability that the person is a wanted criminal, the server immediately generates an alert and notifies the device.

[0131] Device:

[0132] The device receives the alert and displays the message "Possible wanted criminal" on the screen. The device immediately notifies police officers using audio alerts and vibration functions.

[0133] User (police officer):

[0134] Police officers will check the notification and question the person in question. If they are able to confirm the person's identity and find that the person is in fact wanted, the police officers will immediately arrest the person.

[0135] Example 2: Detecting signs of crime

[0136] server:

[0137] A police officer's camera patrolling a park during the day captures a scene in which several young people are gathered and behaving abnormally. The server analyzes the acquired video data using an AI module, and if a suspicious behavior pattern is detected, it determines that there is a "sign of a crime." An alert is generated and sent to the device.

[0138] Device:

[0139] The device receives the alarm notification, displays "Suspicious Activity Detected," and notifies the police. The device immediately issues a visual and audio alert.

[0140] User (police officer):

[0141] Police officers will review the notification, proceed to the scene, approach the group of young people in question, and investigate their behavior. If they find any suspicious items, they will conduct further investigations and take appropriate action on the spot.

[0142] Prompt Sentence Examples

[0143] 1. Sample prompts for police questioning assistance:

[0144] The server receives real-time video data from police officers' cameras, uses an AI module to recognize the faces of wanted criminals, and compares them with a database. If the criminal is determined to be wanted, an alert is generated and sent to the device.

[0145] 2. Example of a prompt for crime detection:

[0146] The AI ​​module analyzes the video data acquired by the server, and if it detects a suspicious pattern of behavior, it determines that it is a "sign of a crime," generates an alert, and notifies the device.

[0147] This system will enable police officers to grasp the situation on the scene in real time and respond quickly and accurately, which is expected to improve the efficiency and reliability of police activities.

[0148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0149] Program processing flow and specific explanation

[0150] Step 1: Capture video data

[0151] Device:

[0152] The police officer carries a device that captures video data in real time through a connected always-on camera, and the device is equipped with a high-performance camera module (e.g., Logitech C920).

[0153] input:

[0154] Successive video frames acquired from a camera.

[0155] output:

[0156] Captured video data.

[0157] Specific behavior:

[0158] As officers walk around on patrol, the device continuously captures video frames and temporarily stores the video data in its internal memory.

[0159] Step 2: Sending video data

[0160] Device:

[0161] The device transmits the captured video data to the server in real time using a 4G / 5G modem.

[0162] input:

[0163] Captured video data.

[0164] output:

[0165] Video data sent to the server.

[0166] Specific behavior:

[0167] The video data is compressed using technology to reduce the amount of data, and then sent to the server via high-speed communication.

[0168] Step 3: Receiving and saving video data

[0169] server:

[0170] The server stores the received video data, which is then securely transferred to cloud storage (e.g., Amazon S3).

[0171] input:

[0172] Video data sent from the device.

[0173] output:

[0174] Securely stored video data.

[0175] Specific behavior:

[0176] The server checks the accuracy and completeness of the data, timestamps it, and stores it in cloud storage.

[0177] Step 4: Analyzing the video data

[0178] server:

[0179] The stored video data is analyzed by an AI module (e.g., TensorFlow or OpenCV) on the server, specifically for person recognition and behavioral pattern analysis.

[0180] input:

[0181] Saved video data.

[0182] output:

[0183] Analysis results (e.g., detection of specific individuals or suspicious behavior).

[0184] Specific behavior:

[0185] AI models process video frames sequentially to detect suspicious behavior or specific individuals, and facial recognition algorithms match them to wanted criminals.

[0186] Step 5: Alert generation and notification

[0187] server:

[0188] Based on the analysis results, an alert is generated as needed, including the detection of wanted criminals and signs of crime, and the generated alert is sent to the device.

[0189] input:

[0190] Analysis results by AI module.

[0191] output:

[0192] Alert notifications sent to your device.

[0193] Specific behavior:

[0194] The server constructs an alert message and sends it to the device via a REST API.

[0195] Step 6: Check the warning and enter the response results

[0196] Device:

[0197] The device notifies the police officer of the alarm received from the server using a visual display, an audible voice alert, and a vibration function.

[0198] input:

[0199] Alert notifications sent from the server.

[0200] output:

[0201] Input of the alert message and response results that will be displayed to the police officer.

[0202] Specific behavior:

[0203] The device displays warning messages such as "Possible wanted fugitive" or "Suspicious behavior detected" and alerts the police officer with beeps and vibrations, who then enters the corresponding results into the device.

[0204] Step 7: Update data

[0205] server:

[0206] The server receives the response results from the police officers and updates the database, which is managed by MySQL or PostgreSQL.

[0207] input:

[0208] Response result data sent from the device.

[0209] output:

[0210] Updated database.

[0211] Specific behavior:

[0212] The entered response result data is sent to the server and immediately saved and updated in the database, allowing the latest status to be always available for reference throughout the system.

[0213] (Application example 1)

[0214] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0215] Modern police activities require rapid and accurate information gathering and situational assessment. However, conventional systems take time to detect suspicious behavior or wanted criminals, making it difficult for police officers to respond quickly on the scene. In addition, there are limited means to visually confirm this information, which places a heavy burden on police officers and can hinder efficient police activities. For these reasons, there is a need for a system that can obtain information in real time and respond quickly and accurately.

[0216] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0217] In this invention, the server includes means for receiving video data from cameras carried by police officers, means for saving the received video data, means for analyzing the saved video data, means for generating an alert based on the analysis results and notifying the police officers, means for receiving response results from the police officers and updating the database, and means for visually and audibly notifying the police officers via the smart glasses. This allows police officers to obtain information in real time at the scene and respond quickly and accurately. The use of the smart glasses allows for visual and audible confirmation of alerts, improving the efficiency and safety of police activities.

[0218] A "camera" is a photographic device carried by a police officer to capture video data at the scene.

[0219] "Video data" is digital data containing visual information captured by a camera.

[0220] The "receiving means" refers to a communication module and software for acquiring the video data transmitted from the camera.

[0221] The "means for storing" refers to data storage and associated software for holding the received video data in a storage device.

[0222] The "analysis means" refers to AI modules and algorithms that analyze stored video data and detect suspicious behavior or anomalies.

[0223] The "means for generating and notifying alerts" refers to a notification system that creates alerts based on the analysis results and notifies police officers.

[0224] The "means for receiving response results and updating the database" refers to a communication module and data processing system for capturing the response status from police officers and reflecting that information in the database.

[0225] "Smart glasses" are augmented reality (AR) devices that are equipped with cameras, displays, and communication functions and are worn by police officers.

[0226] "Visual and audible notification means" refers to the ability to display or audibly communicate warning information through the smart glasses' display or speaker.

[0227] This invention relates to a security system that works in conjunction with cameras carried by police officers, analyzes video data in real time, and issues an immediate alarm. This system is composed of the following elements to enable rapid information gathering and response in police activities:

[0228] System Configuration

[0229] 1. Server

[0230] The server includes hardware and software that receives and stores video data sent from police officers' cameras. It also contains an AI module (using TensorFlow) that analyzes the video data to detect suspicious behavior or individuals.

[0231] It generates alerts based on the analysis results and sends them to police officers' devices or smart glasses in real time. It also receives response results from police officers and updates the database.

[0232] 2. Terminal (smart glasses)

[0233] The smart glasses are equipped with a camera that captures video data in real time, which is then transmitted to a server using WebSocket technology.

[0234] The smart glasses are equipped with an AR display that visually and audibly notifies police officers of warnings based on the analysis results. Police officers can then grasp the situation at the scene through the smart glasses and take necessary action.

[0235] 3. User (police officer)

[0236] Police officers wear smart glasses and patrol while capturing video data. When an alarm is notified, they quickly check the situation at the scene and report the results of their response to the server from the smart glasses.

[0237] Program processing and data calculation

[0238] 1. Camera video capture

[0239] The camera installed in the smart glasses captures video data and transmits it to a server in real time.

[0240] 2. Analysis of video data

[0241] The server analyzes the received video data using an AI module powered by TensorFlow, which detects suspicious behavior and people with high accuracy.

[0242] 3. Alert Generation and Notification

[0243] Based on the analysis results, the server generates an alert and notifies the police officers through the smart glasses' AR display and speaker, allowing them to understand the situation in real time.

[0244] 4. Reporting the results of the response

[0245] Police officers send the results of their response from the smart glasses to a server, which then updates the database with the information.

[0246] Specific examples

[0247] For example, while a police officer is patrolling a park, a camera mounted on a pair of smart glasses captures footage of a group of young people. The video data is sent to the server, where it is analyzed by an AI module and any suspicious behavior is detected. The server immediately generates an alert and notifies the smart glasses that "suspicious behavior has been detected." Police officers then use this information to investigate the scene and take any necessary action.

[0248] Prompt Sentence Examples

[0249] "In the security field, I am developing a system that analyzes real-time video data captured by smart glasses to detect suspicious behavior and wanted criminals. In this system, the server analyzes the video data, generates an alarm, and displays it on the smart glasses. Specifically, could you please tell me what kind of code I should implement?"

[0250] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0251] Step 1:

[0252] The camera in the smart glasses captures the surrounding video data. The input is live video of the scene, and the output is digitized video data. The camera captures video in real time and transmits the data to the built-in communication module.

[0253] Step 2:

[0254] The communication module of the smart glasses transmits the captured video data to the server. The input is the captured video data, and the output is the data to be sent to the server. WebSocket technology is used to transfer the data to the server with low latency.

[0255] Step 3:

[0256] The server stores the received video data. The input is the video data sent from the smart glasses, and the output is the data stored in the storage. The server records the data in a database and keeps it for subsequent analysis.

[0257] Step 4:

[0258] The server analyzes the stored video data. The input is the stored video data, and the output is the analysis results. An AI module (using TensorFlow) extracts and analyzes suspicious behavior and people from the video data.

[0259] Step 5:

[0260] The server generates an alert based on the analysis results. The input is the analysis results and the output is the alert data. If suspicious behavior or people are detected, an alert is immediately created and its contents are defined.

[0261] Step 6:

[0262] The server generates an alarm and notifies the smart glasses. The input is the alarm data, and the output is a visual and audible alarm notification. The alarm is notified to police officers using the AR display and speaker of the smart glasses.

[0263] Step 7:

[0264] The user (police officer) checks the alarm and takes appropriate action at the scene. The input is the alarm notification from the smart glasses, and the output is the response result at the scene. The police officer checks the notification and takes the necessary action on the spot (e.g., questioning, checking the scene).

[0265] Step 8:

[0266] The user (police officer) reports the response result to the server from the smart glasses. The input is the response result, and the output is the response data sent to the server. The result is entered through the input UI of the smart glasses and sent to the server.

[0267] Step 9:

[0268] The server reflects the received response results in the database. The input is the reported response results, and the output is the updated database. The server records the response results in the database, making it easier to understand the overall situation.

[0269] The above are the specific processing steps of this system.

[0270] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0271] This invention relates to a system that combines a system that analyzes video data in real time in conjunction with a constantly running camera carried by police officers, with an emotion engine that recognizes the user's emotions. This system is composed of the following elements:

[0272] 1. Server

[0273] The server includes hardware and software for receiving and storing video data sent from police officers' devices. It also includes an AI module and emotion engine for analyzing the stored video data. The server generates alerts based on the analysis results and sends them to the police officers' devices. It also receives response results from police officers and updates the database.

[0274] 2. Terminal

[0275] The officer's terminal communicates with the camera and captures video data. It is equipped with a communication module that transmits video data to the server in real time. It also notifies officers of received alarms and issues warnings through visual, audible, and vibration alerts. It also provides an interface for officers to input the results of their response and transmits the data to the server.

[0276] 3. User (police officer)

[0277] Police officers use this system when patrolling and questioning police officers. By wearing a camera and a device, video data is sent to a server in real time, and if suspicious behavior or abnormalities are detected, they are immediately notified. An emotion engine is also added to recognize the user's emotions, allowing for more accurate responses according to the situation. After confirming the notification, the police officer takes action according to the situation and updates the database by entering the results into the device.

[0278] As a specific example of the operation, the following scenario can be considered.

[0279] Example 1: Police interrogation assistance and emotion recognition

[0280] server:

[0281] A police officer's camera captures a person while patrolling a busy street at night.

[0282] The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals.

[0283] As a result of the match, it is determined that there is a high probability that the person is a wanted criminal, and the emotion engine analyzes the person's emotions and detects signs of anxiety or fear.

[0284] The server immediately generates an alert and notifies the terminal.

[0285] Device:

[0286] The device receives the alert and displays the message "Possible wanted criminal" along with the emotion analysis results on the screen. It indicates strong emotions of "anxiety" and "fear." The device immediately notifies police officers using a voice alert and vibration function.

[0287] User (police officer):

[0288] Police officers will check this notification and question the person in question. They will proceed with questioning carefully, taking into consideration the results of the emotion analysis. If the person's identity is confirmed and they are in fact wanted, the police officer will immediately arrest the person.

[0289] Example 2: Detecting signs of crime and emotional changes

[0290] server:

[0291] A police officer's camera patrolling a park during the day captures a scene of several young people gathered together and behaving abnormally.

[0292] The server uses an AI module to analyze the acquired video data and detect suspicious behavioral patterns. The emotion engine also analyzes the emotions of young people and detects signs of "excitement" or "agitation." An alert is generated and sent to the device.

[0293] Device:

[0294] The device receives an alarm notification, displays "Suspicious Activity Detected," and also displays the young people's emotional states of "Excitement" and "Arousal." The device immediately issues visual and audio alerts.

[0295] User (police officer):

[0296] Police officers can view the notification, proceed to the scene, approach the group of youths, and observe their behavior. They can also monitor emotional changes and prepare in advance. According to an embodiment of the present invention, police officers can respond not only to suspicious behavior but also to emotional changes in the subjects, allowing them to perform their duties more safely and effectively.

[0297] The processing flow will be explained below.

[0298] Step 1:

[0299] Device: Camera activation and video capture

[0300] The device activates the camera worn by the officer and begins capturing video data. The camera continues to capture video at 30 frames per second and transmits it to the device.

[0301] Step 2:

[0302] Terminal: Compression and transmission of video data

[0303] The device compresses and encrypts the video data acquired from the camera in real time, and then sends the compressed and encrypted video data to the server.

[0304] Step 3:

[0305] Server: Receiving and temporarily storing video data

[0306] The server receives the video data sent from the device and temporarily stores it, adding GPS information and a timestamp to the received video data.

[0307] Step 4:

[0308] Server: AI video analysis

[0309] An AI module on the server analyzes the temporarily stored video data in real time, using facial recognition and motion analysis algorithms to identify suspicious behavior and wanted criminals.

[0310] Step 5:

[0311] Server: Emotion analysis using emotion engine

[0312] In parallel with the AI ​​module's analysis, the server's emotion engine analyzes the subject's facial expressions from the video data and identifies emotions, detecting emotions such as "anxiety," "fear," "elation," and "excitement" in real time.

[0313] Step 6:

[0314] Server: Generate alerts

[0315] Based on the analysis results (AI behavior analysis and emotion analysis by the emotion engine), the server generates an alert if necessary. The alert includes the analysis results, location information, timestamp, and emotion analysis results.

[0316] Step 7:

[0317] Server: Alert notification

[0318] The server sends the generated alert to the terminal, which receives the alert and prepares to notify the police officer.

[0319] Step 8:

[0320] Terminal: Display of alert notification

[0321] The device notifies officers of received alarms visually and audibly. Detailed information about the alarm and the results of emotion analysis are displayed on the device screen, and the device alerts officers with a warning sound and vibration.

[0322] Step 9:

[0323] User (police officer): Response to alarm

[0324] Police officers check the alarms on the device and take action according to the situation at the scene, such as questioning the suspect, investigating the scene, and responding to emergencies if necessary. Based on the results of emotion analysis, the response policy can be adjusted more appropriately.

[0325] Step 10:

[0326] Device: Enter response results

[0327] Police officers use the terminal interface to input the results of their response, including the details of the police stop, any suspicious objects found, and information about the person they have apprehended.

[0328] Step 11:

[0329] Device: Sending response results

[0330] The device sends the inputted response results to the server, where they are encrypted and sent securely to the server.

[0331] Step 12:

[0332] Server:Database Update

[0333] The server receives the correspondence results and updates the database, correlating them with existing video data and analysis results, and saving them for future analysis.

[0334] Through these steps, this system will support police officers in responding to crime scenes and improve the efficiency of crime prevention and resolution. The addition of an emotion engine will enable responses that take into account the emotional changes of the target, further improving reliability.

[0335] Example 2

[0336] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0337] When police officers are on patrol or conducting routine questioning, it is extremely important to quickly and accurately grasp the suspicious behavior and emotional state of the subject. However, with conventional systems, it is difficult to analyze video data in real time and determine emotions, making it difficult for police officers to respond appropriately immediately. In addition, there are limited methods for issuing alarm notifications, making it difficult for police officers to respond immediately.

[0338] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0339] In this invention, the server includes means for receiving video data from a camera carried by a police officer, means for saving the received video data, means for analyzing the saved video data, means for generating an alert based on the analysis results and notifying the police officer, means for receiving response results from the police officer and updating the database, means for analyzing the emotional state from the video data and generating an alert based on the analysis results, and means for notifying the police officer of the alert visually, audibly, and by vibration. This makes it possible to detect suspicious behavior and emotional states in real time and notify the police officer quickly and reliably.

[0340] A "camera carried by a police officer" is a small, continuously operational video camera that is worn on the body by a police officer when patrolling or conducting routine questioning.

[0341] The "means for receiving video data" is a set of hardware and software for receiving real-time video data transmitted from a camera.

[0342] The "means for storing received video data" is a data storage system for safely and efficiently storing received video data.

[0343] "Means for analyzing stored video data" refers to AI modules and analytical software for performing facial recognition and behavioral analysis on stored video data.

[0344] The "means for generating an alert based on the analysis results and notifying police officers" is a system for creating an alert based on the analyzed data and notifying the police officers' terminals.

[0345] The "means for receiving response results from police officers and updating the database" is a system for receiving information entered by police officers after responding to an incident at the scene and keeping the database up to date.

[0346] The "means for analyzing the emotional state from video data and generating an alert based on the analysis results" refers to an emotion recognition engine that analyzes the emotional state of people in the video and generates an appropriate alert based on the results.

[0347] "Means for notifying police officers of an alarm visually, audibly, and by vibration" means a notification system that uses a screen display, audio alerts, and the device's vibration function to quickly communicate an alarm to police officers.

[0348] "Means for analyzing in real time and detecting suspicious behavior and anomalies" refers to an AI-based analysis module that processes video data in real time and instantly detects abnormal or suspicious behavior.

[0349] "Means for obtaining location information of police officers on patrol and synchronizing it with video data" refers to a system for obtaining the current location of police officers using a location information system such as GPS and synchronizing it with video data.

[0350] This invention is a system that uses a constantly running camera and a terminal carried by a police officer to transmit video data to a server in real time and analyzes the video data. This system is equipped with an AI module and an emotion engine, and can detect suspicious behavior and emotional states.

[0351] Hardware and software used

[0352] 1. Camera: A high-resolution video camera (with infrared functionality) is used, which allows for clear image data to be acquired.

[0353] 2. Terminal: A mobile terminal equipped with an LTE or Wi-Fi communication module that transmits video data to a server in real time.

[0354] 3. Server: A high-performance server equipped with cloud storage (e.g., Amazon S3) and AI modules (e.g., OpenCV, TensorFlow). It also uses an emotion analysis engine (e.g., IBM Watson) to analyze the emotional state from video data.

[0355] Program processing

[0356] server

[0357] The server receives video data transmitted in real time from the device and stores it in data storage. It uses an AI module to analyze the stored data and compare it with a database of wanted criminals. It also uses an emotion engine to analyze the emotional state of people in the video and generates an alert based on the analysis results. The alert is transmitted via a system that notifies police officers with visual and audio alerts and vibrations.

[0358] Terminal

[0359] The device captures video data from the always-on camera and transmits it to the server in real time. It displays received alarms visually and notifies officers with audio alerts and vibrations. Officers enter their response results into the device, which then transmits the data to the server, updating the database.

[0360] User (police officer)

[0361] Police officers wear cameras and devices when patrolling and questioning people. They respond based on notifications from the system and enter the results of their actions into the device. This keeps the database updated with the latest information, enabling them to carry out their duties safely and effectively.

[0362] Specific examples

[0363] Example 1: Police interrogation assistance and emotion recognition

[0364] Server: A police officer patrolling a busy downtown area at night captures a person on camera. The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals. As a result of the comparison, it determines that there is a high probability that the person is a wanted criminal, and an emotion engine analyzes the person's emotions and detects signs of anxiety or fear. The server immediately generates an alert stating "possibly wanted criminal" or "anxiety detected" and notifies the device.

[0365] Device: The device receives the alarm notification and displays "Possible wanted criminal" and "Anxiety detected." It immediately notifies the police with an audio alert and vibration.

[0366] User (police officer): The police officer checks the notification and begins to carefully question the suspect. The police officer enters the results into the terminal and sends them to the server to update the database.

[0367] Prompt Sentence Examples

[0368] 1. "While patrolling downtown, the camera captured a person who appears to be a wanted criminal. Please run the AI ​​module to match the person and simultaneously perform sentiment analysis."

[0369] 2. "When conducting a police stop, please demonstrate how to respond if the person's emotions indicate anxiety or fear."

[0370] Example 2: Detecting signs of crime and emotional changes

[0371] Server: Captures a scene of several young people gathering in a park during the day and behaving abnormally. The server analyzes the video data and detects suspicious behavior. The emotion engine also analyzes signs of "excitement" or "fever." The server generates an alert for "suspicious behavior detected" or "excitement detected" and notifies the device.

[0372] Terminal: The terminal receives the alarm notification and displays "Suspicious behavior detected" or "Elevation detected." It immediately notifies the police officer with visual and audio alerts.

[0373] User (police officer): The police officer checks the notification, approaches the group of young people in question, and observes their behavior. He or she takes appropriate action depending on the situation, paying attention to any emotional changes. The results are entered into the terminal and sent to the server to update the database.

[0374] Prompt Sentence Examples

[0375] 1. "Analyze footage of several young people behaving suspiciously in a park and use the emotion engine to detect their emotional state."

[0376] 2. "Please provide procedures for police officers to safely respond to situations involving a group of adolescents who are agitated."

[0377] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0378] Program processing flow

[0379] Step 1: Capture video data

[0380] Subject: Terminal

[0381] The terminal communicates with a constantly running camera carried by the police officer and captures video data in real time. The camera records video in high resolution and transmits the video data to the terminal. Specifically, the camera is attached to the officer's breast pocket and constantly records video of the surrounding area. The input data is the video of the surrounding area, and the output is the captured video data.

[0382] Step 2: Sending video data

[0383] Subject: Terminal

[0384] The device transmits the captured video data to the server in real time. The video data is compressed before transmission to efficiently transfer it to the server, for example, using H.264 encoding to reduce the data volume. The input data is the captured video data, and the output is compressed video data.

[0385] Step 3: Receiving and saving video data

[0386] Subject: Server

[0387] The server receives the video data sent from the device and stores it in cloud storage. The storage used is a highly reliable cloud service (e.g., Amazon S3). The input data is compressed video data, and the output is stored video data.

[0388] Step 4: Analyzing the video data

[0389] Subject: Server

[0390] The server analyzes the stored video data using an AI module. Specifically, it uses OpenCV and TensorFlow to perform facial recognition and motion analysis. For example, it extracts the faces of people in the video and compares them with a database of wanted criminals. The input data is the stored video data, and the output is the analysis results (e.g., facial recognition results, motion analysis results).

[0391] Step 5: Sentiment analysis and alert generation

[0392] Subject: Server

[0393] The server uses an emotion engine to perform emotion analysis based on the analyzed data. For example, it uses IBM Watson's emotion analysis API to analyze emotions such as "anxiety," "fear," and "elation" from the facial expressions of people in the video. Based on the analysis results, it generates an appropriate alert. The input data is the analysis result, and the output is the generated alert.

[0394] Step 6: Alert notification and response

[0395] Subject: Terminal

[0396] The device receives the alert sent from the server and notifies the police officer. It displays the alert content in a visual interface and issues an immediate warning using audio alerts and the device's vibration function. For example, it displays messages such as "Possible wanted fugitive" or "Anxiety detected." The input data is the generated alert, and the output is the alert content notified to the police officer.

[0397] Step 7: Sending response results and updating the database

[0398] Subject: User (police officer)

[0399] After confirming the notification, the police officer responds at the scene. The police officer inputs the response results (e.g., the results of the police questioning or a situation report) into the terminal and sends the data to the server. The server receives the response results and updates the database. The input data is the response results at the scene, and the output is the updated database information.

[0400] (Application example 2)

[0401] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0402] Conventional security systems simply store camera footage, making it difficult to detect suspicious behavior or emotional changes in real time. As a result, there is a lack of information to make appropriate decisions in situations where a quick response is required. As a result, it is difficult to implement effective security measures to prevent danger.

[0403] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0404] In this invention, the server includes means for receiving video data from cameras carried by security guards, means for saving the received video data, means for analyzing the saved video data, and means including an emotion engine for recognizing the emotions of the target based on the analyzed video data. This makes it possible to detect not only suspicious behavior and abnormalities in real time, but also changes in the target's emotions, allowing for faster and more accurate responses.

[0405] A "security guard" is a person whose job is to monitor facilities and locations, detect abnormalities and suspicious behavior, and respond accordingly.

[0406] A "camera" is a photographing device that captures video data and transmits it to a server in real time.

[0407] "Video data" refers to information about images and videos captured by a camera.

[0408] The "receiving means" is a mechanism by which the server receives the video data transmitted from the camera.

[0409] The "storage means" is a mechanism for temporarily or permanently storing received video data in a server.

[0410] The "analysis means" is a mechanism equipped with an AI module that analyzes stored video data and detects suspicious behavior or abnormalities.

[0411] An "alarm" is an alert generated based on the analysis results and notified to security personnel visually, audibly, or in other ways.

[0412] The "notification means" is a mechanism for transmitting the generated alarm to a security guard.

[0413] "Response results" is information that security guards report on the results of actions they take based on an alarm.

[0414] A "database" is a structured collection of information that stores and makes available past response results and other related information.

[0415] The "emotion engine" is a system with an algorithm for recognizing a subject's emotional state (e.g., anxiety, elation, fear, etc.) based on video data.

[0416] "Location information" refers to information about the current location of security guards obtained using technology such as GPS.

[0417] MODE FOR CARRYING OUT THE INVENTION

[0418] System program for implementing this application example

[0419] This system is realized through the cooperation of smart glasses carried by security guards, a server, and a database. The smart glasses are equipped with a camera that is always on and captures video data in real time. The captured video data is sent to the server via a communications module. The server stores the received video data and analyzes it using an AI module and emotion engine. Based on the analysis results, an alarm is generated and notified to the security guard. The system also receives the guard's response results and updates the database.

[0420] Explain the generated program's processing in natural language

[0421] Server Processing

[0422] 1. Hardware and Software:

[0423] The server is equipped with hardware with high-performance processing capabilities, as well as an AI module and emotion engine for analyzing video data.

[0424] The AI ​​module includes image analysis libraries (e.g., OpenCV), and the emotion engine includes emotion recognition models (e.g., IBM Watson Emotion API).

[0425] The server uses a communication module to receive and automatically store video data sent from the smart glasses.

[0426] 2. Data Analysis:

[0427] The received video data is analyzed in real time to detect suspicious behavior and abnormalities.

[0428] The emotion engine recognizes the subject's emotions and analyzes their emotional state (e.g., anxiety, elation, fear, etc.).

[0429] 3. Generate an alert:

[0430] Generate alerts based on the analysis results.

[0431] The alert will include details such as the type of activity detected and the emotional state.

[0432] 4. Notifications and Database Updates:

[0433] The generated alert is sent to the smart glasses terminal via visual, auditory, and vibration notifications.

[0434] The results of the security guard's response are received and the database is updated.

[0435] Smart Glasses Processing

[0436] 1. Video Capture:

[0437] A camera mounted on the smart glasses captures video data in real time.

[0438] 2. Data transmission:

[0439] The video data is transmitted to the server via the communication module.

[0440] 3. Alarm notification:

[0441] Receives alarm notifications from the server and notifies security guards.

[0442] Notification methods include visual (display on the screen), audible (voice alert), and vibration.

[0443] Database Processing

[0444] 1. Data storage and updating:

[0445] The video data received from the server, analysis results, and security guard response results are stored.

[0446] Add specific examples to the description

[0447] Examples:

[0448] Example 1:

[0449] "While a security guard is patrolling a building in the middle of the night, the smart glasses capture a suspicious person. The server analyzes the video and determines that the person is trying to break into a locked room. The emotion engine detects the person's expression of impatience and immediately issues an alarm."

[0450] Example 2:

[0451] "Smart glasses capture a person behaving suspiciously in the crowd at a train station. The server then analyzes the person and, through profiling, identifies that the person has behaved similarly in the past, and detects that they have a particularly dangerous tendency. The emotion engine then detects signs of tension in the person's facial expressions and sends an immediate notification."

[0452] Example prompt for a generative AI model:

[0453] "Security guards wearing smart glasses capture video in real time and send it to a server. The server then uses an AI module to analyze the video data, and if it detects any suspicious behavior, it notifies the security guard in real time. It also detects emotions and helps the security guard respond appropriately based on the emotional state of the person in question."

[0454] The above is the specific content of the "Mode for Carrying Out the Invention." This demonstrates that the present invention is a system that can identify suspicious behavior and emotional changes in real time, enabling rapid and accurate security responses.

[0455] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0456] Step 1:

[0457] Video Capture

[0458] Description: The camera installed in the smart glasses that serve as the terminal captures video data in real time.

[0459] Input: Video of the guard's surroundings.

[0460] What it does: The camera continuously captures video frames and stores them in a buffer.

[0461] Output: Captured video data.

[0462] Step 2:

[0463] Data transmission

[0464] Description: The device sends captured video data to the server.

[0465] Input: Captured video data.

[0466] Specific operation: Using a communications module, video data is sent to a server via the Internet, often using the HTTP protocol.

[0467] Output: Video data sent to the server.

[0468] Step 3:

[0469] Data reception and storage

[0470] Description: The server receives and stores video data sent from the device.

[0471] Input: Video data sent from the device.

[0472] Specific operation: The communication module receives the data and stores it in local storage or a database.

[0473] Output: Saved video data.

[0474] Step 4:

[0475] Data analysis

[0476] Description: The server analyzes the stored video data to detect suspicious behavior and anomalies. It also analyzes the emotional state using an emotion engine.

[0477] Input: Stored video data.

[0478] How it works: An AI module (e.g., OpenCV) is used to analyze behaviors and facial expressions in the video data. An emotion engine (e.g., IBM Watson Emotion API) evaluates the subject's emotional state.

[0479] Output: Behavioral analysis results and emotion analysis results.

[0480] Step 5:

[0481] Generate an alert

[0482] Description: The server generates an alert based on the analysis results.

[0483] Input: Behavioral analysis results and emotion analysis results.

[0484] Specific behavior: Evaluate the analysis results and generate an alert if certain conditions are met (e.g., detecting suspicious behavior or a specific emotional state). The alert can be generated in multiple ways, including visual, auditory, or vibration.

[0485] Output: Generated alert data.

[0486] Step 6:

[0487] Alert notification

[0488] Description: Notifies the terminal of the alarm generated by the server. The alarm is delivered to the security guard by visual, auditory and vibration means.

[0489] Input: Generated alert data.

[0490] Specific operation: The communication module sends alarm data to the terminal, which displays the received alarm on the screen and alerts the security guard with an alarm sound or vibration.

[0491] Output: Alarm notified to security personnel.

[0492] Step 7:

[0493] Enter and submit the response results

[0494] Description: The user (security guard) responds based on the alarm and inputs the response results into the terminal. The terminal then sends the response results to the server.

[0495] Input: The action result entered by the user.

[0496] Specific operation: The security guard uses the terminal interface to input the response results and sends the data to the server using the communication module.

[0497] Output: The response result sent to the server.

[0498] Step 8:

[0499] Database Update

[0500] Description: The server receives the response and updates the database.

[0501] Input: The response result sent.

[0502] Specific operation: The server adds or updates the response result data it receives to the database and stores it as history, thereby accumulating data that can be used for future analysis.

[0503] Output: The updated database.

[0504] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0505] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0506] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0507] [Second embodiment]

[0508] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0509] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0510] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0511] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0512] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0513] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0514] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0515] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0516] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0517] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0518] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0519] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0520] The present invention relates to a system that analyzes video data in real time in conjunction with a constantly running camera carried by a police officer. This system is composed of the following elements:

[0521] 1. Server

[0522] The server includes hardware and software for receiving and storing video data sent from police officers' terminals. It also includes an AI module that analyzes the stored video data. The server generates alerts based on the analysis results and sends them to the police officers' terminals. It also receives response results from police officers and updates the database.

[0523] 2. Terminal

[0524] The officer's terminal communicates with the camera and captures video data. It is equipped with a communication module that transmits video data to the server in real time. It also notifies officers of received alarms and issues warnings through visual, audible, and vibration alerts. It also provides an interface for officers to input the results of their response and transmits the data to the server.

[0525] 3. User (police officer)

[0526] Police officers use this system when patrolling or questioning police officers. By wearing cameras and devices, video data is sent to a server in real time, and if suspicious behavior or abnormalities are detected, they are immediately notified. Upon receiving the notification, the police officer takes appropriate action according to the situation and updates the database by entering the results into the device.

[0527] As a specific example of the operation, the following scenario can be considered.

[0528] Example 1: Supporting police questioning

[0529] server:

[0530] A police officer's camera captures a person while patrolling a busy street at night.

[0531] The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals.

[0532] If the result of the comparison indicates that there is a high probability that the person is a wanted criminal, the server immediately generates an alert and notifies the terminal.

[0533] Device:

[0534] The device receives the alert and displays the message "Possible wanted criminal" on the screen. The device immediately notifies police officers using audio alerts and vibration functions.

[0535] User (police officer):

[0536] Police officers will check this notification and question the person in question. If they are able to confirm the person's identity and find that the person is in fact wanted, the police officers will immediately arrest the person.

[0537] Example 2: Detecting signs of crime

[0538] server:

[0539] A police officer's camera patrolling a park during the day captures a scene of several young people gathered together and behaving abnormally.

[0540] The server analyzes the acquired video data using an AI module, and if a suspicious behavior pattern is detected, it determines that there is a "sign of a crime," generates an alert, and notifies the device.

[0541] Device:

[0542] The device receives the alarm notification, displays "Suspicious Activity Detected," and notifies the police. The device immediately issues a visual and audio alert.

[0543] User (police officer):

[0544] Police officers will review the notification, proceed to the scene, approach the group of young people in question, and investigate their behavior. If they find any suspicious items, they will conduct further investigations and take necessary action on the spot.

[0545] The above specific example is intended to enable immediate response at the scene of a crime and to prevent and resolve it quickly. This system is expected to improve the accuracy of police officers' judgment, and to increase the efficiency and reliability of police activities.

[0546] The processing flow will be explained below.

[0547] Step 1:

[0548] Device: Camera activation and video capture

[0549] The device activates the camera worn by the officer and begins capturing video data. The camera continues to capture video at 30 frames per second and transmits it to the device.

[0550] Step 2:

[0551] Terminal: Compression and transmission of video data

[0552] The device compresses and encrypts the video data acquired from the camera in real time, and then sends the compressed and encrypted video data to the server.

[0553] Step 3:

[0554] Server: Receiving and temporarily storing video data

[0555] The server receives the video data sent from the device and temporarily stores it, adding GPS information and a timestamp to the received video data.

[0556] Step 4:

[0557] Server: AI video analysis

[0558] An AI module on the server analyzes the temporarily stored video data in real time, using facial recognition and motion analysis algorithms to identify suspicious behavior and wanted criminals.

[0559] Step 5:

[0560] Server: Generate alerts

[0561] Based on the analysis results of the AI ​​module, the server generates an alert if necessary, which includes the analysis results, location information, and a timestamp.

[0562] Step 6:

[0563] Server: Alert notification

[0564] The server sends the generated alert to the terminal, which receives the alert and prepares to notify the police officer.

[0565] Step 7:

[0566] Terminal: Display of alert notification

[0567] The device notifies officers of received alarms visually and audibly, displays detailed information on the alarm on the device's screen, and alerts officers with a warning sound and vibration.

[0568] Step 8:

[0569] User (police officer): Response to alarm

[0570] Police officers will check the alarm on the device and take action according to the situation at the scene, such as questioning the suspect, checking the scene, and in some cases, responding to an emergency.

[0571] Step 9:

[0572] Device: Enter response results

[0573] Police officers use the terminal interface to input the results of their response, including the details of the police stop, any suspicious objects found, and information about the person they have apprehended.

[0574] Step 10:

[0575] Device: Sending response results

[0576] The device sends the inputted response results to the server, where they are encrypted and sent securely to the server.

[0577] Step 11:

[0578] Server:Database Update

[0579] The server receives the correspondence results and updates the database, correlating them with existing video data and analysis results, and saving them for future analysis.

[0580] Through these steps, the system will support police officers in responding to crimes on the scene and improve the efficiency of crime prevention and resolution.

[0581] Example 1

[0582] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0583] In traditional police work, police officers are required to provide fast and accurate information so that they can grasp the situation at the scene and respond immediately. However, identifying suspicious individuals and detecting signs of crime at the scene relies heavily on the experience and intuition of police officers, which can lead to delayed responses and incorrect judgments. Furthermore, delayed feedback of information from the scene can affect database updates and subsequent investigative activities. To solve these issues, a system is needed that allows police officers to grasp the situation at the scene in real time and respond appropriately.

[0584] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0585] In this invention, the server includes means for receiving video data from a camera carried by a police officer, means for saving the received video data, means for using an AI module to analyze the saved video data, means for generating an alert based on the analysis results and notifying the police officer, and means for receiving response results from the police officer and updating the database, thereby enabling the police officer to visually grasp the situation at the scene in real time and respond quickly and accurately based on the analysis results of the AI.

[0586] A "police officer" is a public servant who performs duties to maintain law and order and ensure public safety.

[0587] "Photographic equipment" refers to equipment carried by police officers and used to capture video data at the scene, including, for example, cameras and body cams.

[0588] "Video Data" means information in the form of video captured by a camera that records police officers' activities and the surrounding environment.

[0589] "Means for receiving" refers to the technical mechanism by which the server captures the video data transmitted from the imaging device.

[0590] "Storage means" means the technical mechanism used to safely and efficiently store received video data for a certain period of time, including, for example, a database or cloud storage.

[0591] "Analysing means" means technical mechanisms, such as artificial intelligence modules, used to process stored video data and extract specific information.

[0592] An "artificial intelligence module" is a software or hardware component used to analyze or recognize data, including, for example, machine learning algorithms and deep learning models.

[0593] "Means for generating an alert" means the technical mechanism that generates an alert to notify police officers when an abnormality or suspicious activity is detected based on the analysis results.

[0594] "Means for notifying" refers to the technical mechanism for transmitting the generated alert to the police officer's terminal and conveying the information to the police officer.

[0595] "Means for receiving response results" refers to the technical mechanism by which police officers input the results of their response at the scene and import the results into a server.

[0596] "Means for updating the database" refers to the technical mechanisms for amending and adding to the database in order to keep the data in the entire system up to date based on the results of police officers' responses.

[0597] "Suspicious behavior" refers to behavior that deviates from typical patterns of behavior and is considered to be an indication of a potential crime.

[0598] "Anomaly" means a condition or occurrence that is not observed or expected under normal circumstances.

[0599] "Location information" refers to geographical data about a police officer's current location, such as GPS data.

[0600] The present invention relates to a system that analyzes video data in real time in conjunction with a constantly running camera carried by a police officer. This system is composed of the following elements:

[0601] server

[0602] The server includes hardware and software for receiving and storing video data sent from police officers' devices. The received video data is securely stored using cloud storage (e.g., Amazon S3). The server then analyzes the stored video data using an AI module (e.g., TensorFlow or OpenCV). Specifically, the AI ​​module analyzes video frames and recognizes suspicious behavior or specific individuals. The server also generates alerts based on the analysis results and notifies the police officers' devices of these alerts. It also receives response results from police officers and updates a database (e.g., MySQL or PostgreSQL).

[0603] Terminal

[0604] The officer's terminal is responsible for capturing video data in conjunction with the always-on camera. The terminal is equipped with a high-performance camera module (e.g., Logitech C920). It also has a communication module (e.g., 4G / 5G modem) that transmits video data to the server in real time. Received alerts are notified to the officer using a visual display, an audible voice alert, and a vibration function. An interface (e.g., touchscreen, keyboard) is also provided for the officer to input the response results. This data is then sent to the server.

[0605] User (police officer)

[0606] Police officers use this system when patrolling or questioning police officers. By using a constantly running camera and wearing a terminal, video data is sent to a server in real time, and if suspicious behavior or abnormalities are detected, they are immediately notified. Upon receiving the notification, the police officer responds according to the situation. The results are entered into the terminal and sent to and recorded on the server.

[0607] Specific examples

[0608] Example 1: Supporting police questioning

[0609] server:

[0610] A police officer's camera patrolling a busy downtown area at night captures a person. The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals. If the comparison results in a high probability that the person is a wanted criminal, the server immediately generates an alert and notifies the device.

[0611] Device:

[0612] The device receives the alert and displays the message "Possible wanted criminal" on the screen. The device immediately notifies police officers using audio alerts and vibration functions.

[0613] User (police officer):

[0614] Police officers will check the notification and question the person in question. If they are able to confirm the person's identity and find that the person is in fact wanted, the police officers will immediately arrest the person.

[0615] Example 2: Detecting signs of crime

[0616] server:

[0617] A police officer's camera patrolling a park during the day captures a scene in which several young people are gathered and behaving abnormally. The server analyzes the acquired video data using an AI module, and if a suspicious behavior pattern is detected, it determines that there is a "sign of a crime." An alert is generated and sent to the device.

[0618] Device:

[0619] The device receives the alarm notification, displays "Suspicious Activity Detected," and notifies the police. The device immediately issues a visual and audio alert.

[0620] User (police officer):

[0621] Police officers will review the notification, proceed to the scene, approach the group of young people in question, and investigate their behavior. If they find any suspicious items, they will conduct further investigations and take appropriate action on the spot.

[0622] Prompt Sentence Examples

[0623] 1. Sample prompts for police questioning assistance:

[0624] The server receives real-time video data from police officers' cameras, uses an AI module to recognize the faces of wanted criminals, and compares them with a database. If the criminal is determined to be wanted, an alert is generated and sent to the device.

[0625] 2. Example of a prompt for crime detection:

[0626] The AI ​​module analyzes the video data acquired by the server, and if it detects a suspicious pattern of behavior, it determines that it is a "sign of a crime," generates an alert, and notifies the device.

[0627] This system will enable police officers to grasp the situation on the scene in real time and respond quickly and accurately, which is expected to improve the efficiency and reliability of police activities.

[0628] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0629] Program processing flow and specific explanation

[0630] Step 1: Capture video data

[0631] Device:

[0632] The police officer carries a device that captures video data in real time through a connected always-on camera, and the device is equipped with a high-performance camera module (e.g., Logitech C920).

[0633] input:

[0634] Successive video frames acquired from a camera.

[0635] output:

[0636] Captured video data.

[0637] Specific behavior:

[0638] As officers walk around on patrol, the device continuously captures video frames and temporarily stores the video data in its internal memory.

[0639] Step 2: Sending video data

[0640] Device:

[0641] The device transmits the captured video data to the server in real time using a 4G / 5G modem.

[0642] input:

[0643] Captured video data.

[0644] output:

[0645] Video data sent to the server.

[0646] Specific behavior:

[0647] The video data is compressed using technology to reduce the amount of data, and then sent to the server via high-speed communication.

[0648] Step 3: Receiving and saving video data

[0649] server:

[0650] The server stores the received video data, which is then securely transferred to cloud storage (e.g., Amazon S3).

[0651] input:

[0652] Video data sent from the device.

[0653] output:

[0654] Securely stored video data.

[0655] Specific behavior:

[0656] The server checks the accuracy and completeness of the data, timestamps it, and stores it in cloud storage.

[0657] Step 4: Analyzing the video data

[0658] server:

[0659] The stored video data is analyzed by an AI module (e.g., TensorFlow or OpenCV) on the server, specifically for person recognition and behavioral pattern analysis.

[0660] input:

[0661] Saved video data.

[0662] output:

[0663] Analysis results (e.g., detection of specific individuals or suspicious behavior).

[0664] Specific behavior:

[0665] AI models process video frames sequentially to detect suspicious behavior or specific individuals, and facial recognition algorithms match them to wanted criminals.

[0666] Step 5: Alert generation and notification

[0667] server:

[0668] Based on the analysis results, an alert is generated as needed, including the detection of wanted criminals and signs of crime, and the generated alert is sent to the device.

[0669] input:

[0670] Analysis results by AI module.

[0671] output:

[0672] Alert notifications sent to your device.

[0673] Specific behavior:

[0674] The server constructs an alert message and sends it to the device via a REST API.

[0675] Step 6: Check the warning and enter the response results

[0676] Device:

[0677] The device notifies the police officer of the alarm received from the server using a visual display, an audible voice alert, and a vibration function.

[0678] input:

[0679] Alert notifications sent from the server.

[0680] output:

[0681] Input of the alert message and response results that will be displayed to the police officer.

[0682] Specific behavior:

[0683] The device displays warning messages such as "Possible wanted fugitive" or "Suspicious behavior detected" and beeps and vibrates to alert the police officer, who then enters the corresponding results into the device.

[0684] Step 7: Update data

[0685] server:

[0686] The server receives the response results from the police officers and updates the database, which is managed by MySQL or PostgreSQL.

[0687] input:

[0688] Response result data sent from the device.

[0689] output:

[0690] Updated database.

[0691] Specific behavior:

[0692] The entered response result data is sent to the server and immediately saved and updated in the database, allowing the latest status to be always available for reference throughout the system.

[0693] (Application example 1)

[0694] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0695] Modern police activities require rapid and accurate information gathering and situational assessment. However, conventional systems take time to detect suspicious behavior or wanted criminals, making it difficult for police officers to respond quickly on the scene. In addition, there are limited means to visually confirm this information, which places a heavy burden on police officers and can hinder efficient police activities. For these reasons, there is a need for a system that can obtain information in real time and respond quickly and accurately.

[0696] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0697] In this invention, the server includes means for receiving video data from cameras carried by police officers, means for saving the received video data, means for analyzing the saved video data, means for generating an alert based on the analysis results and notifying the police officers, means for receiving response results from the police officers and updating the database, and means for visually and audibly notifying the police officers via the smart glasses. This allows police officers to obtain information in real time at the scene and respond quickly and accurately. The use of the smart glasses allows for visual and audible confirmation of alerts, improving the efficiency and safety of police activities.

[0698] A "camera" is a photographic device carried by a police officer to capture video data at the scene.

[0699] "Video data" is digital data containing visual information captured by a camera.

[0700] The "receiving means" refers to a communication module and software for acquiring the video data transmitted from the camera.

[0701] The "means for storing" refers to data storage and associated software for holding the received video data in a storage device.

[0702] The "analysis means" refers to AI modules and algorithms that analyze stored video data and detect suspicious behavior or anomalies.

[0703] The "means for generating and notifying alerts" refers to a notification system that creates alerts based on the analysis results and notifies police officers.

[0704] The "means for receiving response results and updating the database" refers to a communication module and data processing system for capturing the response status from police officers and reflecting that information in the database.

[0705] "Smart glasses" are augmented reality (AR) devices that are equipped with cameras, displays, and communication functions and are worn by police officers.

[0706] "Visual and audible notification means" refers to the ability to display or audibly communicate warning information through the smart glasses' display or speaker.

[0707] This invention relates to a security system that works in conjunction with cameras carried by police officers, analyzes video data in real time, and issues an immediate alarm. This system is composed of the following elements to enable rapid information gathering and response in police activities:

[0708] System Configuration

[0709] 1. Server

[0710] The server includes hardware and software that receives and stores video data sent from police officers' cameras. It also contains an AI module (using TensorFlow) that analyzes the video data to detect suspicious behavior or individuals.

[0711] It generates alerts based on the analysis results and sends them to police officers' devices or smart glasses in real time. It also receives response results from police officers and updates the database.

[0712] 2. Terminal (smart glasses)

[0713] The smart glasses are equipped with a camera that captures video data in real time, which is then transmitted to a server using WebSocket technology.

[0714] The smart glasses are equipped with an AR display that visually and audibly notifies police officers of warnings based on the analysis results. Police officers can then grasp the situation at the scene through the smart glasses and take necessary action.

[0715] 3. User (police officer)

[0716] Police officers wear smart glasses and patrol while capturing video data. When an alarm is notified, they quickly check the situation at the scene and report the results of their response to the server from the smart glasses.

[0717] Program processing and data calculation

[0718] 1. Camera video capture

[0719] The camera installed in the smart glasses captures video data and transmits it to a server in real time.

[0720] 2. Analysis of video data

[0721] The server analyzes the received video data using an AI module powered by TensorFlow, which detects suspicious behavior and people with high accuracy.

[0722] 3. Alert Generation and Notification

[0723] Based on the analysis results, the server generates an alert and notifies the police officers through the smart glasses' AR display and speaker, allowing them to understand the situation in real time.

[0724] 4. Reporting the results of the response

[0725] Police officers send the results of their response from the smart glasses to a server, which then updates the database with the information.

[0726] Specific examples

[0727] For example, while a police officer is patrolling a park, a camera mounted on a pair of smart glasses captures footage of a group of young people. The video data is sent to the server, where it is analyzed by an AI module and any suspicious behavior is detected. The server immediately generates an alert and notifies the smart glasses that "suspicious behavior has been detected." Police officers then use this information to investigate the scene and take any necessary action.

[0728] Prompt Sentence Examples

[0729] "In the security field, I am developing a system that analyzes real-time video data captured by smart glasses to detect suspicious behavior and wanted criminals. In this system, the server analyzes the video data, generates an alarm, and displays it on the smart glasses. Specifically, could you please tell me what kind of code I should implement?"

[0730] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0731] Step 1:

[0732] The camera in the smart glasses captures the surrounding video data. The input is live video of the scene, and the output is digitized video data. The camera captures video in real time and transmits the data to the built-in communication module.

[0733] Step 2:

[0734] The communication module of the smart glasses transmits the captured video data to the server. The input is the captured video data, and the output is the data to be sent to the server. WebSocket technology is used to transfer the data to the server with low latency.

[0735] Step 3:

[0736] The server stores the received video data. The input is the video data sent from the smart glasses, and the output is the data stored in the storage. The server records the data in a database and keeps it for subsequent analysis.

[0737] Step 4:

[0738] The server analyzes the stored video data. The input is the stored video data, and the output is the analysis results. An AI module (using TensorFlow) extracts and analyzes suspicious behavior and people from the video data.

[0739] Step 5:

[0740] The server generates an alert based on the analysis results. The input is the analysis results and the output is the alert data. If suspicious behavior or people are detected, an alert is immediately created and its contents are defined.

[0741] Step 6:

[0742] The server generates an alarm and notifies the smart glasses. The input is the alarm data, and the output is a visual and audible alarm notification. The alarm is notified to police officers using the AR display and speaker of the smart glasses.

[0743] Step 7:

[0744] The user (police officer) checks the alarm and takes appropriate action at the scene. The input is the alarm notification from the smart glasses, and the output is the response result at the scene. The police officer checks the notification and takes the necessary action on the spot (e.g., questioning, checking the scene).

[0745] Step 8:

[0746] The user (police officer) reports the response result to the server from the smart glasses. The input is the response result, and the output is the response data sent to the server. The result is entered through the input UI of the smart glasses and sent to the server.

[0747] Step 9:

[0748] The server reflects the received response results in the database. The input is the reported response results, and the output is the updated database. The server records the response results in the database, making it easier to understand the overall situation.

[0749] The above are the specific processing steps of this system.

[0750] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0751] This invention relates to a system that combines a system that analyzes video data in real time in conjunction with a constantly running camera carried by police officers, with an emotion engine that recognizes the user's emotions. This system is composed of the following elements:

[0752] 1. Server

[0753] The server includes hardware and software for receiving and storing video data sent from police officers' devices. It also includes an AI module and emotion engine for analyzing the stored video data. The server generates alerts based on the analysis results and sends them to the police officers' devices. It also receives response results from police officers and updates the database.

[0754] 2. Terminal

[0755] The officer's terminal communicates with the camera and captures video data. It is equipped with a communication module that transmits video data to the server in real time. It also notifies officers of received alarms and issues warnings through visual, audible, and vibration alerts. It also provides an interface for officers to input the results of their response and transmits the data to the server.

[0756] 3. User (police officer)

[0757] Police officers use this system when patrolling and questioning police officers. By wearing a camera and a device, video data is sent to a server in real time, and if suspicious behavior or abnormalities are detected, they are immediately notified. An emotion engine is also added to recognize the user's emotions, allowing for more accurate responses according to the situation. After confirming the notification, the police officer takes action according to the situation and updates the database by entering the results into the device.

[0758] As a specific example of the operation, the following scenario can be considered.

[0759] Example 1: Police interrogation assistance and emotion recognition

[0760] server:

[0761] A police officer's camera captures a person while patrolling a busy street at night.

[0762] The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals.

[0763] As a result of the match, it is determined that there is a high probability that the person is a wanted criminal, and the emotion engine analyzes the person's emotions and detects signs of anxiety or fear.

[0764] The server immediately generates an alert and notifies the terminal.

[0765] Device:

[0766] The device receives the alert and displays the message "Possible wanted criminal" along with the emotion analysis results on the screen. It indicates strong emotions of "anxiety" and "fear." The device immediately notifies police officers using a voice alert and vibration function.

[0767] User (police officer):

[0768] Police officers will check this notification and question the person in question. They will proceed with questioning carefully, taking into consideration the results of the emotion analysis. If the person's identity is confirmed and they are in fact wanted, the police officer will immediately arrest the person.

[0769] Example 2: Detecting signs of crime and emotional changes

[0770] server:

[0771] A police officer's camera patrolling a park during the day captures a scene of several young people gathered together and behaving abnormally.

[0772] The server uses an AI module to analyze the acquired video data and detect suspicious behavioral patterns. The emotion engine also analyzes the emotions of young people and detects signs of "excitement" or "agitation." An alert is generated and sent to the device.

[0773] Device:

[0774] The device receives an alarm notification, displays "Suspicious Activity Detected," and also displays the young people's emotional states of "Excitement" and "Arousal." The device immediately issues visual and audio alerts.

[0775] User (police officer):

[0776] Police officers can view the notification, proceed to the scene, approach the group of youths, and observe their behavior. They can also monitor emotional changes and prepare in advance. According to an embodiment of the present invention, police officers can respond not only to suspicious behavior but also to emotional changes in the subjects, allowing them to perform their duties more safely and effectively.

[0777] The processing flow will be explained below.

[0778] Step 1:

[0779] Device: Camera activation and video capture

[0780] The device activates the camera worn by the officer and begins capturing video data. The camera continues to capture video at 30 frames per second and transmits it to the device.

[0781] Step 2:

[0782] Terminal: Compression and transmission of video data

[0783] The device compresses and encrypts the video data acquired from the camera in real time, and then sends the compressed and encrypted video data to the server.

[0784] Step 3:

[0785] Server: Receiving and temporarily storing video data

[0786] The server receives the video data sent from the device and temporarily stores it, adding GPS information and a timestamp to the received video data.

[0787] Step 4:

[0788] Server: AI video analysis

[0789] An AI module on the server analyzes the temporarily stored video data in real time, using facial recognition and motion analysis algorithms to identify suspicious behavior and wanted criminals.

[0790] Step 5:

[0791] Server: Emotion analysis using emotion engine

[0792] In parallel with the AI ​​module's analysis, the server's emotion engine analyzes the subject's facial expressions from the video data and identifies emotions, detecting emotions such as "anxiety," "fear," "elation," and "excitement" in real time.

[0793] Step 6:

[0794] Server: Generate alerts

[0795] Based on the analysis results (AI behavior analysis and emotion analysis by the emotion engine), the server generates an alert if necessary. The alert includes the analysis results, location information, timestamp, and emotion analysis results.

[0796] Step 7:

[0797] Server: Alert notification

[0798] The server sends the generated alert to the terminal, which receives the alert and prepares to notify the police officer.

[0799] Step 8:

[0800] Terminal: Display of alert notification

[0801] The device notifies officers of received alarms visually and audibly. Detailed information about the alarm and the results of emotion analysis are displayed on the device screen, and the device alerts officers with a warning sound and vibration.

[0802] Step 9:

[0803] User (police officer): Response to alarm

[0804] Police officers check the alarms on the device and take action according to the situation at the scene, such as questioning the suspect, investigating the scene, and responding to emergencies if necessary. Based on the results of emotion analysis, the response policy can be adjusted more appropriately.

[0805] Step 10:

[0806] Device: Enter response results

[0807] Police officers use the terminal interface to input the results of their response, including the details of the police stop, any suspicious objects found, and information about the people they have apprehended.

[0808] Step 11:

[0809] Device: Sending response results

[0810] The device sends the inputted response results to the server, where they are encrypted and sent securely to the server.

[0811] Step 12:

[0812] Server:Database Update

[0813] The server receives the correspondence results, updates the database, correlates them with existing video data and analysis results, and stores them for future analysis.

[0814] Through these steps, this system will support police officers in responding to crime scenes and improve the efficiency of crime prevention and resolution. The addition of an emotion engine will enable responses that take into account the emotional changes of the target, further improving reliability.

[0815] Example 2

[0816] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0817] When police officers are on patrol or conducting routine questioning, it is extremely important to quickly and accurately grasp the suspicious behavior and emotional state of the subject. However, with conventional systems, it is difficult to analyze video data in real time and determine emotions, making it difficult for police officers to respond appropriately immediately. In addition, there are limited methods for issuing alarm notifications, making it difficult for police officers to respond immediately.

[0818] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0819] In this invention, the server includes means for receiving video data from a camera carried by a police officer, means for saving the received video data, means for analyzing the saved video data, means for generating an alert based on the analysis results and notifying the police officer, means for receiving response results from the police officer and updating the database, means for analyzing the emotional state from the video data and generating an alert based on the analysis results, and means for notifying the police officer of the alert visually, audibly, and by vibration. This makes it possible to detect suspicious behavior and emotional states in real time and notify the police officer quickly and reliably.

[0820] A "camera carried by a police officer" is a small, continuously operational video camera that is worn on the body by a police officer when patrolling or conducting routine questioning.

[0821] The "means for receiving video data" is a set of hardware and software for receiving real-time video data transmitted from a camera.

[0822] The "means for storing received video data" is a data storage system for safely and efficiently storing received video data.

[0823] "Means for analyzing stored video data" refers to AI modules and analytical software for performing facial recognition and behavioral analysis on stored video data.

[0824] The "means for generating an alert based on the analysis results and notifying police officers" is a system for creating an alert based on the analyzed data and notifying the police officers' terminals.

[0825] The "means for receiving response results from police officers and updating the database" is a system for receiving information entered by police officers after responding to an incident at the scene and keeping the database up to date.

[0826] The "means for analyzing the emotional state from video data and generating an alert based on the analysis results" refers to an emotion recognition engine that analyzes the emotional state of people in the video and generates an appropriate alert based on the results.

[0827] "Means for notifying police officers of an alarm visually, audibly, and by vibration" means a notification system that uses a screen display, audio alerts, and the device's vibration function to quickly communicate an alarm to police officers.

[0828] "Means for analyzing in real time and detecting suspicious behavior and anomalies" refers to an AI-based analysis module that processes video data in real time and instantly detects abnormal or suspicious behavior.

[0829] "Means for obtaining location information of police officers on patrol and synchronizing it with video data" refers to a system for obtaining the current location of police officers using a location information system such as GPS and synchronizing it with video data.

[0830] This invention is a system that uses a constantly running camera and a terminal carried by a police officer to transmit video data to a server in real time and analyzes the video data. This system is equipped with an AI module and an emotion engine, and can detect suspicious behavior and emotional states.

[0831] Hardware and software used

[0832] 1. Camera: A high-resolution video camera (with infrared functionality) is used, which allows for clear image data to be acquired.

[0833] 2. Terminal: A mobile terminal equipped with an LTE or Wi-Fi communication module that transmits video data to a server in real time.

[0834] 3. Server: A high-performance server equipped with cloud storage (e.g., Amazon S3) and AI modules (e.g., OpenCV, TensorFlow). It also uses an emotion analysis engine (e.g., IBM Watson) to analyze the emotional state from video data.

[0835] Program processing

[0836] server

[0837] The server receives video data transmitted in real time from the device and stores it in data storage. It uses an AI module to analyze the stored data and compare it with a database of wanted criminals. It also uses an emotion engine to analyze the emotional state of people in the video and generates an alert based on the analysis results. The alert is transmitted via a system that notifies police officers with visual and audio alerts and vibrations.

[0838] Terminal

[0839] The device captures video data from the always-on camera and transmits it to the server in real time. It displays received alarms visually and notifies officers with audio alerts and vibrations. Officers enter their response results into the device, which then transmits the data to the server, updating the database.

[0840] User (police officer)

[0841] Police officers wear cameras and devices when patrolling and questioning people. They respond based on notifications from the system and enter the results of their actions into the device. This keeps the database updated with the latest information, enabling them to carry out their duties safely and effectively.

[0842] Specific examples

[0843] Example 1: Police interrogation assistance and emotion recognition

[0844] Server: A police officer patrolling a busy downtown area at night captures a person on camera. The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals. As a result of the comparison, it determines that there is a high probability that the person is a wanted criminal, and an emotion engine analyzes the person's emotions and detects signs of anxiety or fear. The server immediately generates an alert stating "possibly wanted criminal" or "anxiety detected" and notifies the device.

[0845] Device: The device receives the alarm notification and displays "Possible wanted criminal" and "Anxiety detected." It immediately notifies the police with an audio alert and vibration.

[0846] User (police officer): The police officer checks the notification and begins to carefully question the suspect. The police officer enters the results into the terminal and sends them to the server to update the database.

[0847] Prompt Sentence Examples

[0848] 1. "While patrolling downtown, the camera captured a person who appears to be a wanted criminal. Please run the AI ​​module to match the person and simultaneously perform sentiment analysis."

[0849] 2. "When conducting a police stop, please demonstrate how to respond if the person's emotions indicate anxiety or fear."

[0850] Example 2: Detecting signs of crime and emotional changes

[0851] Server: Captures a scene of several young people gathering in a park during the day and behaving abnormally. The server analyzes the video data and detects suspicious behavior. The emotion engine also analyzes signs of "excitement" or "fever." The server generates an alert for "suspicious behavior detected" or "excitement detected" and notifies the device.

[0852] Terminal: The terminal receives the alarm notification and displays "Suspicious behavior detected" or "Elevation detected." It immediately notifies the police officer with visual and audio alerts.

[0853] User (police officer): The police officer checks the notification, approaches the group of young people in question, and observes their behavior. He or she takes appropriate action depending on the situation, paying attention to any emotional changes. The results are entered into the terminal and sent to the server to update the database.

[0854] Prompt Sentence Examples

[0855] 1. "Analyze footage of several young people behaving suspiciously in a park and use the emotion engine to detect their emotional state."

[0856] 2. "Please provide procedures for police officers to safely respond to situations involving a group of adolescents who are agitated."

[0857] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0858] Program processing flow

[0859] Step 1: Capture video data

[0860] Subject: Terminal

[0861] The terminal communicates with a constantly running camera carried by the police officer and captures video data in real time. The camera records video in high resolution and transmits the video data to the terminal. Specifically, the camera is attached to the officer's breast pocket and constantly records video of the surrounding area. The input data is the video of the surrounding area, and the output is the captured video data.

[0862] Step 2: Sending video data

[0863] Subject: Terminal

[0864] The device transmits the captured video data to the server in real time. The video data is compressed before transmission to efficiently transfer it to the server, for example, using H.264 encoding to reduce the data volume. The input data is the captured video data, and the output is compressed video data.

[0865] Step 3: Receiving and saving video data

[0866] Subject: Server

[0867] The server receives the video data sent from the device and stores it in cloud storage. The storage used is a highly reliable cloud service (e.g., Amazon S3). The input data is compressed video data, and the output is stored video data.

[0868] Step 4: Analyzing the video data

[0869] Subject: Server

[0870] The server analyzes the stored video data using an AI module. Specifically, it uses OpenCV and TensorFlow to perform facial recognition and motion analysis. For example, it extracts the faces of people in the video and compares them with a database of wanted criminals. The input data is the stored video data, and the output is the analysis results (e.g., facial recognition results, motion analysis results).

[0871] Step 5: Sentiment analysis and alert generation

[0872] Subject: Server

[0873] The server uses an emotion engine to perform emotion analysis based on the analyzed data. For example, it uses IBM Watson's emotion analysis API to analyze emotions such as "anxiety," "fear," and "elation" from the facial expressions of people in the video. Based on the analysis results, it generates an appropriate alert. The input data is the analysis result, and the output is the generated alert.

[0874] Step 6: Alert notification and response

[0875] Subject: Terminal

[0876] The device receives the alert sent from the server and notifies the police officer. It displays the alert content in a visual interface and issues an immediate warning using audio alerts and the device's vibration function. For example, it displays messages such as "Possible wanted fugitive" or "Anxiety detected." The input data is the generated alert, and the output is the alert content notified to the police officer.

[0877] Step 7: Sending response results and updating the database

[0878] Subject: User (police officer)

[0879] After confirming the notification, the police officer responds at the scene. The police officer inputs the response results (e.g., the results of the police questioning or a situation report) into the terminal and sends the data to the server. The server receives the response results and updates the database. The input data is the response results at the scene, and the output is the updated database information.

[0880] (Application example 2)

[0881] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0882] Conventional security systems simply store camera footage, making it difficult to detect suspicious behavior or emotional changes in real time. As a result, there is a lack of information to make appropriate decisions in situations where a quick response is required. As a result, it is difficult to implement effective security measures to prevent danger.

[0883] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0884] In this invention, the server includes means for receiving video data from cameras carried by security guards, means for saving the received video data, means for analyzing the saved video data, and means including an emotion engine for recognizing the emotions of the target based on the analyzed video data. This makes it possible to detect not only suspicious behavior and abnormalities in real time, but also changes in the target's emotions, allowing for faster and more accurate responses.

[0885] A "security guard" is a person whose job is to monitor facilities and locations, detect abnormalities and suspicious behavior, and respond accordingly.

[0886] A "camera" is a photographing device that captures video data and transmits it to a server in real time.

[0887] "Video data" refers to information about images and videos captured by a camera.

[0888] The "receiving means" is a mechanism by which the server receives the video data transmitted from the camera.

[0889] The "storage means" is a mechanism for temporarily or permanently storing received video data in a server.

[0890] The "analysis means" is a mechanism equipped with an AI module that analyzes stored video data and detects suspicious behavior or abnormalities.

[0891] An "alarm" is an alert generated based on the analysis results and notified to security personnel visually, audibly, or in other ways.

[0892] The "notification means" is a mechanism for transmitting the generated alarm to a security guard.

[0893] "Response results" is information that security guards report on the results of actions they take based on an alarm.

[0894] A "database" is a structured collection of information that stores and makes available past response results and other related information.

[0895] The "emotion engine" is a system with an algorithm for recognizing a subject's emotional state (e.g., anxiety, elation, fear, etc.) based on video data.

[0896] "Location information" refers to information about the current location of security guards obtained using technology such as GPS.

[0897] MODE FOR CARRYING OUT THE INVENTION

[0898] System program for implementing this application example

[0899] This system is realized through the cooperation of smart glasses carried by security guards, a server, and a database. The smart glasses are equipped with a camera that is always on and captures video data in real time. The captured video data is sent to the server via a communications module. The server stores the received video data and analyzes it using an AI module and emotion engine. Based on the analysis results, an alarm is generated and notified to the security guard. The system also receives the guard's response results and updates the database.

[0900] Explain the generated program's processing in natural language

[0901] Server Processing

[0902] 1. Hardware and Software:

[0903] The server is equipped with hardware with high-performance processing capabilities, as well as an AI module and emotion engine for analyzing video data.

[0904] The AI ​​module includes image analysis libraries (e.g., OpenCV), and the emotion engine includes emotion recognition models (e.g., IBM Watson Emotion API).

[0905] The server uses a communication module to receive and automatically store video data sent from the smart glasses.

[0906] 2. Data Analysis:

[0907] The received video data is analyzed in real time to detect suspicious behavior and abnormalities.

[0908] The emotion engine recognizes the subject's emotions and analyzes their emotional state (e.g., anxiety, elation, fear, etc.).

[0909] 3. Generate an alert:

[0910] Generate alerts based on the analysis results.

[0911] The alert will include details such as the type of activity detected and the emotional state.

[0912] 4. Notifications and Database Updates:

[0913] The generated alert is sent to the smart glasses terminal via visual, auditory, and vibration notifications.

[0914] The results of the security guard's response are received and the database is updated.

[0915] Smart Glasses Processing

[0916] 1. Video Capture:

[0917] A camera mounted on the smart glasses captures video data in real time.

[0918] 2. Data transmission:

[0919] The video data is transmitted to the server via the communication module.

[0920] 3. Alarm notification:

[0921] Receives alarm notifications from the server and notifies security guards.

[0922] Notification methods include visual (display on the screen), audible (voice alert), and vibration.

[0923] Database Processing

[0924] 1. Data storage and updating:

[0925] The video data received from the server, analysis results, and security guard response results are stored.

[0926] Add specific examples to the description

[0927] Examples:

[0928] Example 1:

[0929] "While a security guard is patrolling a building in the middle of the night, the smart glasses capture a suspicious person. The server analyzes the video and determines that the person is trying to break into a locked room. The emotion engine detects the person's expression of impatience and immediately issues an alarm."

[0930] Example 2:

[0931] "Smart glasses capture a person behaving suspiciously in the crowd at a train station. The server then analyzes the person and, through profiling, identifies that the person has behaved similarly in the past, and detects that they have a particularly dangerous tendency. The emotion engine then detects signs of tension in the person's facial expressions and sends an immediate notification."

[0932] Example prompt for a generative AI model:

[0933] "Security guards wearing smart glasses capture video in real time and send it to a server. The server then uses an AI module to analyze the video data, and if it detects any suspicious behavior, it notifies the security guard in real time. It also detects emotions and helps the security guard respond appropriately based on the emotional state of the person in question."

[0934] The above is the specific content of the "Mode for Carrying Out the Invention." This demonstrates that the present invention is a system that can identify suspicious behavior and emotional changes in real time, enabling rapid and accurate security responses.

[0935] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0936] Step 1:

[0937] Video Capture

[0938] Description: The camera installed in the smart glasses that serve as the terminal captures video data in real time.

[0939] Input: Video of the guard's surroundings.

[0940] What it does: The camera continuously captures video frames and stores them in a buffer.

[0941] Output: Captured video data.

[0942] Step 2:

[0943] Data transmission

[0944] Description: The device sends captured video data to the server.

[0945] Input: Captured video data.

[0946] Specific operation: Using a communication module, the video data is sent to a server via the Internet, often using the HTTP protocol.

[0947] Output: Video data sent to the server.

[0948] Step 3:

[0949] Data reception and storage

[0950] Description: The server receives and stores video data sent from the device.

[0951] Input: Video data sent from the device.

[0952] Specific operation: The communication module receives the data and stores it in local storage or a database.

[0953] Output: Saved video data.

[0954] Step 4:

[0955] Data analysis

[0956] Description: The server analyzes the stored video data to detect suspicious behavior and anomalies. It also analyzes the emotional state using an emotion engine.

[0957] Input: Stored video data.

[0958] How it works: An AI module (e.g., OpenCV) is used to analyze behaviors and facial expressions in the video data. An emotion engine (e.g., IBM Watson Emotion API) evaluates the subject's emotional state.

[0959] Output: Behavioral analysis results and emotion analysis results.

[0960] Step 5:

[0961] Generate an alert

[0962] Description: The server generates an alert based on the analysis results.

[0963] Input: Behavioral analysis results and emotion analysis results.

[0964] Specific behavior: Evaluate the analysis results and generate an alert if certain conditions are met (e.g., detecting suspicious behavior or a specific emotional state). The alert can be generated in multiple ways, including visual, auditory, or vibration.

[0965] Output: Generated alert data.

[0966] Step 6:

[0967] Alert notification

[0968] Description: Notifies the terminal of the alarm generated by the server. The alarm is delivered to the security guard by visual, auditory and vibration means.

[0969] Input: Generated alert data.

[0970] Specific operation: The communication module sends alarm data to the terminal, which displays the received alarm on the screen and alerts the security guard with an alarm sound or vibration.

[0971] Output: Alarm notified to security personnel.

[0972] Step 7:

[0973] Enter and submit the response results

[0974] Description: The user (security guard) responds based on the alarm and inputs the response results into the terminal. The terminal then sends the response results to the server.

[0975] Input: The action result entered by the user.

[0976] Specific operation: The security guard uses the terminal interface to input the response results and sends the data to the server using the communication module.

[0977] Output: The response result sent to the server.

[0978] Step 8:

[0979] Database Update

[0980] Description: The server receives the response and updates the database.

[0981] Input: The response result sent.

[0982] Specific operation: The server adds or updates the response result data it receives to the database and stores it as history, thereby accumulating data that can be used for future analysis.

[0983] Output: The updated database.

[0984] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0985] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0986] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0987] [Third embodiment]

[0988] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0989] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0990] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0991] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0992] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0993] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0994] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0995] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0996] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0997] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0998] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0999] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1000] The present invention relates to a system that analyzes video data in real time in conjunction with a constantly running camera carried by a police officer. This system is composed of the following elements:

[1001] 1. Server

[1002] The server includes hardware and software for receiving and storing video data sent from police officers' terminals. It also includes an AI module that analyzes the stored video data. The server generates alerts based on the analysis results and sends them to the police officers' terminals. It also receives response results from police officers and updates the database.

[1003] 2. Terminal

[1004] The officer's terminal communicates with the camera and captures video data. It is equipped with a communication module that transmits video data to the server in real time. It also notifies officers of received alarms and issues warnings through visual, audible, and vibration alerts. It also provides an interface for officers to input the results of their response and transmits the data to the server.

[1005] 3. User (police officer)

[1006] Police officers use this system when patrolling or questioning police officers. By wearing cameras and devices, video data is sent to a server in real time, and if suspicious behavior or abnormalities are detected, they are immediately notified. Upon receiving the notification, the police officer takes appropriate action according to the situation and updates the database by entering the results into the device.

[1007] As a specific example of the operation, the following scenario can be considered.

[1008] Example 1: Supporting police questioning

[1009] server:

[1010] A police officer's camera captures a person while patrolling a busy street at night.

[1011] The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals.

[1012] If the result of the comparison indicates that there is a high probability that the person is a wanted criminal, the server immediately generates an alert and notifies the terminal.

[1013] Device:

[1014] The device receives the alert and displays the message "Possible wanted criminal" on the screen. The device immediately notifies police officers using audio alerts and vibration functions.

[1015] User (police officer):

[1016] Police officers will check this notification and question the person in question. If they are able to confirm the person's identity and find that the person is in fact wanted, the police officers will immediately arrest the person.

[1017] Example 2: Detecting signs of crime

[1018] server:

[1019] A police officer's camera patrolling a park during the day captures a scene of several young people gathered together and behaving abnormally.

[1020] The server analyzes the acquired video data using an AI module, and if a suspicious behavior pattern is detected, it determines that there is a "sign of a crime," generates an alert, and notifies the device.

[1021] Device:

[1022] The device receives the alarm notification, displays "Suspicious Activity Detected," and notifies the police. The device immediately issues a visual and audio alert.

[1023] User (police officer):

[1024] Police officers will review the notification, proceed to the scene, approach the group of young people in question, and investigate their behavior. If they find any suspicious items, they will conduct further investigations and take necessary action on the spot.

[1025] The above specific example is intended to enable immediate response at the scene of a crime and to prevent and resolve it quickly. This system is expected to improve the accuracy of police officers' judgment, and to increase the efficiency and reliability of police activities.

[1026] The processing flow will be explained below.

[1027] Step 1:

[1028] Device: Camera activation and video capture

[1029] The device activates the camera worn by the officer and begins capturing video data. The camera continues to capture video at 30 frames per second and transmits it to the device.

[1030] Step 2:

[1031] Terminal: Compression and transmission of video data

[1032] The device compresses and encrypts the video data acquired from the camera in real time, and then sends the compressed and encrypted video data to the server.

[1033] Step 3:

[1034] Server: Receiving and temporarily storing video data

[1035] The server receives the video data sent from the device and temporarily stores it, adding GPS information and a timestamp to the received video data.

[1036] Step 4:

[1037] Server: AI video analysis

[1038] An AI module on the server analyzes the temporarily stored video data in real time, using facial recognition and motion analysis algorithms to identify suspicious behavior and wanted criminals.

[1039] Step 5:

[1040] Server: Generate alerts

[1041] Based on the analysis results of the AI ​​module, the server generates an alert if necessary, which includes the analysis results, location information, and a timestamp.

[1042] Step 6:

[1043] Server: Alert notification

[1044] The server sends the generated alert to the terminal, which receives the alert and prepares to notify the police officer.

[1045] Step 7:

[1046] Terminal: Display of alert notification

[1047] The device notifies officers of received alarms visually and audibly, displays detailed information on the alarm on the device's screen, and alerts officers with a warning sound and vibration.

[1048] Step 8:

[1049] User (police officer): Response to alarm

[1050] Police officers will check the alarm on the device and take action according to the situation at the scene, such as questioning the suspect, checking the scene, and in some cases, responding to an emergency.

[1051] Step 9:

[1052] Device: Enter response results

[1053] Police officers use the terminal interface to input the results of their response, including the details of the police stop, any suspicious objects found, and information about the person they have apprehended.

[1054] Step 10:

[1055] Device: Sending response results

[1056] The device sends the inputted response results to the server, where they are encrypted and sent securely to the server.

[1057] Step 11:

[1058] Server:Database Update

[1059] The server receives the correspondence results and updates the database, correlating them with existing video data and analysis results, and saving them for future analysis.

[1060] Through these steps, the system will support police officers in responding to crimes on the scene and improve the efficiency of crime prevention and resolution.

[1061] Example 1

[1062] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1063] In traditional police work, police officers are required to provide fast and accurate information so that they can grasp the situation at the scene and respond immediately. However, identifying suspicious individuals and detecting signs of crime at the scene relies heavily on the experience and intuition of police officers, which can lead to delayed responses and incorrect judgments. Furthermore, delayed feedback of information from the scene can affect database updates and subsequent investigative activities. To solve these issues, a system is needed that allows police officers to grasp the situation at the scene in real time and respond appropriately.

[1064] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1065] In this invention, the server includes means for receiving video data from a camera carried by a police officer, means for saving the received video data, means for using an AI module to analyze the saved video data, means for generating an alert based on the analysis results and notifying the police officer, and means for receiving response results from the police officer and updating the database, thereby enabling the police officer to visually grasp the situation at the scene in real time and respond quickly and accurately based on the analysis results of the AI.

[1066] A "police officer" is a public servant who performs duties to maintain law and order and ensure public safety.

[1067] "Photographic equipment" refers to equipment carried by police officers and used to capture video data at the scene, including, for example, cameras and body cams.

[1068] "Video Data" means information in the form of video captured by a camera that records police officers' activities and the surrounding environment.

[1069] "Means for receiving" refers to the technical mechanism by which the server captures the video data transmitted from the imaging device.

[1070] "Storage means" means the technical mechanism used to safely and efficiently store received video data for a certain period of time, including, for example, a database or cloud storage.

[1071] "Analysing means" means technical mechanisms, such as artificial intelligence modules, used to process stored video data and extract specific information.

[1072] An "artificial intelligence module" is a software or hardware component used to analyze or recognize data, including, for example, machine learning algorithms and deep learning models.

[1073] "Means for generating an alert" means the technical mechanism that generates an alert to notify police officers when an abnormality or suspicious activity is detected based on the analysis results.

[1074] "Means for notifying" refers to the technical mechanism for transmitting the generated alert to the police officer's terminal and conveying the information to the police officer.

[1075] "Means for receiving response results" refers to the technical mechanism by which police officers input the results of their response at the scene and import the results into a server.

[1076] "Means for updating the database" refers to the technical mechanisms for amending and adding to the database in order to keep the data in the entire system up to date based on the results of police officers' responses.

[1077] "Suspicious behavior" refers to behavior that deviates from typical patterns of behavior and is considered to be an indication of a potential crime.

[1078] "Anomaly" means a condition or occurrence that is not observed or expected under normal circumstances.

[1079] "Location information" refers to geographical data about a police officer's current location, such as GPS data.

[1080] The present invention relates to a system that analyzes video data in real time in conjunction with a constantly running camera carried by a police officer. This system is composed of the following elements:

[1081] server

[1082] The server includes hardware and software for receiving and storing video data sent from police officers' devices. The received video data is securely stored using cloud storage (e.g., Amazon S3). The server then analyzes the stored video data using an AI module (e.g., TensorFlow or OpenCV). Specifically, the AI ​​module analyzes video frames and recognizes suspicious behavior or specific individuals. The server also generates alerts based on the analysis results and notifies the police officers' devices of these alerts. It also receives response results from police officers and updates a database (e.g., MySQL or PostgreSQL).

[1083] Terminal

[1084] The officer's terminal is responsible for capturing video data in conjunction with the always-on camera. The terminal is equipped with a high-performance camera module (e.g., Logitech C920). It also has a communication module (e.g., 4G / 5G modem) that transmits video data to the server in real time. Received alerts are notified to the officer using a visual display, an audible voice alert, and a vibration function. An interface (e.g., touchscreen, keyboard) is also provided for the officer to input the response results. This data is then sent to the server.

[1085] User (police officer)

[1086] Police officers use this system when patrolling or questioning police officers. By using a constantly running camera and wearing a terminal, video data is sent to a server in real time, and if suspicious behavior or abnormalities are detected, they are immediately notified. Upon receiving the notification, the police officer responds according to the situation. The results are entered into the terminal and sent to and recorded on the server.

[1087] Specific examples

[1088] Example 1: Supporting police questioning

[1089] server:

[1090] A police officer's camera patrolling a busy downtown area at night captures a person. The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals. If the comparison results in a high probability that the person is a wanted criminal, the server immediately generates an alert and notifies the device.

[1091] Device:

[1092] The device receives the alert and displays the message "Possible wanted criminal" on the screen. The device immediately notifies police officers using audio alerts and vibration functions.

[1093] User (police officer):

[1094] Police officers will check the notification and question the person in question. If they are able to confirm the person's identity and find that the person is in fact wanted, the police officers will immediately arrest the person.

[1095] Example 2: Detecting signs of crime

[1096] server:

[1097] A police officer's camera patrolling a park during the day captures a scene in which several young people are gathered and behaving abnormally. The server analyzes the acquired video data using an AI module, and if a suspicious behavior pattern is detected, it determines that there is a "sign of a crime." An alert is generated and sent to the device.

[1098] Device:

[1099] The device receives the alarm notification, displays "Suspicious Activity Detected," and notifies the police. The device immediately issues a visual and audio alert.

[1100] User (police officer):

[1101] Police officers will review the notification, proceed to the scene, approach the group of young people in question, and investigate their behavior. If they find any suspicious items, they will conduct further investigations and take appropriate action on the spot.

[1102] Prompt Sentence Examples

[1103] 1. Sample prompts for police questioning assistance:

[1104] The server receives real-time video data from police officers' cameras, uses an AI module to recognize the faces of wanted criminals, and compares them with a database. If the criminal is determined to be wanted, an alert is generated and sent to the device.

[1105] 2. Example of a prompt for crime detection:

[1106] The AI ​​module analyzes the video data acquired by the server, and if it detects a suspicious pattern of behavior, it determines that it is a "sign of a crime," generates an alert, and notifies the device.

[1107] This system will enable police officers to grasp the situation on the scene in real time and respond quickly and accurately, which is expected to improve the efficiency and reliability of police activities.

[1108] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1109] Program processing flow and specific explanation

[1110] Step 1: Capture video data

[1111] Device:

[1112] The police officer carries a device that captures video data in real time through a connected always-on camera, and the device is equipped with a high-performance camera module (e.g., Logitech C920).

[1113] input:

[1114] Successive video frames acquired from a camera.

[1115] output:

[1116] Captured video data.

[1117] Specific behavior:

[1118] As officers walk around on patrol, the device continuously captures video frames and temporarily stores the video data in its internal memory.

[1119] Step 2: Sending video data

[1120] Device:

[1121] The device transmits the captured video data to the server in real time using a 4G / 5G modem.

[1122] input:

[1123] Captured video data.

[1124] output:

[1125] Video data sent to the server.

[1126] Specific behavior:

[1127] The video data is compressed using technology to reduce the amount of data, and then sent to the server via high-speed communication.

[1128] Step 3: Receiving and saving video data

[1129] server:

[1130] The server stores the received video data, which is then securely transferred to cloud storage (e.g., Amazon S3).

[1131] input:

[1132] Video data sent from the device.

[1133] output:

[1134] Securely stored video data.

[1135] Specific behavior:

[1136] The server checks the accuracy and completeness of the data, timestamps it, and stores it in cloud storage.

[1137] Step 4: Analyzing the video data

[1138] server:

[1139] The stored video data is analyzed by an AI module (e.g., TensorFlow or OpenCV) on the server, specifically for person recognition and behavioral pattern analysis.

[1140] input:

[1141] Saved video data.

[1142] output:

[1143] Analysis results (e.g., detection of specific individuals or suspicious behavior).

[1144] Specific behavior:

[1145] AI models process video frames sequentially to detect suspicious behavior or specific individuals, and facial recognition algorithms match them to wanted criminals.

[1146] Step 5: Alert generation and notification

[1147] server:

[1148] Based on the analysis results, an alert is generated as needed, including the detection of wanted criminals and signs of crime, and the generated alert is sent to the device.

[1149] input:

[1150] Analysis results by AI module.

[1151] output:

[1152] Alert notifications sent to your device.

[1153] Specific behavior:

[1154] The server constructs an alert message and sends it to the device via a REST API.

[1155] Step 6: Check the warning and enter the response results

[1156] Device:

[1157] The device notifies the police officer of the alarm received from the server using a visual display, an audible voice alert, and a vibration function.

[1158] input:

[1159] Alert notifications sent from the server.

[1160] output:

[1161] Input of the alert message and response results that will be displayed to the police officer.

[1162] Specific behavior:

[1163] The device displays warning messages such as "Possible wanted fugitive" or "Suspicious behavior detected" and alerts the police officer with beeps and vibrations, who then enters the corresponding results into the device.

[1164] Step 7: Update data

[1165] server:

[1166] The server receives the response results from the police officers and updates the database, which is managed by MySQL or PostgreSQL.

[1167] input:

[1168] Response result data sent from the device.

[1169] output:

[1170] Updated database.

[1171] Specific behavior:

[1172] The entered response result data is sent to the server and immediately saved and updated in the database, allowing the latest status to be always available for reference throughout the system.

[1173] (Application example 1)

[1174] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1175] Modern police activities require rapid and accurate information gathering and situational assessment. However, conventional systems take time to detect suspicious behavior or wanted criminals, making it difficult for police officers to respond quickly on the scene. In addition, there are limited means to visually confirm this information, which places a heavy burden on police officers and can hinder efficient police activities. For these reasons, there is a need for a system that can obtain information in real time and respond quickly and accurately.

[1176] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1177] In this invention, the server includes means for receiving video data from cameras carried by police officers, means for saving the received video data, means for analyzing the saved video data, means for generating an alert based on the analysis results and notifying the police officers, means for receiving response results from the police officers and updating the database, and means for visually and audibly notifying the police officers via the smart glasses. This allows police officers to obtain information in real time at the scene and respond quickly and accurately. The use of the smart glasses allows for visual and audible confirmation of alerts, improving the efficiency and safety of police activities.

[1178] A "camera" is a photographic device carried by a police officer to capture video data at the scene.

[1179] "Video data" is digital data containing visual information captured by a camera.

[1180] The "receiving means" refers to a communication module and software for acquiring the video data transmitted from the camera.

[1181] The "means for storing" refers to data storage and associated software for holding the received video data in a storage device.

[1182] The "analysis means" refers to AI modules and algorithms that analyze stored video data and detect suspicious behavior or anomalies.

[1183] The "means for generating and notifying alerts" refers to a notification system that creates alerts based on the analysis results and notifies police officers.

[1184] The "means for receiving response results and updating the database" refers to a communication module and data processing system for capturing the response status from police officers and reflecting that information in the database.

[1185] "Smart glasses" are augmented reality (AR) devices that are equipped with cameras, displays, and communication functions and are worn by police officers.

[1186] "Visual and audible notification means" refers to the ability to display or audibly communicate warning information through the smart glasses' display or speaker.

[1187] This invention relates to a security system that works in conjunction with cameras carried by police officers, analyzes video data in real time, and issues an immediate alarm. This system is composed of the following elements to enable rapid information gathering and response in police activities:

[1188] System Configuration

[1189] 1. Server

[1190] The server includes hardware and software that receives and stores video data sent from police officers' cameras. It also contains an AI module (using TensorFlow) that analyzes the video data to detect suspicious behavior or individuals.

[1191] It generates alerts based on the analysis results and sends them to police officers' devices or smart glasses in real time. It also receives response results from police officers and updates the database.

[1192] 2. Terminal (smart glasses)

[1193] The smart glasses are equipped with a camera that captures video data in real time, which is then transmitted to a server using WebSocket technology.

[1194] The smart glasses are equipped with an AR display that visually and audibly notifies police officers of warnings based on the analysis results. Police officers can then grasp the situation at the scene through the smart glasses and take necessary action.

[1195] 3. User (police officer)

[1196] Police officers wear smart glasses and patrol while capturing video data. When an alarm is notified, they quickly check the situation at the scene and report the results of their response to the server from the smart glasses.

[1197] Program processing and data calculation

[1198] 1. Camera video capture

[1199] The camera installed in the smart glasses captures video data and transmits it to a server in real time.

[1200] 2. Analysis of video data

[1201] The server analyzes the received video data using an AI module powered by TensorFlow, which detects suspicious behavior and people with high accuracy.

[1202] 3. Alert Generation and Notification

[1203] Based on the analysis results, the server generates an alert and notifies the police officers through the smart glasses' AR display and speaker, allowing them to understand the situation in real time.

[1204] 4. Reporting the results of the response

[1205] Police officers send the results of their response from the smart glasses to a server, which then updates the database with the information.

[1206] Specific examples

[1207] For example, while a police officer is patrolling a park, a camera mounted on a pair of smart glasses captures footage of a group of young people. The video data is sent to the server, where it is analyzed by an AI module and any suspicious behavior is detected. The server immediately generates an alert and notifies the smart glasses that "suspicious behavior has been detected." Police officers then use this information to investigate the scene and take any necessary action.

[1208] Prompt Sentence Examples

[1209] "In the security field, I am developing a system that analyzes real-time video data captured by smart glasses to detect suspicious behavior and wanted criminals. In this system, the server analyzes the video data, generates an alarm, and displays it on the smart glasses. Specifically, could you please tell me what kind of code I should implement?"

[1210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1211] Step 1:

[1212] The camera in the smart glasses captures the surrounding video data. The input is live video of the scene, and the output is digitized video data. The camera captures video in real time and transmits the data to the built-in communication module.

[1213] Step 2:

[1214] The communication module of the smart glasses transmits the captured video data to the server. The input is the captured video data, and the output is the data to be sent to the server. WebSocket technology is used to transfer the data to the server with low latency.

[1215] Step 3:

[1216] The server stores the received video data. The input is the video data sent from the smart glasses, and the output is the data stored in the storage. The server records the data in a database and keeps it for subsequent analysis.

[1217] Step 4:

[1218] The server analyzes the stored video data. The input is the stored video data, and the output is the analysis results. An AI module (using TensorFlow) extracts and analyzes suspicious behavior and people from the video data.

[1219] Step 5:

[1220] The server generates an alert based on the analysis results. The input is the analysis results and the output is the alert data. If suspicious behavior or people are detected, an alert is immediately created and its contents are defined.

[1221] Step 6:

[1222] The server generates an alarm and notifies the smart glasses. The input is the alarm data, and the output is a visual and audible alarm notification. The alarm is notified to police officers using the AR display and speaker of the smart glasses.

[1223] Step 7:

[1224] The user (police officer) checks the alarm and takes appropriate action at the scene. The input is the alarm notification from the smart glasses, and the output is the response result at the scene. The police officer checks the notification and takes the necessary action on the spot (e.g., questioning, checking the scene).

[1225] Step 8:

[1226] The user (police officer) reports the response result to the server from the smart glasses. The input is the response result, and the output is the response data sent to the server. The result is entered through the input UI of the smart glasses and sent to the server.

[1227] Step 9:

[1228] The server reflects the received response results in the database. The input is the reported response results, and the output is the updated database. The server records the response results in the database, making it easier to understand the overall situation.

[1229] The above are the specific processing steps of this system.

[1230] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1231] This invention relates to a system that combines a system that analyzes video data in real time in conjunction with a constantly running camera carried by police officers, with an emotion engine that recognizes the user's emotions. This system is composed of the following elements:

[1232] 1. Server

[1233] The server includes hardware and software for receiving and storing video data sent from police officers' devices. It also includes an AI module and emotion engine for analyzing the stored video data. The server generates alerts based on the analysis results and sends them to the police officers' devices. It also receives response results from police officers and updates the database.

[1234] 2. Terminal

[1235] The officer's terminal communicates with the camera and captures video data. It is equipped with a communication module that transmits video data to the server in real time. It also notifies officers of received alarms and issues warnings through visual, audible, and vibration alerts. It also provides an interface for officers to input the results of their response and transmits the data to the server.

[1236] 3. User (police officer)

[1237] Police officers use this system when patrolling and questioning police officers. By wearing a camera and a device, video data is sent to a server in real time, and if suspicious behavior or abnormalities are detected, they are immediately notified. An emotion engine is also added to recognize the user's emotions, allowing for more accurate responses according to the situation. After confirming the notification, the police officer takes action according to the situation and updates the database by entering the results into the device.

[1238] As a specific example of the operation, the following scenario can be considered.

[1239] Example 1: Police interrogation assistance and emotion recognition

[1240] server:

[1241] A police officer's camera captures a person while patrolling a busy street at night.

[1242] The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals.

[1243] As a result of the match, it is determined that there is a high probability that the person is a wanted criminal, and the emotion engine analyzes the person's emotions and detects signs of anxiety or fear.

[1244] The server immediately generates an alert and notifies the terminal.

[1245] Device:

[1246] The device receives the alert and displays the message "Possible wanted criminal" along with the emotion analysis results on the screen. It indicates strong emotions of "anxiety" and "fear." The device immediately notifies police officers using a voice alert and vibration function.

[1247] User (police officer):

[1248] Police officers will check this notification and question the person in question. They will proceed with questioning carefully, taking into consideration the results of the emotion analysis. If the person's identity is confirmed and they are in fact wanted, the police officer will immediately arrest the person.

[1249] Example 2: Detecting signs of crime and emotional changes

[1250] server:

[1251] A police officer's camera patrolling a park during the day captures a scene of several young people gathered together and behaving abnormally.

[1252] The server uses an AI module to analyze the acquired video data and detect suspicious behavioral patterns. The emotion engine also analyzes the emotions of young people and detects signs of "excitement" or "agitation." An alert is generated and sent to the device.

[1253] Device:

[1254] The device receives an alarm notification, displays "Suspicious Activity Detected," and also displays the young people's emotional states of "Excitement" and "Arousal." The device immediately issues visual and audio alerts.

[1255] User (police officer):

[1256] Police officers can view the notification, proceed to the scene, approach the group of youths, and observe their behavior. They can also monitor emotional changes and prepare in advance. According to an embodiment of the present invention, police officers can respond not only to suspicious behavior but also to emotional changes in the subjects, allowing them to perform their duties more safely and effectively.

[1257] The processing flow will be explained below.

[1258] Step 1:

[1259] Device: Camera activation and video capture

[1260] The device activates the camera worn by the officer and begins capturing video data. The camera continues to capture video at 30 frames per second and transmits it to the device.

[1261] Step 2:

[1262] Terminal: Compression and transmission of video data

[1263] The device compresses and encrypts the video data acquired from the camera in real time, and then sends the compressed and encrypted video data to the server.

[1264] Step 3:

[1265] Server: Receiving and temporarily storing video data

[1266] The server receives the video data sent from the device and temporarily stores it, adding GPS information and a timestamp to the received video data.

[1267] Step 4:

[1268] Server: AI video analysis

[1269] An AI module on the server analyzes the temporarily stored video data in real time, using facial recognition and motion analysis algorithms to identify suspicious behavior and wanted criminals.

[1270] Step 5:

[1271] Server: Emotion analysis using emotion engine

[1272] In parallel with the AI ​​module's analysis, the server's emotion engine analyzes the subject's facial expressions from the video data and identifies emotions, detecting "anxiety," "fear," "elation," "excitement," and other emotions in real time.

[1273] Step 6:

[1274] Server: Generate alerts

[1275] Based on the analysis results (AI behavior analysis and emotion analysis by the emotion engine), the server generates an alert if necessary. The alert includes the analysis results, location information, timestamp, and emotion analysis results.

[1276] Step 7:

[1277] Server: Alert notification

[1278] The server sends the generated alert to the terminal, which receives the alert and prepares to notify the police officer.

[1279] Step 8:

[1280] Terminal: Display of alert notification

[1281] The device notifies officers of received alarms visually and audibly. Detailed information about the alarm and the results of emotion analysis are displayed on the device screen, and the device alerts officers with a warning sound and vibration.

[1282] Step 9:

[1283] User (police officer): Response to alarm

[1284] Police officers check the alarms on the device and take action according to the situation at the scene, such as questioning the suspect, investigating the scene, and responding to emergencies if necessary. Based on the results of emotion analysis, the response policy can be adjusted more appropriately.

[1285] Step 10:

[1286] Device: Enter response results

[1287] Police officers use the terminal interface to input the results of their response, including the details of the police stop, any suspicious objects found, and information about the person they have apprehended.

[1288] Step 11:

[1289] Device: Sending response results

[1290] The device sends the inputted response results to the server, where they are encrypted and sent securely to the server.

[1291] Step 12:

[1292] Server:Database Update

[1293] The server receives the correspondence results and updates the database, correlating them with existing video data and analysis results, and saving them for future analysis.

[1294] Through these steps, this system will support police officers in responding to crime scenes and improve the efficiency of crime prevention and resolution. The addition of an emotion engine will enable responses that take into account the emotional changes of the target, further improving reliability.

[1295] Example 2

[1296] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1297] When police officers are on patrol or conducting routine questioning, it is extremely important to quickly and accurately grasp the suspicious behavior and emotional state of the subject. However, with conventional systems, it is difficult to analyze video data in real time and determine emotions, making it difficult for police officers to respond appropriately immediately. In addition, there are limited methods for issuing alarm notifications, making it difficult for police officers to respond immediately.

[1298] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1299] In this invention, the server includes means for receiving video data from a camera carried by a police officer, means for saving the received video data, means for analyzing the saved video data, means for generating an alert based on the analysis results and notifying the police officer, means for receiving response results from the police officer and updating the database, means for analyzing the emotional state from the video data and generating an alert based on the analysis results, and means for notifying the police officer of the alert visually, audibly, and by vibration. This makes it possible to detect suspicious behavior and emotional states in real time and notify the police officer quickly and reliably.

[1300] A "camera carried by a police officer" is a small, continuously operational video camera that is worn on the body by a police officer when patrolling or conducting routine questioning.

[1301] The "means for receiving video data" is a set of hardware and software for receiving real-time video data transmitted from a camera.

[1302] The "means for storing received video data" is a data storage system for safely and efficiently storing received video data.

[1303] "Means for analyzing stored video data" refers to AI modules and analytical software for performing facial recognition and behavioral analysis on stored video data.

[1304] The "means for generating an alert based on the analysis results and notifying police officers" is a system for creating an alert based on the analyzed data and notifying the police officers' terminals.

[1305] The "means for receiving response results from police officers and updating the database" is a system for receiving information entered by police officers after responding to an incident at the scene and keeping the database up to date.

[1306] The "means for analyzing the emotional state from video data and generating an alert based on the analysis results" refers to an emotion recognition engine that analyzes the emotional state of people in the video and generates an appropriate alert based on the results.

[1307] "Means for notifying police officers of an alarm visually, audibly, and by vibration" means a notification system that uses a screen display, audio alerts, and the device's vibration function to quickly communicate an alarm to police officers.

[1308] "Means for analyzing in real time and detecting suspicious behavior and anomalies" refers to an AI-based analysis module that processes video data in real time and instantly detects abnormal or suspicious behavior.

[1309] "Means for obtaining location information of police officers on patrol and synchronizing it with video data" refers to a system for obtaining the current location of police officers using a location information system such as GPS and synchronizing it with video data.

[1310] This invention is a system that uses a constantly running camera and a terminal carried by a police officer to transmit video data to a server in real time and analyzes the video data. This system is equipped with an AI module and an emotion engine, and can detect suspicious behavior and emotional states.

[1311] Hardware and software used

[1312] 1. Camera: A high-resolution video camera (with infrared functionality) is used, which allows for clear image data to be acquired.

[1313] 2. Terminal: A mobile terminal equipped with an LTE or Wi-Fi communication module that transmits video data to a server in real time.

[1314] 3. Server: A high-performance server equipped with cloud storage (e.g., Amazon S3) and AI modules (e.g., OpenCV, TensorFlow). It also uses an emotion analysis engine (e.g., IBM Watson) to analyze the emotional state from video data.

[1315] Program processing

[1316] server

[1317] The server receives video data transmitted in real time from the device and stores it in data storage. It uses an AI module to analyze the stored data and compare it with a database of wanted criminals. It also uses an emotion engine to analyze the emotional state of people in the video and generates an alert based on the analysis results. The alert is transmitted via a system that notifies police officers with visual and audio alerts and vibrations.

[1318] Terminal

[1319] The device captures video data from the always-on camera and transmits it to the server in real time. It displays received alarms visually and notifies officers with audio alerts and vibrations. Officers enter their response results into the device, which then transmits the data to the server, updating the database.

[1320] User (police officer)

[1321] Police officers wear cameras and devices when patrolling and questioning people. They respond based on notifications from the system and enter the results of their actions into the device. This keeps the database updated with the latest information, enabling them to carry out their duties safely and effectively.

[1322] Specific examples

[1323] Example 1: Police interrogation assistance and emotion recognition

[1324] Server: A police officer patrolling a busy downtown area at night captures a person on camera. The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals. As a result of the comparison, it determines that there is a high probability that the person is a wanted criminal, and an emotion engine analyzes the person's emotions and detects signs of anxiety or fear. The server immediately generates an alert stating "possibly wanted criminal" or "anxiety detected" and notifies the device.

[1325] Device: The device receives the alarm notification and displays "Possible wanted criminal" and "Anxiety detected." It immediately notifies the police with an audio alert and vibration.

[1326] User (police officer): The police officer checks the notification and begins to carefully question the suspect. The police officer enters the results into the terminal and sends them to the server to update the database.

[1327] Prompt Sentence Examples

[1328] 1. "While patrolling downtown, the camera captured a person who appears to be a wanted criminal. Please run the AI ​​module to match the person and simultaneously perform sentiment analysis."

[1329] 2. "When conducting a police stop, please demonstrate how to respond if the person's emotions indicate anxiety or fear."

[1330] Example 2: Detecting signs of crime and emotional changes

[1331] Server: Captures a scene of several young people gathering in a park during the day and behaving abnormally. The server analyzes the video data and detects suspicious behavior. The emotion engine also analyzes signs of "excitement" or "fever." The server generates an alert for "suspicious behavior detected" or "excitement detected" and notifies the device.

[1332] Terminal: The terminal receives the alarm notification and displays "Suspicious behavior detected" or "Elevation detected." It immediately notifies the police officer with visual and audio alerts.

[1333] User (police officer): The police officer checks the notification, approaches the group of young people in question, and observes their behavior. He or she takes appropriate action depending on the situation, paying attention to any emotional changes. The results are entered into the terminal and sent to the server to update the database.

[1334] Prompt Sentence Examples

[1335] 1. "Analyze footage of several young people behaving suspiciously in a park and use the emotion engine to detect their emotional state."

[1336] 2. "Please provide procedures for police officers to safely respond to situations involving a group of adolescents who are agitated."

[1337] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1338] Program processing flow

[1339] Step 1: Capture video data

[1340] Subject: Terminal

[1341] The terminal communicates with a constantly running camera carried by the police officer and captures video data in real time. The camera records video in high resolution and transmits the video data to the terminal. Specifically, the camera is attached to the officer's breast pocket and constantly records video of the surrounding area. The input data is the video of the surrounding area, and the output is the captured video data.

[1342] Step 2: Sending video data

[1343] Subject: Terminal

[1344] The device transmits the captured video data to the server in real time. The video data is compressed before transmission to efficiently transfer it to the server, for example, using H.264 encoding to reduce the data volume. The input data is the captured video data, and the output is compressed video data.

[1345] Step 3: Receiving and saving video data

[1346] Subject: Server

[1347] The server receives the video data sent from the device and stores it in cloud storage. The storage used is a highly reliable cloud service (e.g., Amazon S3). The input data is compressed video data, and the output is stored video data.

[1348] Step 4: Analyzing the video data

[1349] Subject: Server

[1350] The server analyzes the stored video data using an AI module. Specifically, it uses OpenCV and TensorFlow to perform facial recognition and motion analysis. For example, it extracts the faces of people in the video and compares them with a database of wanted criminals. The input data is the stored video data, and the output is the analysis results (e.g., facial recognition results, motion analysis results).

[1351] Step 5: Sentiment analysis and alert generation

[1352] Subject: Server

[1353] The server uses an emotion engine to perform emotion analysis based on the analyzed data. For example, it uses IBM Watson's emotion analysis API to analyze emotions such as "anxiety," "fear," and "elation" from the facial expressions of people in the video. Based on the analysis results, it generates an appropriate alert. The input data is the analysis result, and the output is the generated alert.

[1354] Step 6: Alert notification and response

[1355] Subject: Terminal

[1356] The device receives the alert sent from the server and notifies the police officer. It displays the alert content in a visual interface and issues an immediate warning using audio alerts and the device's vibration function. For example, it displays messages such as "Possible wanted fugitive" or "Anxiety detected." The input data is the generated alert, and the output is the alert content notified to the police officer.

[1357] Step 7: Sending response results and updating the database

[1358] Subject: User (police officer)

[1359] After confirming the notification, the police officer responds at the scene. The police officer inputs the response results (e.g., the results of the police questioning or a situation report) into the terminal and sends the data to the server. The server receives the response results and updates the database. The input data is the response results at the scene, and the output is the updated database information.

[1360] (Application example 2)

[1361] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1362] Conventional security systems simply store camera footage, making it difficult to detect suspicious behavior or emotional changes in real time. As a result, there is a lack of information to make appropriate decisions in situations where a quick response is required. As a result, it is difficult to implement effective security measures to prevent danger.

[1363] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1364] In this invention, the server includes means for receiving video data from cameras carried by security guards, means for saving the received video data, means for analyzing the saved video data, and means including an emotion engine for recognizing the emotions of the target based on the analyzed video data. This makes it possible to detect not only suspicious behavior and abnormalities in real time, but also changes in the target's emotions, allowing for faster and more accurate responses.

[1365] A "security guard" is a person whose job is to monitor facilities and locations, detect abnormalities and suspicious behavior, and respond accordingly.

[1366] A "camera" is a photographing device that captures video data and transmits it to a server in real time.

[1367] "Video data" refers to information about images and videos captured by a camera.

[1368] The "receiving means" is a mechanism by which the server receives the video data transmitted from the camera.

[1369] The "storage means" is a mechanism for temporarily or permanently storing received video data in a server.

[1370] The "analysis means" is a mechanism equipped with an AI module that analyzes stored video data and detects suspicious behavior or abnormalities.

[1371] An "alarm" is an alert generated based on the analysis results and notified to security personnel visually, audibly, or in other ways.

[1372] The "notification means" is a mechanism for transmitting the generated alarm to a security guard.

[1373] "Response results" is information that security guards report on the results of actions they take based on an alarm.

[1374] A "database" is a structured collection of information that stores and makes available past response results and other related information.

[1375] The "emotion engine" is a system with an algorithm for recognizing a subject's emotional state (e.g., anxiety, elation, fear, etc.) based on video data.

[1376] "Location information" refers to information about the current location of security guards obtained using technology such as GPS.

[1377] MODE FOR CARRYING OUT THE INVENTION

[1378] System program for implementing this application example

[1379] This system is realized through the cooperation of smart glasses carried by security guards, a server, and a database. The smart glasses are equipped with a camera that is always on and captures video data in real time. The captured video data is sent to the server via a communications module. The server stores the received video data and analyzes it using an AI module and emotion engine. Based on the analysis results, an alarm is generated and notified to the security guard. The system also receives the guard's response results and updates the database.

[1380] Explain the generated program's processing in natural language

[1381] Server Processing

[1382] 1. Hardware and Software:

[1383] The server is equipped with hardware with high-performance processing capabilities, as well as an AI module and emotion engine for analyzing video data.

[1384] The AI ​​module includes image analysis libraries (e.g., OpenCV), and the emotion engine includes emotion recognition models (e.g., IBM Watson Emotion API).

[1385] The server uses a communication module to receive and automatically store video data sent from the smart glasses.

[1386] 2. Data Analysis:

[1387] The received video data is analyzed in real time to detect suspicious behavior and abnormalities.

[1388] The emotion engine recognizes the subject's emotions and analyzes their emotional state (e.g., anxiety, elation, fear, etc.).

[1389] 3. Generate an alert:

[1390] Generate alerts based on the analysis results.

[1391] The alert will include details such as the type of activity detected and the emotional state.

[1392] 4. Notifications and Database Updates:

[1393] The generated alert is sent to the smart glasses terminal via visual, auditory, and vibration notifications.

[1394] The results of the security guard's response are received and the database is updated.

[1395] Smart Glasses Processing

[1396] 1. Video Capture:

[1397] A camera mounted on the smart glasses captures video data in real time.

[1398] 2. Data transmission:

[1399] The video data is transmitted to the server via the communication module.

[1400] 3. Alarm notification:

[1401] Receives alarm notifications from the server and notifies security guards.

[1402] Notification methods include visual (display on the screen), audible (voice alert), and vibration.

[1403] Database Processing

[1404] 1. Data storage and updating:

[1405] The video data received from the server, analysis results, and security guard response results are stored.

[1406] Add specific examples to the description

[1407] Examples:

[1408] Example 1:

[1409] "While a security guard is patrolling a building in the middle of the night, the smart glasses capture a suspicious person. The server analyzes the video and determines that the person is trying to break into a locked room. The emotion engine detects the person's expression of impatience and immediately issues an alarm."

[1410] Example 2:

[1411] "Smart glasses capture a person behaving suspiciously in the crowd at a train station. The server then analyzes the person and, through profiling, identifies that the person has behaved similarly in the past, and detects that they have a particularly dangerous tendency. The emotion engine then detects signs of tension in the person's facial expressions and sends an immediate notification."

[1412] Example prompt for a generative AI model:

[1413] "Security guards wearing smart glasses capture video in real time and send it to a server. The server then uses an AI module to analyze the video data, and if it detects any suspicious behavior, it notifies the security guard in real time. It also detects emotions and helps the security guard respond appropriately based on the emotional state of the person in question."

[1414] The above is the specific content of the "Mode for Carrying Out the Invention." This demonstrates that the present invention is a system that can identify suspicious behavior and emotional changes in real time, enabling rapid and accurate security responses.

[1415] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1416] Step 1:

[1417] Video Capture

[1418] Description: The camera installed in the smart glasses that serve as the terminal captures video data in real time.

[1419] Input: Video of the guard's surroundings.

[1420] What it does: The camera continuously captures video frames and stores them in a buffer.

[1421] Output: Captured video data.

[1422] Step 2:

[1423] Data transmission

[1424] Description: The device sends captured video data to the server.

[1425] Input: Captured video data.

[1426] Specific operation: Using a communications module, video data is sent to a server via the Internet, often using the HTTP protocol.

[1427] Output: Video data sent to the server.

[1428] Step 3:

[1429] Data reception and storage

[1430] Description: The server receives and stores video data sent from the device.

[1431] Input: Video data sent from the device.

[1432] Specific operation: The communication module receives the data and stores it in local storage or a database.

[1433] Output: Saved video data.

[1434] Step 4:

[1435] Data analysis

[1436] Description: The server analyzes the stored video data to detect suspicious behavior and anomalies. It also analyzes the emotional state using an emotion engine.

[1437] Input: Stored video data.

[1438] How it works: An AI module (e.g., OpenCV) is used to analyze behaviors and facial expressions in the video data. An emotion engine (e.g., IBM Watson Emotion API) evaluates the subject's emotional state.

[1439] Output: Behavioral analysis results and emotion analysis results.

[1440] Step 5:

[1441] Generate an alert

[1442] Description: The server generates an alert based on the analysis results.

[1443] Input: Behavioral analysis results and emotion analysis results.

[1444] Specific behavior: Evaluate the analysis results and generate an alert if certain conditions are met (e.g., detecting suspicious behavior or a specific emotional state). The alert can be generated in multiple ways, including visual, auditory, or vibration.

[1445] Output: Generated alert data.

[1446] Step 6:

[1447] Alert notification

[1448] Description: Notifies the terminal of the alarm generated by the server. The alarm is delivered to the security guard by visual, auditory and vibration means.

[1449] Input: Generated alert data.

[1450] Specific operation: The communication module sends alarm data to the terminal, which displays the received alarm on the screen and alerts the security guard with an alarm sound or vibration.

[1451] Output: Alarm notified to security personnel.

[1452] Step 7:

[1453] Enter and submit the response results

[1454] Description: The user (security guard) responds based on the alarm and inputs the response results into the terminal. The terminal then sends the response results to the server.

[1455] Input: The action result entered by the user.

[1456] Specific operation: The security guard uses the terminal interface to input the response results and sends the data to the server using the communication module.

[1457] Output: The response result sent to the server.

[1458] Step 8:

[1459] Database Update

[1460] Description: The server receives the response and updates the database.

[1461] Input: The response result sent.

[1462] Specific operation: The server adds or updates the response result data it receives to the database and stores it as history, thereby accumulating data that can be used for future analysis.

[1463] Output: The updated database.

[1464] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1465] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1466] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1467] [Fourth embodiment]

[1468] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1469] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1470] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1471] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1472] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1473] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1474] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1475] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1476] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1477] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1478] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1479] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1480] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1481] The present invention relates to a system that analyzes video data in real time in conjunction with a constantly running camera carried by a police officer. This system is composed of the following elements:

[1482] 1. Server

[1483] The server includes hardware and software for receiving and storing video data sent from police officers' terminals. It also includes an AI module that analyzes the stored video data. The server generates alerts based on the analysis results and sends them to the police officers' terminals. It also receives response results from police officers and updates the database.

[1484] 2. Terminal

[1485] The officer's terminal communicates with the camera and captures video data. It is equipped with a communication module that transmits video data to the server in real time. It also notifies officers of received alarms and issues warnings through visual, audible, and vibration alerts. It also provides an interface for officers to input the results of their response and transmits the data to the server.

[1486] 3. User (police officer)

[1487] Police officers use this system when patrolling or questioning police officers. By wearing cameras and devices, video data is sent to a server in real time, and if suspicious behavior or abnormalities are detected, they are immediately notified. Upon receiving the notification, the police officer takes appropriate action according to the situation and updates the database by entering the results into the device.

[1488] As a specific example of the operation, the following scenario can be considered.

[1489] Example 1: Supporting police questioning

[1490] server:

[1491] A police officer's camera captures a person while patrolling a busy street at night.

[1492] The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals.

[1493] If the result of the comparison indicates that there is a high probability that the person is a wanted criminal, the server immediately generates an alert and notifies the terminal.

[1494] Device:

[1495] The device receives the alert and displays the message "Possible wanted criminal" on the screen. The device immediately notifies police officers using audio alerts and vibration functions.

[1496] User (police officer):

[1497] Police officers will check this notification and question the person in question. If they are able to confirm the person's identity and find that the person is in fact wanted, the police officers will immediately arrest the person.

[1498] Example 2: Detecting signs of crime

[1499] server:

[1500] A police officer's camera patrolling a park during the day captures a scene of several young people gathered together and behaving abnormally.

[1501] The server analyzes the acquired video data using an AI module, and if a suspicious behavior pattern is detected, it determines that there is a "sign of a crime," generates an alert, and notifies the device.

[1502] Device:

[1503] The device receives the alarm notification, displays "Suspicious Activity Detected," and notifies the police. The device immediately issues a visual and audio alert.

[1504] User (police officer):

[1505] Police officers will review the notification, proceed to the scene, approach the group of young people in question, and investigate their behavior. If they find any suspicious items, they will conduct further investigations and take necessary action on the spot.

[1506] The above specific example is intended to enable immediate response at the scene of a crime and to prevent and resolve it quickly. This system is expected to improve the accuracy of police officers' judgment, and to increase the efficiency and reliability of police activities.

[1507] The processing flow will be explained below.

[1508] Step 1:

[1509] Device: Camera activation and video capture

[1510] The device activates the camera worn by the officer and begins capturing video data. The camera continues to capture video at 30 frames per second and transmits it to the device.

[1511] Step 2:

[1512] Terminal: Compression and transmission of video data

[1513] The device compresses and encrypts the video data acquired from the camera in real time, and then sends the compressed and encrypted video data to the server.

[1514] Step 3:

[1515] Server: Receiving and temporarily storing video data

[1516] The server receives the video data sent from the device and temporarily stores it, adding GPS information and a timestamp to the received video data.

[1517] Step 4:

[1518] Server: AI video analysis

[1519] An AI module on the server analyzes the temporarily stored video data in real time, using facial recognition and motion analysis algorithms to identify suspicious behavior and wanted criminals.

[1520] Step 5:

[1521] Server: Generate alerts

[1522] Based on the analysis results of the AI ​​module, the server generates an alert if necessary, which includes the analysis results, location information, and a timestamp.

[1523] Step 6:

[1524] Server: Alert notification

[1525] The server sends the generated alert to the terminal, which receives the alert and prepares to notify the police officer.

[1526] Step 7:

[1527] Terminal: Display of alert notification

[1528] The device notifies officers of received alarms visually and audibly, displays detailed information on the alarm on the device's screen, and alerts officers with a warning sound and vibration.

[1529] Step 8:

[1530] User (police officer): Response to alarm

[1531] Police officers will check the alarm on the device and take action according to the situation at the scene, such as questioning the suspect, checking the scene, and in some cases, responding to an emergency.

[1532] Step 9:

[1533] Device: Enter response results

[1534] Police officers use the terminal interface to input the results of their response, including the details of the police stop, any suspicious objects found, and information about the people they have apprehended.

[1535] Step 10:

[1536] Device: Sending response results

[1537] The device sends the inputted response results to the server, where they are encrypted and sent securely to the server.

[1538] Step 11:

[1539] Server:Database Update

[1540] The server receives the correspondence results, updates the database, correlates them with existing video data and analysis results, and stores them for future analysis.

[1541] Through these steps, the system will support police officers in responding to crimes on the scene and improve the efficiency of crime prevention and resolution.

[1542] Example 1

[1543] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1544] In traditional police work, police officers are required to provide fast and accurate information so that they can grasp the situation at the scene and respond immediately. However, identifying suspicious individuals and detecting signs of crime at the scene relies heavily on the experience and intuition of police officers, which can lead to delayed responses and incorrect judgments. Furthermore, delayed feedback of information from the scene can affect database updates and subsequent investigative activities. To solve these issues, a system is needed that allows police officers to grasp the situation at the scene in real time and respond appropriately.

[1545] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1546] In this invention, the server includes means for receiving video data from a camera carried by a police officer, means for saving the received video data, means for using an AI module to analyze the saved video data, means for generating an alert based on the analysis results and notifying the police officer, and means for receiving response results from the police officer and updating the database, thereby enabling the police officer to visually grasp the situation at the scene in real time and respond quickly and accurately based on the analysis results of the AI.

[1547] A "police officer" is a public servant who performs duties to maintain law and order and ensure public safety.

[1548] "Photographic equipment" refers to equipment carried by police officers and used to capture video data at the scene, including, for example, cameras and body cams.

[1549] "Video Data" means information in the form of video captured by a camera that records police officers' activities and the surrounding environment.

[1550] "Means for receiving" refers to the technical mechanism by which the server captures the video data transmitted from the imaging device.

[1551] "Storage means" means the technical mechanism used to safely and efficiently store received video data for a certain period of time, including, for example, a database or cloud storage.

[1552] "Analysing means" means technical mechanisms, such as artificial intelligence modules, used to process stored video data and extract specific information.

[1553] An "artificial intelligence module" is a software or hardware component used to analyze or recognize data, including, for example, machine learning algorithms and deep learning models.

[1554] "Means for generating an alert" means the technical mechanism that generates an alert to notify police officers when an abnormality or suspicious activity is detected based on the analysis results.

[1555] "Means for notifying" refers to the technical mechanism for transmitting the generated alert to the police officer's terminal and conveying the information to the police officer.

[1556] "Means for receiving response results" refers to the technical mechanism by which police officers input the results of their response at the scene and import the results into a server.

[1557] "Means for updating the database" refers to the technical mechanisms for amending and adding to the database in order to keep the data in the entire system up to date based on the results of police officers' responses.

[1558] "Suspicious behavior" refers to behavior that deviates from typical patterns of behavior and is considered to be an indication of a potential crime.

[1559] "Anomaly" means a condition or occurrence that is not observed or expected under normal circumstances.

[1560] "Location information" refers to geographical data about a police officer's current location, such as GPS data.

[1561] The present invention relates to a system that analyzes video data in real time in conjunction with a constantly running camera carried by a police officer. This system is composed of the following elements:

[1562] server

[1563] The server includes hardware and software for receiving and storing video data sent from police officers' devices. The received video data is securely stored using cloud storage (e.g., Amazon S3). The server then analyzes the stored video data using an AI module (e.g., TensorFlow or OpenCV). Specifically, the AI ​​module analyzes video frames and recognizes suspicious behavior or specific individuals. The server also generates alerts based on the analysis results and notifies the police officers' devices of these alerts. It also receives response results from police officers and updates a database (e.g., MySQL or PostgreSQL).

[1564] Terminal

[1565] The officer's terminal is responsible for capturing video data in conjunction with the always-on camera. The terminal is equipped with a high-performance camera module (e.g., Logitech C920). It also has a communication module (e.g., 4G / 5G modem) that transmits video data to the server in real time. Received alerts are notified to the officer using a visual display, an audible voice alert, and a vibration function. An interface (e.g., touchscreen, keyboard) is also provided for the officer to input the response results. This data is then sent to the server.

[1566] User (police officer)

[1567] Police officers use this system when patrolling or questioning police officers. By using a constantly running camera and wearing a terminal, video data is sent to a server in real time, and if suspicious behavior or abnormalities are detected, they are immediately notified. Upon receiving the notification, the police officer responds according to the situation. The results are entered into the terminal and sent to and recorded on the server.

[1568] Specific examples

[1569] Example 1: Supporting police questioning

[1570] server:

[1571] A police officer's camera patrolling a busy downtown area at night captures a person. The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals. If the comparison results in a high probability that the person is a wanted criminal, the server immediately generates an alert and notifies the device.

[1572] Device:

[1573] The device receives the alert and displays the message "Possible wanted criminal" on the screen. The device immediately notifies police officers using audio alerts and vibration functions.

[1574] User (police officer):

[1575] Police officers will check the notification and question the person in question. If they are able to confirm the person's identity and find that the person is in fact wanted, the police officers will immediately arrest the person.

[1576] Example 2: Detecting signs of crime

[1577] server:

[1578] A police officer's camera patrolling a park during the day captures a scene in which several young people are gathered and behaving abnormally. The server analyzes the acquired video data using an AI module, and if a suspicious behavior pattern is detected, it determines that there is a "sign of a crime." An alert is generated and sent to the device.

[1579] Device:

[1580] The device receives the alarm notification, displays "Suspicious Activity Detected," and notifies the police. The device immediately issues a visual and audio alert.

[1581] User (police officer):

[1582] Police officers will review the notification, proceed to the scene, approach the group of young people in question, and investigate their behavior. If they find any suspicious items, they will conduct further investigations and take appropriate action on the spot.

[1583] Prompt Sentence Examples

[1584] 1. Sample prompts for police questioning assistance:

[1585] The server receives real-time video data from police officers' cameras, uses an AI module to recognize the faces of wanted criminals, and compares them with a database. If the criminal is determined to be wanted, an alert is generated and sent to the device.

[1586] 2. Example of a prompt for crime detection:

[1587] The AI ​​module analyzes the video data acquired by the server, and if it detects a suspicious pattern of behavior, it determines that it is a "sign of a crime," generates an alert, and notifies the device.

[1588] This system will enable police officers to grasp the situation on the scene in real time and respond quickly and accurately, which is expected to improve the efficiency and reliability of police activities.

[1589] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1590] Program processing flow and specific explanation

[1591] Step 1: Capture video data

[1592] Device:

[1593] The police officer carries a device that captures video data in real time through a connected always-on camera, and the device is equipped with a high-performance camera module (e.g., Logitech C920).

[1594] input:

[1595] Successive video frames acquired from a camera.

[1596] output:

[1597] Captured video data.

[1598] Specific behavior:

[1599] As officers walk around on patrol, the device continuously captures video frames and temporarily stores the video data in its internal memory.

[1600] Step 2: Sending video data

[1601] Device:

[1602] The device transmits the captured video data to the server in real time using a 4G / 5G modem.

[1603] input:

[1604] Captured video data.

[1605] output:

[1606] Video data sent to the server.

[1607] Specific behavior:

[1608] The video data is compressed using technology to reduce the amount of data, and then sent to the server via high-speed communication.

[1609] Step 3: Receiving and saving video data

[1610] server:

[1611] The server stores the received video data, which is then securely transferred to cloud storage (e.g., Amazon S3).

[1612] input:

[1613] Video data sent from the device.

[1614] output:

[1615] Securely stored video data.

[1616] Specific behavior:

[1617] The server checks the accuracy and completeness of the data, timestamps it, and stores it in cloud storage.

[1618] Step 4: Analyzing the video data

[1619] server:

[1620] The stored video data is analyzed by an AI module (e.g., TensorFlow or OpenCV) on the server, specifically for person recognition and behavioral pattern analysis.

[1621] input:

[1622] Saved video data.

[1623] output:

[1624] Analysis results (e.g., detection of specific individuals or suspicious behavior).

[1625] Specific behavior:

[1626] AI models process video frames sequentially to detect suspicious behavior or specific individuals, and facial recognition algorithms match them to wanted criminals.

[1627] Step 5: Alert generation and notification

[1628] server:

[1629] Based on the analysis results, an alert is generated as needed, including the detection of wanted criminals and signs of crime, and the generated alert is sent to the device.

[1630] input:

[1631] Analysis results by AI module.

[1632] output:

[1633] Alert notifications sent to your device.

[1634] Specific behavior:

[1635] The server constructs an alert message and sends it to the device via a REST API.

[1636] Step 6: Check the warning and enter the response results

[1637] Device:

[1638] The device notifies the police officer of the alarm received from the server using a visual display, an audible voice alert, and a vibration function.

[1639] input:

[1640] Alert notifications sent from the server.

[1641] output:

[1642] Input of the alert message and response results that will be displayed to the police officer.

[1643] Specific behavior:

[1644] The device displays warning messages such as "Possible wanted fugitive" or "Suspicious behavior detected" and alerts the police officer with beeps and vibrations, who then enters the corresponding results into the device.

[1645] Step 7: Update data

[1646] server:

[1647] The server receives the response results from the police officers and updates the database, which is managed by MySQL or PostgreSQL.

[1648] input:

[1649] Response result data sent from the device.

[1650] output:

[1651] Updated database.

[1652] Specific behavior:

[1653] The entered response result data is sent to the server and immediately saved and updated in the database, allowing the latest status to be always available for reference throughout the system.

[1654] (Application example 1)

[1655] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1656] Modern police activities require rapid and accurate information gathering and situational assessment. However, conventional systems take time to detect suspicious behavior or wanted criminals, making it difficult for police officers to respond quickly on the scene. In addition, there are limited means to visually confirm this information, which places a heavy burden on police officers and can hinder efficient police activities. For these reasons, there is a need for a system that can obtain information in real time and respond quickly and accurately.

[1657] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1658] In this invention, the server includes means for receiving video data from cameras carried by police officers, means for saving the received video data, means for analyzing the saved video data, means for generating an alert based on the analysis results and notifying the police officers, means for receiving response results from the police officers and updating the database, and means for visually and audibly notifying the police officers via the smart glasses. This allows police officers to obtain information in real time at the scene and respond quickly and accurately. The use of the smart glasses allows for visual and audible confirmation of alerts, improving the efficiency and safety of police activities.

[1659] A "camera" is a photographic device carried by a police officer to capture video data at the scene.

[1660] "Video data" is digital data containing visual information captured by a camera.

[1661] The "receiving means" refers to a communication module and software for acquiring the video data transmitted from the camera.

[1662] The "means for storing" refers to data storage and associated software for holding the received video data in a storage device.

[1663] The "analysis means" refers to AI modules and algorithms that analyze stored video data and detect suspicious behavior or anomalies.

[1664] The "means for generating and notifying alerts" refers to a notification system that creates alerts based on the analysis results and notifies police officers.

[1665] The "means for receiving response results and updating the database" refers to a communication module and data processing system for capturing the response status from police officers and reflecting that information in the database.

[1666] "Smart glasses" are augmented reality (AR) devices that are equipped with cameras, displays, and communication functions and are worn by police officers.

[1667] "Visual and audible notification means" refers to the ability to display or audibly communicate warning information through the smart glasses' display or speaker.

[1668] This invention relates to a security system that works in conjunction with cameras carried by police officers, analyzes video data in real time, and issues an immediate alarm. This system is composed of the following elements to enable rapid information gathering and response in police activities:

[1669] System Configuration

[1670] 1. Server

[1671] The server includes hardware and software that receives and stores video data sent from police officers' cameras. It also contains an AI module (using TensorFlow) that analyzes the video data to detect suspicious behavior or individuals.

[1672] It generates alerts based on the analysis results and sends them to police officers' devices or smart glasses in real time. It also receives response results from police officers and updates the database.

[1673] 2. Terminal (smart glasses)

[1674] The smart glasses are equipped with a camera that captures video data in real time, which is then transmitted to a server using WebSocket technology.

[1675] The smart glasses are equipped with an AR display that visually and audibly notifies police officers of warnings based on the analysis results. Police officers can then grasp the situation at the scene through the smart glasses and take necessary action.

[1676] 3. User (police officer)

[1677] Police officers wear smart glasses and patrol while capturing video data. When an alarm is notified, they quickly check the situation at the scene and report the results of their response to the server from the smart glasses.

[1678] Program processing and data calculation

[1679] 1. Camera video capture

[1680] The camera installed in the smart glasses captures video data and transmits it to a server in real time.

[1681] 2. Analysis of video data

[1682] The server analyzes the received video data using an AI module powered by TensorFlow, which detects suspicious behavior and people with high accuracy.

[1683] 3. Alert Generation and Notification

[1684] Based on the analysis results, the server generates an alert and notifies the police officers through the smart glasses' AR display and speaker, allowing them to understand the situation in real time.

[1685] 4. Reporting the results of the response

[1686] Police officers send the results of their response from the smart glasses to a server, which then updates the database with the information.

[1687] Specific examples

[1688] For example, while a police officer is patrolling a park, a camera mounted on a pair of smart glasses captures footage of a group of young people. The video data is sent to the server, where it is analyzed by an AI module and any suspicious behavior is detected. The server immediately generates an alert and notifies the smart glasses that "suspicious behavior has been detected." Police officers then use this information to investigate the scene and take any necessary action.

[1689] Prompt Sentence Examples

[1690] "In the security field, I am developing a system that analyzes real-time video data captured by smart glasses to detect suspicious behavior and wanted criminals. In this system, the server analyzes the video data, generates an alarm, and displays it on the smart glasses. Specifically, could you please tell me what kind of code I should implement?"

[1691] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1692] Step 1:

[1693] The camera in the smart glasses captures the surrounding video data. The input is live video of the scene, and the output is digitized video data. The camera captures video in real time and transmits the data to the built-in communication module.

[1694] Step 2:

[1695] The communication module of the smart glasses transmits the captured video data to the server. The input is the captured video data, and the output is the data to be sent to the server. WebSocket technology is used to transfer the data to the server with low latency.

[1696] Step 3:

[1697] The server stores the received video data. The input is the video data sent from the smart glasses, and the output is the data stored in the storage. The server records the data in a database and keeps it for subsequent analysis.

[1698] Step 4:

[1699] The server analyzes the stored video data. The input is the stored video data, and the output is the analysis results. An AI module (using TensorFlow) extracts and analyzes suspicious behavior and people from the video data.

[1700] Step 5:

[1701] The server generates an alert based on the analysis results. The input is the analysis results and the output is the alert data. If suspicious behavior or people are detected, an alert is immediately created and its contents are defined.

[1702] Step 6:

[1703] The server generates an alarm and notifies the smart glasses. The input is the alarm data, and the output is a visual and audible alarm notification. The alarm is notified to police officers using the AR display and speaker of the smart glasses.

[1704] Step 7:

[1705] The user (police officer) checks the alarm and takes appropriate action at the scene. The input is the alarm notification from the smart glasses, and the output is the response result at the scene. The police officer checks the notification and takes the necessary action on the spot (e.g., questioning, checking the scene).

[1706] Step 8:

[1707] The user (police officer) reports the response result to the server from the smart glasses. The input is the response result, and the output is the response data sent to the server. The result is entered through the input UI of the smart glasses and sent to the server.

[1708] Step 9:

[1709] The server reflects the received response results in the database. The input is the reported response results, and the output is the updated database. The server records the response results in the database, making it easier to understand the overall situation.

[1710] The above are the specific processing steps of this system.

[1711] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1712] This invention relates to a system that combines a system that analyzes video data in real time in conjunction with a constantly running camera carried by police officers, with an emotion engine that recognizes the user's emotions. This system is composed of the following elements:

[1713] 1. Server

[1714] The server includes hardware and software for receiving and storing video data sent from police officers' devices. It also includes an AI module and emotion engine for analyzing the stored video data. The server generates alerts based on the analysis results and sends them to the police officers' devices. It also receives response results from police officers and updates the database.

[1715] 2. Terminal

[1716] The officer's terminal communicates with the camera and captures video data. It is equipped with a communication module that transmits video data to the server in real time. It also notifies officers of received alarms and issues warnings through visual, audible, and vibration alerts. It also provides an interface for officers to input the results of their response and transmits the data to the server.

[1717] 3. User (police officer)

[1718] Police officers use this system when patrolling and questioning police officers. By wearing a camera and a device, video data is sent to a server in real time, and if suspicious behavior or abnormalities are detected, they are immediately notified. An emotion engine is also added to recognize the user's emotions, allowing for more accurate responses according to the situation. After confirming the notification, the police officer takes action according to the situation and updates the database by entering the results into the device.

[1719] As a specific example of the operation, the following scenario can be considered.

[1720] Example 1: Police interrogation assistance and emotion recognition

[1721] server:

[1722] A police officer's camera captures a person while patrolling a busy street at night.

[1723] The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals.

[1724] As a result of the match, it is determined that there is a high probability that the person is a wanted criminal, and the emotion engine analyzes the person's emotions and detects signs of anxiety or fear.

[1725] The server immediately generates an alert and notifies the terminal.

[1726] Device:

[1727] The device receives the alert and displays the message "Possible wanted criminal" along with the emotion analysis results on the screen. It indicates strong emotions of "anxiety" and "fear." The device immediately notifies police officers using a voice alert and vibration function.

[1728] User (police officer):

[1729] Police officers will check this notification and question the person in question. They will proceed with questioning carefully, taking into consideration the results of the emotion analysis. If the person's identity is confirmed and they are in fact wanted, the police officer will immediately arrest the person.

[1730] Example 2: Detecting signs of crime and emotional changes

[1731] server:

[1732] A police officer's camera patrolling a park during the day captures a scene of several young people gathered together and behaving abnormally.

[1733] The server uses an AI module to analyze the acquired video data and detect suspicious behavioral patterns. The emotion engine also analyzes the emotions of young people and detects signs of "excitement" or "agitation." An alert is generated and sent to the device.

[1734] Device:

[1735] The device receives an alarm notification, displays "Suspicious Activity Detected," and also displays the young people's emotional states of "Excitement" and "Arousal." The device immediately issues visual and audio alerts.

[1736] User (police officer):

[1737] Police officers can view the notification, proceed to the scene, approach the group of youths, and observe their behavior. They can also monitor emotional changes and prepare in advance. According to an embodiment of the present invention, police officers can respond not only to suspicious behavior but also to emotional changes in the subjects, allowing them to perform their duties more safely and effectively.

[1738] The processing flow will be explained below.

[1739] Step 1:

[1740] Device: Camera activation and video capture

[1741] The device activates the camera worn by the officer and begins capturing video data. The camera continues to capture video at 30 frames per second and transmits it to the device.

[1742] Step 2:

[1743] Terminal: Compression and transmission of video data

[1744] The device compresses and encrypts the video data acquired from the camera in real time, and then sends the compressed and encrypted video data to the server.

[1745] Step 3:

[1746] Server: Receiving and temporarily storing video data

[1747] The server receives the video data sent from the device and temporarily stores it, adding GPS information and a timestamp to the received video data.

[1748] Step 4:

[1749] Server: AI video analysis

[1750] An AI module on the server analyzes the temporarily stored video data in real time, using facial recognition and motion analysis algorithms to identify suspicious behavior and wanted criminals.

[1751] Step 5:

[1752] Server: Emotion analysis using emotion engine

[1753] In parallel with the AI ​​module's analysis, the server's emotion engine analyzes the subject's facial expressions from the video data and identifies emotions, detecting emotions such as "anxiety," "fear," "elation," and "excitement" in real time.

[1754] Step 6:

[1755] Server: Generate alerts

[1756] Based on the analysis results (AI behavior analysis and emotion analysis by the emotion engine), the server generates an alert if necessary. The alert includes the analysis results, location information, timestamp, and emotion analysis results.

[1757] Step 7:

[1758] Server: Alert notification

[1759] The server sends the generated alert to the terminal, which receives the alert and prepares to notify the police officer.

[1760] Step 8:

[1761] Terminal: Display of alert notification

[1762] The device notifies officers of received alarms visually and audibly. Detailed information about the alarm and the results of emotion analysis are displayed on the device screen, and the device alerts officers with a warning sound and vibration.

[1763] Step 9:

[1764] User (police officer): Response to alarm

[1765] Police officers check the alarms on the device and take action according to the situation at the scene, such as questioning the suspect, investigating the scene, and responding to emergencies if necessary. Based on the results of emotion analysis, the response policy can be adjusted more appropriately.

[1766] Step 10:

[1767] Device: Enter response results

[1768] Police officers use the terminal interface to input the results of their response, including the details of the police stop, any suspicious objects found, and information about the people they have apprehended.

[1769] Step 11:

[1770] Device: Sending response results

[1771] The device sends the inputted response results to the server, where they are encrypted and sent securely to the server.

[1772] Step 12:

[1773] Server:Database Update

[1774] The server receives the correspondence results, updates the database, correlates them with existing video data and analysis results, and stores them for future analysis.

[1775] Through these steps, this system will support police officers in responding to crime scenes and improve the efficiency of crime prevention and resolution. The addition of an emotion engine will enable responses that take into account the emotional changes of the target, further improving reliability.

[1776] Example 2

[1777] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1778] When police officers are on patrol or conducting routine questioning, it is extremely important to quickly and accurately grasp the suspicious behavior and emotional state of the subject. However, with conventional systems, it is difficult to analyze video data in real time and determine emotions, making it difficult for police officers to respond appropriately immediately. In addition, there are limited methods for issuing alarm notifications, making it difficult for police officers to respond immediately.

[1779] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1780] In this invention, the server includes means for receiving video data from a camera carried by a police officer, means for saving the received video data, means for analyzing the saved video data, means for generating an alert based on the analysis results and notifying the police officer, means for receiving response results from the police officer and updating the database, means for analyzing the emotional state from the video data and generating an alert based on the analysis results, and means for notifying the police officer of the alert visually, audibly, and by vibration. This makes it possible to detect suspicious behavior and emotional states in real time and notify the police officer quickly and reliably.

[1781] A "camera carried by a police officer" is a small, continuously operational video camera that is worn on the body by a police officer when patrolling or conducting routine questioning.

[1782] The "means for receiving video data" is a set of hardware and software for receiving real-time video data transmitted from a camera.

[1783] The "means for storing received video data" is a data storage system for safely and efficiently storing received video data.

[1784] "Means for analyzing stored video data" refers to AI modules and analytical software for performing facial recognition and behavioral analysis on stored video data.

[1785] The "means for generating an alert based on the analysis results and notifying police officers" is a system for creating an alert based on the analyzed data and notifying the police officers' terminals.

[1786] The "means for receiving response results from police officers and updating the database" is a system for receiving information entered by police officers after responding to an incident at the scene and keeping the database up to date.

[1787] The "means for analyzing the emotional state from video data and generating an alert based on the analysis results" refers to an emotion recognition engine that analyzes the emotional state of people in the video and generates an appropriate alert based on the results.

[1788] "Means for notifying police officers of an alarm visually, audibly, and by vibration" means a notification system that uses a screen display, audio alerts, and the device's vibration function to quickly communicate an alarm to police officers.

[1789] "Means for analyzing in real time and detecting suspicious behavior and anomalies" refers to an AI-based analysis module that processes video data in real time and instantly detects abnormal or suspicious behavior.

[1790] "Means for obtaining location information of police officers on patrol and synchronizing it with video data" refers to a system for obtaining the current location of police officers using a location information system such as GPS and synchronizing it with video data.

[1791] This invention is a system that uses a constantly running camera and a terminal carried by a police officer to transmit video data to a server in real time and analyzes the video data. This system is equipped with an AI module and an emotion engine, and can detect suspicious behavior and emotional states.

[1792] Hardware and software used

[1793] 1. Camera: A high-resolution video camera (with infrared functionality) is used, which allows for clear image data to be acquired.

[1794] 2. Terminal: A mobile terminal equipped with an LTE or Wi-Fi communication module that transmits video data to a server in real time.

[1795] 3. Server: A high-performance server equipped with cloud storage (e.g., Amazon S3) and AI modules (e.g., OpenCV, TensorFlow). It also uses an emotion analysis engine (e.g., IBM Watson) to analyze the emotional state from video data.

[1796] Program processing

[1797] server

[1798] The server receives video data transmitted in real time from the device and stores it in data storage. It uses an AI module to analyze the stored data and compare it with a database of wanted criminals. It also uses an emotion engine to analyze the emotional state of people in the video and generates an alert based on the analysis results. The alert is transmitted via a system that notifies police officers with visual and audio alerts and vibrations.

[1799] Terminal

[1800] The device captures video data from the always-on camera and transmits it to the server in real time. It displays received alarms visually and notifies officers with audio alerts and vibrations. Officers enter their response results into the device, which then transmits the data to the server, updating the database.

[1801] User (police officer)

[1802] Police officers wear cameras and devices when patrolling and questioning people. They respond based on notifications from the system and enter the results of their actions into the device. This keeps the database updated with the latest information, enabling them to carry out their duties safely and effectively.

[1803] Specific examples

[1804] Example 1: Police interrogation assistance and emotion recognition

[1805] Server: A police officer patrolling a busy downtown area at night captures a person on camera. The server receives this video data, uses an AI module to recognize the person's face, and compares it with a database of wanted criminals. As a result of the comparison, it determines that there is a high probability that the person is a wanted criminal, and an emotion engine analyzes the person's emotions and detects signs of anxiety or fear. The server immediately generates an alert stating "possibly wanted criminal" or "anxiety detected" and notifies the device.

[1806] Device: The device receives the alarm notification and displays "Possible wanted criminal" and "Anxiety detected." It immediately notifies the police with an audio alert and vibration.

[1807] User (police officer): The police officer checks the notification and begins to carefully question the suspect. The police officer enters the results into the terminal and sends them to the server to update the database.

[1808] Prompt Sentence Examples

[1809] 1. "While patrolling downtown, the camera captured a person who appears to be a wanted criminal. Please run the AI ​​module to match the person and simultaneously perform sentiment analysis."

[1810] 2. "When conducting a police stop, please demonstrate how to respond if the person's emotions indicate anxiety or fear."

[1811] Example 2: Detecting signs of crime and emotional changes

[1812] Server: Captures a scene of several young people gathering in a park during the day and behaving abnormally. The server analyzes the video data and detects suspicious behavior. The emotion engine also analyzes signs of "excitement" or "fever." The server generates an alert for "suspicious behavior detected" or "excitement detected" and notifies the device.

[1813] Terminal: The terminal receives the alarm notification and displays "Suspicious behavior detected" or "Elevation detected." It immediately notifies the police officer with visual and audio alerts.

[1814] User (police officer): The police officer checks the notification, approaches the group of young people in question, and observes their behavior. He or she takes appropriate action depending on the situation, paying attention to any emotional changes. The results are entered into the terminal and sent to the server to update the database.

[1815] Prompt Sentence Examples

[1816] 1. "Analyze footage of several young people behaving suspiciously in a park and use the emotion engine to detect their emotional state."

[1817] 2. "Please provide procedures for police officers to safely respond to situations involving a group of adolescents who are agitated."

[1818] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1819] Program processing flow

[1820] Step 1: Capture video data

[1821] Subject: Terminal

[1822] The terminal communicates with a constantly running camera carried by the police officer and captures video data in real time. The camera records video in high resolution and transmits the video data to the terminal. Specifically, the camera is attached to the officer's breast pocket and constantly records video of the surrounding area. The input data is the video of the surrounding area, and the output is the captured video data.

[1823] Step 2: Sending video data

[1824] Subject: Terminal

[1825] The device transmits the captured video data to the server in real time. The video data is compressed before transmission to efficiently transfer it to the server, for example, using H.264 encoding to reduce the data volume. The input data is the captured video data, and the output is compressed video data.

[1826] Step 3: Receiving and saving video data

[1827] Subject: Server

[1828] The server receives the video data sent from the device and stores it in cloud storage. The storage used is a highly reliable cloud service (e.g., Amazon S3). The input data is compressed video data, and the output is stored video data.

[1829] Step 4: Analyzing the video data

[1830] Subject: Server

[1831] The server analyzes the stored video data using an AI module. Specifically, it uses OpenCV and TensorFlow to perform facial recognition and motion analysis. For example, it extracts the faces of people in the video and compares them with a database of wanted criminals. The input data is the stored video data, and the output is the analysis results (e.g., facial recognition results, motion analysis results).

[1832] Step 5: Sentiment analysis and alert generation

[1833] Subject: Server

[1834] The server uses an emotion engine to perform emotion analysis based on the analyzed data. For example, it uses IBM Watson's emotion analysis API to analyze emotions such as "anxiety," "fear," and "elation" from the facial expressions of people in the video. Based on the analysis results, it generates an appropriate alert. The input data is the analysis result, and the output is the generated alert.

[1835] Step 6: Alert notification and response

[1836] Subject: Terminal

[1837] The device receives the alert sent from the server and notifies the police officer. It displays the alert content in a visual interface and issues an immediate warning using audio alerts and the device's vibration function. For example, it displays messages such as "Possible wanted fugitive" or "Anxiety detected." The input data is the generated alert, and the output is the alert content notified to the police officer.

[1838] Step 7: Sending response results and updating the database

[1839] Subject: User (police officer)

[1840] After confirming the notification, the police officer responds at the scene. The police officer inputs the response results (e.g., the results of the police questioning or a situation report) into the terminal and sends the data to the server. The server receives the response results and updates the database. The input data is the response results at the scene, and the output is the updated database information.

[1841] (Application example 2)

[1842] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1843] Conventional security systems simply store camera footage, making it difficult to detect suspicious behavior or emotional changes in real time. As a result, there is a lack of information to make appropriate decisions in situations where a quick response is required. As a result, it is difficult to implement effective security measures to prevent danger.

[1844] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1845] In this invention, the server includes means for receiving video data from cameras carried by security guards, means for saving the received video data, means for analyzing the saved video data, and means including an emotion engine for recognizing the emotions of the target based on the analyzed video data. This makes it possible to detect not only suspicious behavior and abnormalities in real time, but also changes in the target's emotions, allowing for faster and more accurate responses.

[1846] A "security guard" is a person whose job is to monitor facilities and locations, detect abnormalities and suspicious behavior, and respond accordingly.

[1847] A "camera" is a photographing device that captures video data and transmits it to a server in real time.

[1848] "Video data" refers to information about images and videos captured by a camera.

[1849] The "receiving means" is a mechanism by which the server receives the video data transmitted from the camera.

[1850] The "storage means" is a mechanism for temporarily or permanently storing received video data in a server.

[1851] The "analysis means" is a mechanism equipped with an AI module that analyzes stored video data and detects suspicious behavior or abnormalities.

[1852] An "alarm" is an alert generated based on the analysis results and notified to security personnel visually, audibly, or in other ways.

[1853] The "notification means" is a mechanism for transmitting the generated alarm to a security guard.

[1854] "Response results" is information that security guards report on the results of actions they take based on an alarm.

[1855] A "database" is a structured collection of information that stores and makes available past response results and other related information.

[1856] The "emotion engine" is a system with an algorithm for recognizing a subject's emotional state (e.g., anxiety, elation, fear, etc.) based on video data.

[1857] "Location information" refers to information about the current location of security guards obtained using technology such as GPS.

[1858] MODE FOR CARRYING OUT THE INVENTION

[1859] System program for implementing this application example

[1860] This system is realized through the cooperation of smart glasses carried by security guards, a server, and a database. The smart glasses are equipped with a camera that is always on and captures video data in real time. The captured video data is sent to the server via a communications module. The server stores the received video data and analyzes it using an AI module and emotion engine. Based on the analysis results, an alarm is generated and notified to the security guard. The system also receives the guard's response results and updates the database.

[1861] Explain the generated program's processing in natural language

[1862] Server Processing

[1863] 1. Hardware and Software:

[1864] The server is equipped with hardware with high-performance processing capabilities, as well as an AI module and emotion engine for analyzing video data.

[1865] The AI ​​module includes image analysis libraries (e.g., OpenCV), and the emotion engine includes emotion recognition models (e.g., IBM Watson Emotion API).

[1866] The server uses a communication module to receive and automatically store video data sent from the smart glasses.

[1867] 2. Data Analysis:

[1868] The received video data is analyzed in real time to detect suspicious behavior and abnormalities.

[1869] The emotion engine recognizes the subject's emotions and analyzes their emotional state (e.g., anxiety, elation, fear, etc.).

[1870] 3. Generate an alert:

[1871] Generate alerts based on the analysis results.

[1872] The alert will include details such as the type of activity detected and the emotional state.

[1873] 4. Notifications and Database Updates:

[1874] The generated alert is sent to the smart glasses terminal via visual, auditory, and vibration notifications.

[1875] The results of the security guard's response are received and the database is updated.

[1876] Smart Glasses Processing

[1877] 1. Video Capture:

[1878] A camera mounted on the smart glasses captures video data in real time.

[1879] 2. Data transmission:

[1880] The video data is transmitted to the server via the communication module.

[1881] 3. Alarm notification:

[1882] Receives alarm notifications from the server and notifies security guards.

[1883] Notification methods include visual (display on the screen), audible (voice alert), and vibration.

[1884] Database Processing

[1885] 1. Data storage and updating:

[1886] The video data received from the server, analysis results, and security guard response results are stored.

[1887] Add specific examples to the description

[1888] Examples:

[1889] Example 1:

[1890] "While a security guard is patrolling a building in the middle of the night, the smart glasses capture a suspicious person. The server analyzes the video and determines that the person is trying to break into a locked room. The emotion engine detects the person's expression of impatience and immediately issues an alarm."

[1891] Example 2:

[1892] "Smart glasses capture a person behaving suspiciously in the crowd at a train station. The server then analyzes the person and, through profiling, identifies that the person has behaved similarly in the past, and detects that they have a particularly dangerous tendency. The emotion engine then detects signs of tension in the person's facial expressions and sends an immediate notification."

[1893] Example prompt for a generative AI model:

[1894] "Security guards wearing smart glasses capture video in real time and send it to a server. The server then uses an AI module to analyze the video data, and if it detects any suspicious behavior, it notifies the security guard in real time. It also detects emotions and helps the security guard respond appropriately based on the emotional state of the person in question."

[1895] The above is the specific content of the "Mode for Carrying Out the Invention." This demonstrates that the present invention is a system that can identify suspicious behavior and emotional changes in real time, enabling rapid and accurate security responses.

[1896] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1897] Step 1:

[1898] Video Capture

[1899] Description: The camera installed in the smart glasses that serve as the terminal captures video data in real time.

[1900] Input: Video of the guard's surroundings.

[1901] What it does: The camera continuously captures video frames and stores them in a buffer.

[1902] Output: Captured video data.

[1903] Step 2:

[1904] Data transmission

[1905] Description: The device sends captured video data to the server.

[1906] Input: Captured video data.

[1907] Specific operation: Using a communication module, the video data is sent to a server via the Internet, often using the HTTP protocol.

[1908] Output: Video data sent to the server.

[1909] Step 3:

[1910] Data reception and storage

[1911] Description: The server receives and stores video data sent from the device.

[1912] Input: Video data sent from the device.

[1913] Specific operation: The communication module receives the data and stores it in local storage or a database.

[1914] Output: Saved video data.

[1915] Step 4:

[1916] Data analysis

[1917] Description: The server analyzes the stored video data to detect suspicious behavior and anomalies. It also analyzes the emotional state using an emotion engine.

[1918] Input: Stored video data.

[1919] How it works: An AI module (e.g., OpenCV) is used to analyze behaviors and facial expressions in the video data. An emotion engine (e.g., IBM Watson Emotion API) evaluates the subject's emotional state.

[1920] Output: Behavioral analysis results and emotion analysis results.

[1921] Step 5:

[1922] Generate an alert

[1923] Description: The server generates an alert based on the analysis results.

[1924] Input: Behavioral analysis results and emotion analysis results.

[1925] Specific behavior: Evaluate the analysis results and generate an alert if certain conditions are met (e.g., detecting suspicious behavior or a specific emotional state). The alert can be generated in multiple ways, including visual, auditory, or vibration.

[1926] Output: Generated alert data.

[1927] Step 6:

[1928] Alert notification

[1929] Description: Notifies the terminal of the alarm generated by the server. The alarm is delivered to the security guard by visual, auditory and vibration means.

[1930] Input: Generated alert data.

[1931] Specific operation: The communication module sends alarm data to the terminal, which displays the received alarm on the screen and alerts the security guard with an alarm sound or vibration.

[1932] Output: Alarm notified to security personnel.

[1933] Step 7:

[1934] Enter and submit the response results

[1935] Description: The user (security guard) responds based on the alarm and inputs the response results into the terminal. The terminal then sends the response results to the server.

[1936] Input: The action result entered by the user.

[1937] Specific operation: The security guard uses the terminal interface to input the response results and sends the data to the server using the communication module.

[1938] Output: The response result sent to the server.

[1939] Step 8:

[1940] Database Update

[1941] Description: The server receives the response and updates the database.

[1942] Input: The response result sent.

[1943] Specific operation: The server adds or updates the response result data it receives to the database and stores it as history, thereby accumulating data that can be used for future analysis.

[1944] Output: The updated database.

[1945] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1946] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1947] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1948] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1949] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1950] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1951] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1952] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1953] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1954] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1955] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1956] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1957] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1958] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1959] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1960] The hardware resource for executing a specific process can be any of the following processors: A CPU is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A dedicated electrical circuit, such as a field-programmable gate array (FPGA), a programmable logic device (PLD), or an application-specific integrated circuit (ASIC), is a processor with a circuit configuration specifically designed to execute a specific process. Each processor has built-in or connected memory, and uses the memory to execute the specific process.

[1961] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1962] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1963] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1964] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1965] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1966] The following is further disclosed regarding the above embodiment.

[1967] (Claim 1)

[1968] means for receiving video data from a camera carried by a police officer;

[1969] means for storing the received video data;

[1970] means for analyzing the stored video data;

[1971] A means for generating an alert based on the analysis results and notifying police officers;

[1972] a means for receiving the response results from the police officers and updating the database;

[1973] A system including:

[1974] (Claim 2)

[1975] 10. The system according to claim 1, further comprising means for analyzing the received video data in real time to detect suspicious behavior or abnormalities.

[1976] (Claim 3)

[1977] 10. The system of claim 1, further comprising means for acquiring and synchronizing GPS information of police officers on patrol with the video data.

[1978] "Example 1"

[1979] (Claim 1)

[1980] means for receiving video data from a photographing device carried by a police officer;

[1981] means for storing the received video data;

[1982] using an artificial intelligence module to analyze the stored video data;

[1983] A means for generating an alert based on the analysis results and notifying police officers;

[1984] a means for receiving the response results from the police officers and updating the database;

[1985] A system including:

[1986] (Claim 2)

[1987] 10. The system according to claim 1, further comprising means for analyzing the received video data in real time to detect suspicious behavior or abnormalities.

[1988] (Claim 3)

[1989] 10. The system of claim 1, further comprising means for acquiring and synchronizing location information of police officers on patrol with the video data.

[1990] "Application Example 1"

[1991] (Claim 1)

[1992] means for receiving video data from a camera carried by a police officer;

[1993] means for storing the received video data;

[1994] means for analyzing the stored video data;

[1995] A means for generating an alert based on the analysis results and notifying police officers;

[1996] a means for receiving the response results from the police officers and updating the database;

[1997] a means of visually and audibly notifying the police officer via the smart glasses;

[1998] A system including:

[1999] (Claim 2)

[2000] A means of analyzing received video data in real time to detect suspicious behavior and abnormalities,

[2001] 10. The system of claim 1, further comprising means for displaying the alert on the smart glasses in real time.

[2002] (Claim 3)

[2003] A means for acquiring location information of police officers on patrol and synchronizing it with video data;

[2004] 10. The system of claim 1, further comprising means for optimizing notification of an alert based on the obtained location information.

[2005] "Example 2: Combining Emotion Engines"

[2006] (Claim 1)

[2007] means for receiving video data from a camera carried by a police officer;

[2008] means for storing the received video data;

[2009] means for analyzing the stored video data;

[2010] A means for generating an alert based on the analysis results and notifying police officers;

[2011] a means for receiving the response results from the police officers and updating the database;

[2012] means for analyzing an emotional state from video data and generating an alert based on the analysis result;

[2013] means for notifying police officers of the alarm by visual, audible and vibration signals;

[2014] A system including:

[2015] (Claim 2)

[2016] 10. The system according to claim 1, further comprising means for analyzing the received video data in real time to detect suspicious behavior or abnormalities.

[2017] (Claim 3)

[2018] 10. The system of claim 1, further comprising means for acquiring and synchronizing location information of police officers on patrol with the video data.

[2019] "Application example 2 when combining emotion engines"

[2020] (Claim 1)

[2021] means for receiving video data from a camera carried by a security guard;

[2022] means for storing the received video data;

[2023] means for analyzing the stored video data;

[2024] A means for generating an alarm based on the analysis results and notifying security personnel;

[2025] a means for receiving the response results from the security guards and updating the database;

[2026] A means to send the video data captured by the camera to a server and analyze it in real time,

[2027] a server including an emotion engine that recognizes the emotion of a subject based on the analyzed video data;

[2028] means for generating and notifying an alert using the emotion analysis result detected by the emotion engine;

[2029] A system including:

[2030] (Claim 2)

[2031] 2. The system of claim 1, further comprising: means for analyzing the received video data in real time to detect suspicious behavior or abnormalities; and means for analyzing the emotional state of the subject using an emotion engine.

[2032] (Claim 3)

[2033] 10. The system according to claim 1, further comprising means for acquiring and synchronizing location information of patrolling security personnel with the video data. [Explanation of symbols]

[2034] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving video data from a camera carried by a police officer; means for storing the received video data; means for analyzing the stored video data; A means for generating an alert based on the analysis results and notifying police officers; a means for receiving the response results from the police officers and updating the database; A system including:

2. 2. The system according to claim 1, further comprising means for analyzing the received video data in real time to detect suspicious behavior or abnormalities.

3. 10. The system of claim 1, further comprising means for acquiring and synchronizing GPS information of police officers on patrol with the video data.

Images