system

Abstract

We provide the system. [Solution] A monitoring device is provided, and means are used to acquire video information within the monitoring area using image analysis means and to detect abnormal operation. A control unit that issues warning information when an abnormality is detected, A means for periodically acquiring the position information of an object using a position confirmation device and storing it in an information collection, A means of using an object detection model to analyze video footage within a surveillance area in real time and identify suspicious activity, A means of transmitting warning information to an information exchange device using a communication system, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, the theft of luxury cars has been increasing, especially the crimes committed by non - professional groups are prominent. In such a situation, due to the insufficient security in parking lots, it has become difficult to prevent vehicle theft. Also, it is difficult to quickly track and recover the vehicle after theft occurs. As a result, the demand for parking lots where vehicle owners can use with confidence is increasing. Therefore, it is an object to provide a parking lot management system with advanced security functions.

Means for Solving the Problems

[0005] This invention provides a system that can acquire video data within a parking lot using a monitoring device and detect abnormal behavior using image analysis means. It includes a control means that immediately issues a warning signal when an abnormality is detected. Furthermore, it has a function to periodically acquire vehicle location information using a location tracking device and store it in a database. In addition, it can transmit warning notifications from the server device to terminal devices in real time, quickly informing users of the situation. This significantly improves parking lot security, prevents vehicle theft, and enables a rapid response in the event of theft.

[0006] A "monitoring device" is a device that acquires video footage from within a parking lot and monitors the situation in real time.

[0007] "Image analysis means" refers to a software or hardware system for analyzing acquired video data and detecting abnormal behavior.

[0008] "Abnormal behavior" refers to actions or movements that are not normal, and includes behaviors that may pose a security risk.

[0009] A "warning signal" is a signal that is immediately issued when an abnormality is detected, and its purpose is to convey an alarm to the site or terminal equipment.

[0010] A "control mechanism" is a mechanism for managing the operation of the entire monitoring system and for controlling related devices when an anomaly is detected.

[0011] A "location tracking device" is a device that tracks a vehicle's location information in real time and acquires and manages that information.

[0012] A "database" is a system for recording acquired information over a long period of time and storing it in a format that allows for referencing and analysis as needed.

[0013] A "server device" is a central device that manages information between multiple devices and user terminals within a monitoring system and provides necessary services.

[0014] A "terminal device" is a device that a user can directly operate and is responsible for receiving information from a server and transmitting it to the user. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the language used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] As an example of how to implement the present invention, we will describe the design of a high-security parking lot using an AI monitoring system. This system consists of three main elements: a server, a terminal, and a user.

[0037] The server acquires video data in real time from multiple surveillance cameras in the parking lot and uses AI image analysis to detect suspicious behavior. For example, it can instantly recognize a person loitering unnaturally in the parking lot at night or a vehicle exhibiting unusual movements. When the server detects an anomaly, it immediately sends a warning signal and activates sensor lights and sirens to deter any abnormal behavior at that location.

[0038] The terminal receives warnings and notifications from the server in real time and provides them to the user. For example, when abnormal behavior is detected, the terminal sends a notification to the user's smartphone, informing them of the specific nature and location of the abnormality to prompt action. It is also possible to send confirmation feedback of the abnormality to the server through the terminal.

[0039] Users can check the vehicle's security status at any time through the terminal's interface. Furthermore, the vehicle's location information obtained by the location tracking device is displayed on the terminal, allowing users to understand the vehicle's current location and take appropriate action. For example, if a user receives an anomaly notification, they can check the vehicle's location information displayed on the map and quickly notify the police or security company, thereby minimizing theft damage.

[0040] Thus, this invention enhances security in parking lots, prevents vehicle theft, and enables a rapid response in emergencies.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The server acquires video data in real time from surveillance cameras installed in the parking lot. It then starts processing the video captured by the surveillance cameras as input data.

[0044] Step 2:

[0045] The server inputs the acquired video data into an AI image analysis system to analyze whether there is any suspicious activity. For example, it can detect prolonged stays within a specific area or the movement of suspicious objects.

[0046] Step 3:

[0047] If the server detects abnormal behavior based on AI analysis, it will issue a warning signal, activate sensor lights in the parking lot, and sound a siren. This creates a deterrent effect against suspicious individuals at the scene.

[0048] Step 4:

[0049] The server sends information about the occurrence of an anomaly to the terminal. This information includes the type of anomaly, the location, and the time it occurred, providing the terminal with a specific alert.

[0050] Step 5:

[0051] The device receives an alert from the server and displays it to the user as a push notification. This notification includes detailed information about the problem, along with a message urging prompt action.

[0052] Step 6:

[0053] The user checks the notification on their device and presses the feedback button if necessary. This action sends feedback to the server indicating that an anomaly has been detected.

[0054] Step 7:

[0055] The server periodically acquires vehicle location information from the location tracking device and continuously monitors for any abnormalities.

[0056] Step 8:

[0057] The device receives location information from the server and displays the vehicle's current location on a map. The user reviews this information and decides whether or not to proceed with actual tracking.

[0058] Step 9:

[0059] Users can check their location and, if necessary, report it to the police or security company. The device interface provides contact information for reporting and supports a quick response.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] Conventional parking lot security systems have difficulty accurately detecting suspicious activity in real time, and also struggle to respond quickly after detection. Therefore, they have been unable to prevent vehicle theft and fraudulent activity. Furthermore, there have been insufficient means for users to easily track the location of their vehicles remotely.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes digital signal processing means, control means for issuing a warning signal when an anomaly is detected, and means for activating a notification device using light and sound. This enables high-speed and accurate detection of abnormal behavior and prompt issuance of warnings. In addition, it periodically acquires vehicle location information, making it easy for the user to understand the vehicle's current location.

[0065] A "monitoring device" is a device that monitors the environment within a parking lot and acquires image data through digital signal processing.

[0066] "Digital signal processing means" is a general term for methods and devices used to analyze image data and detect abnormal behavior.

[0067] A "warning signal" is a notification signal that is emitted when an abnormality is detected, and its purpose is to inform those around that an abnormality has occurred.

[0068] "Control means" refers to systems or mechanisms that control the transmission of warning signals or the operation of notification devices.

[0069] A "location tracking device" is a device that tracks the movement and current location of a vehicle and provides accurate location information.

[0070] A "storage device" is a device that stores acquired data and allows it to be retrieved as needed.

[0071] A "notification device" is a device that uses light or sound to warn those in the surrounding area when an abnormality is detected.

[0072] A "communication device" is a device or interface used to send and receive data between a server and a terminal.

[0073] A "mobile device" is a device that a user can carry and use to receive warnings and notifications.

[0074] An "information display device" is a device that visually displays the location and movement path of a vehicle.

[0075] This invention is a system for enhancing parking lot security and includes a monitoring device, digital signal processing means, control means, location tracking device, storage device, notification device, communication device, and mobile terminal. Embodiments of this system are described in detail below.

[0076] Server Role

[0077] The server acquires video data in real time from multiple monitoring devices. This process utilizes image processing software as a digital signal processing tool to detect suspicious behavior. Video analysis uses a multi-layer neural network to identify individuals or vehicles exhibiting abnormal behavior. For example, an AI model can be used to identify individuals exhibiting unnatural movements in a parking lot. Furthermore, if an anomaly is detected, the server uses control devices to send a warning signal and activate notification devices.

[0078] Terminal role

[0079] The terminal receives warning notifications sent from the server in real time and informs the user of the occurrence of anomalies. The terminal includes mobile devices, and users can receive anomaly notifications through a smartphone app. For example, the app may display a notification such as "Suspicious activity detected. Location: B3 area." Through the terminal, users can send information about confirmed anomalies as feedback to the server.

[0080] User roles

[0081] Users can manage the vehicle's security status through an application on their device. Furthermore, vehicle location information obtained from the location tracking device is displayed in real time on the device's information display, allowing users to understand the vehicle's current location and respond quickly as needed. For example, upon receiving an anomaly notification, it is recommended to check the vehicle's location on a map and report it to the police.

[0082] Examples of generative AI models and prompts

[0083] This system uses a generative AI model to automatically generate responses under specific conditions. An example of a prompt message is: "Generate a user notification message when suspicious activity is detected in the parking lot. The notification should include the detected area and recommended actions."

[0084] As described above, this invention significantly improves the safety of parking lots and makes it possible to prevent vehicle theft and fraudulent activities.

[0085] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0086] Step 1:

[0087] The server acquires video data in real time from the monitoring device as input. This data serves as the initial input for the program. The server performs digital signal processing frame by frame, applying pre-processing such as noise reduction and motion extraction. This generates high-quality video information. The server then passes the pre-processed data to the next analysis step.

[0088] Step 2:

[0089] The server inputs pre-processed video data into an AI image analysis model. This model includes a generative AI model that detects behavioral patterns indicating abnormal activity. Specifically, it uses a multi-layer neural network to identify the movements of people and vehicles. The analysis yields an anomaly detection output, determining whether or not suspicious behavior is present. The server then uses this result in the next control step.

[0090] Step 3:

[0091] Based on the analysis results of the AI ​​model, the server outputs a warning signal if an anomaly is detected. This signal activates a notification device via a control mechanism. Specifically, it outputs control commands that cause warning lights to flash or sirens to sound. This action aims to deter suspicious individuals.

[0092] Step 4:

[0093] The terminal receives warning notifications from the server as input data. These notifications are output to the user's mobile device in real time. The terminal displays information about the specific nature of the anomaly and its location on the user interface, informing the user visually and audibly. This notification prompts the user to take immediate action.

[0094] Step 5:

[0095] The user receives notifications from their device as input and uses the app's map function to check the vehicle's location. This operation outputs the vehicle's current location and its travel route. If necessary, the user can take concrete action, such as contacting the police or administrators based on the map information. This enables quick problem resolution.

[0096] (Application Example 1)

[0097] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0098] Conventional security systems struggled to detect abnormal activity within monitored areas in real time and respond quickly. Furthermore, there was a lack of means for users to appropriately respond when an anomaly occurred, necessitating enhanced security. Additionally, there was a lack of efficient means to manage and track the location information of objects within monitored areas.

[0099] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0100] In this invention, the server includes a monitoring device, means for acquiring video information within a monitoring area using image analysis means and detecting abnormal operation, a control unit that issues warning information when an abnormality is detected, means for periodically acquiring the location information of objects using a location confirmation device and storing it in an information collection, means for analyzing video within the monitoring area in real time and using an object detection model to identify suspicious operation, and means for transmitting warning information to an information exchange device using a communication system. This makes it possible to quickly and accurately detect abnormal operation within a monitoring area and immediately notify users. In addition, it becomes possible to efficiently manage the location information of objects and make it easy for users to track their movements.

[0101] A "monitoring device" is a device used to acquire video information within a monitored area.

[0102] "Image analysis means" refers to a technology that analyzes acquired video information to detect suspicious or abnormal behavior.

[0103] "Abnormal behavior" refers to suspicious movements or actions that deviate from normal movements.

[0104] "Warning information" refers to information used to notify users when abnormal operation is detected.

[0105] A "control unit" is a device that performs control to issue warning information when abnormal operation is detected.

[0106] A "position confirmation device" is a device that acquires and provides information about the position of an object.

[0107] An "information collection" is a database used to store acquired location information and other data.

[0108] An "object detection model" is a machine learning algorithm used to identify objects and actions in video footage.

[0109] A "communication system" is a general term for network technologies and devices used to exchange information.

[0110] An "information exchange device" is a terminal device used to provide information to users.

[0111] In order to implement the present invention, it is necessary for each component of the monitoring system to work closely together to enhance security within the monitoring area.

[0112] First, the server uses monitoring equipment to acquire video information within the monitored area in real time. The video is analyzed by image analysis tools, and an AI model (such as an object detection algorithm like YOLOv5) is used to identify suspicious or abnormal behavior. The information obtained from this analysis is immediately generated as warning information.

[0113] Next, the control unit, triggered by this warning information, uses the communication system to send a notification to information exchange devices, specifically the user's mobile terminal. This notification includes details of the anomaly and the situation on site, enabling the user to respond quickly.

[0114] Furthermore, the location tracking device periodically acquires location information of objects within the monitoring area and stores it in an information database. This location information is provided to the user via a communication system, making it possible to track the movement of objects in real time.

[0115] As a concrete example, when a user uses the application on their smartphone, if a suspicious person is detected near the entrance at night, a notification is immediately sent. At this point, the user can view the video on the application and take appropriate action, such as contacting a security company or the police.

[0116] By using generative AI models, it is possible to achieve highly accurate identification of suspicious individuals even in unknown situations, significantly improving security levels. To support such systems, image analysis prompts are used in the format of "Analyze the following video data and detect any suspicious human or vehicle movements: [link to video data]".

[0117] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0118] Step 1:

[0119] The server acquires video information from the monitoring device. The input here is the raw video data transmitted from the monitoring device, and the output is a file stored on the server as this video data. This process converts the video stream into a digital format, creating a structure suitable for analysis.

[0120] Step 2:

[0121] The server analyzes the acquired video information using a generating AI model. The input for this step is video data, and the output is information about detected suspicious or abnormal behavior. The AI ​​model (e.g., YOLOv5) is used to identify objects in the video quickly and accurately and to identify suspicious behavior.

[0122] Step 3:

[0123] The server generates warning information based on the detection results of abnormal operation and sends it to the terminal using the communication system. The input to this process is the abnormal operation information from step 2, and the output is the warning information sent to the terminal. In this process, a message with details of the abnormality is created and sent to the user's terminal in real time.

[0124] Step 4:

[0125] The terminal displays the received warning information to the user as a notification. The input for this step is the warning information received from the server, and the output is the notification displayed on the user's screen. This notification typically informs the user of an anomaly as a pop-up on the screen or a sound alert.

[0126] Step 5:

[0127] Users can access detailed information and take appropriate action through their device. The input consists of warning information and real-time video displayed on the device, while the output is the action the user takes (e.g., calling the police). Users check the situation and select the necessary action by operating the response buttons within the app.

[0128] Step 6:

[0129] The location tracking device periodically updates the location information of objects within the monitoring area and stores it in an information database. The input here is the object's current location, and the output is the location information stored in the database. This data is later used for location tracking and made accessible to users.

[0130] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0131] As an embodiment of the present invention, the following example describes a high-security parking lot configuration that combines an AI monitoring system with an emotion engine that detects the user's emotional state and appropriately adjusts alerts and information based on that state. This system includes the emotion engine in addition to three main elements: a server, a terminal, and a user.

[0132] The server acquires video data from surveillance cameras in real time and processes it using AI image analysis. This process detects suspicious movements and behaviors, and if abnormal behavior is confirmed, it issues a warning signal and transmits the information to the terminal. Furthermore, it periodically acquires the vehicle's location information using a location tracking device and records it in a database.

[0133] The device receives alerts from the server and notifies the user. During this process, the emotion engine analyzes the user's biometric information and voice input. For example, when a user gives a voice command, the system analyzes their emotional state and adjusts the alert volume and notification method according to the urgency. If the user is under high stress, the notification is delivered in a faster and clearer manner. Conversely, under normal circumstances, the standard notification settings are maintained.

[0134] Users can receive security status notifications tailored to their emotions through their device. For example, if they receive a notification while in an agitated state, the notification will include detailed instructions and reassuring messages, allowing them to quickly understand the situation and take appropriate action. Furthermore, the device's interface allows users to check the vehicle's location in real time, enabling smooth operation to contact the police or security company if necessary.

[0135] Thus, by combining a monitoring function with an emotion engine, the present invention can provide appropriate responses tailored to the user's psychological state, thereby creating an environment where users can use parking lots with greater peace of mind.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] The server acquires video data in real time from surveillance cameras installed in the parking lot. The AI ​​image analysis system then starts processing the video captured by the surveillance cameras as input data.

[0139] Step 2:

[0140] The server analyzes suspicious movements and behaviors using AI image analysis to detect abnormal behavior. For example, it may identify prolonged stays in a specific area or objects exhibiting sudden movements as abnormal.

[0141] Step 3:

[0142] If the server detects abnormal behavior, it immediately issues a warning signal, controls the sensor lights in the parking lot to illuminate them, and sounds a siren to alert on-site personnel.

[0143] Step 4:

[0144] The server sends information about the occurrence of an anomaly to the terminal, which includes the type of anomaly detected, the location, and the time of occurrence.

[0145] Step 5:

[0146] The device receives alerts from the server and uses an emotion engine to analyze the user's biometric information and voice input. It then acquires data to determine the user's emotional state.

[0147] Step 6:

[0148] Based on the results analyzed by the emotion engine, the device adjusts how alert notifications are delivered according to the user's emotional state. For example, if the user is agitated, the notification volume may be increased or more detailed explanations may be added.

[0149] Step 7:

[0150] The user checks the notification from the device and follows the instructions provided, contacting the police or security company if necessary. The device displays contact information to support quick reporting.

[0151] Step 8:

[0152] The server constantly monitors the vehicle's location based on data from the location tracking device, continuously checking for any abnormalities. Users can view the vehicle's current location on a map in real time via their device.

[0153] (Example 2)

[0154] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0155] Conventional parking lot monitoring systems were limited to detecting abnormal behavior and lacked the ability to provide appropriate responses that considered the user's psychological state. This meant that notifications received by users were not situation-specific, making prompt responses difficult. Furthermore, there was a lack of efficient means to manage and provide real-time location information of moving objects to users.

[0156] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0157] In this invention, the server includes a monitoring mechanism, means for acquiring video information within the parking lot using an image analysis method and detecting abnormal operation, a control function for issuing a warning signal when an abnormality is detected, means for periodically acquiring the location information of moving objects using location tracking technology and storing it in a storage device, and an emotion analysis function for detecting the user's emotional state and appropriately adjusting the notification method. This enables prompt responses with appropriate notifications and explanations according to the user's psychological state, as well as real-time tracking of the location of moving objects.

[0158] A "monitoring mechanism" refers to a device that includes cameras and sensors installed to acquire video information within the parking lot.

[0159] "Image analysis techniques" refer to algorithms and technologies used to identify suspicious behavior or anomalies from acquired video information.

[0160] "Abnormal behavior" refers to suspicious movements or actions that differ from normal behavior in a parking lot, and should be monitored.

[0161] A "control function" is a mechanism that controls the operation to issue a warning signal based on detected abnormalities.

[0162] "Location tracking technology" refers to technologies and methods for continuously acquiring and tracking the current location of a moving object.

[0163] "Storage device" refers to a database or storage device used to save acquired data and information.

[0164] "Emotional analysis function" is a technology used to analyze a user's psychological state based on their biometric information and voice, and to determine the appropriate notification method.

[0165] "Moving object" refers to vehicles or other moving objects being tracked within a parking lot.

[0166] A "display mechanism" refers to a screen or interface that provides information visually, making it easier for users to access the information.

[0167] This invention is a monitoring system designed to improve parking lot safety and includes three main components: a server, a terminal, and a user. The server acquires video information in real time from surveillance cameras and sensors and processes this information using AI image analysis means. In doing so, it uses software such as OpenCV and TENSORFLOW® to detect suspicious movements and behaviors. When an anomaly is detected, the server generates a warning signal using its control function and transmits this information to the terminal using a secure communication protocol. The server also manages the system by periodically acquiring the location information of moving objects using location tracking technology and storing it in a storage device.

[0168] When the device receives a warning signal from the server, it uses its built-in emotion analysis function to analyze the user's biometric information and voice data. Biosensors and microphones assist in this analysis, and based on the analysis results, the device determines the urgency and adjusts the content and volume of the notification to deliver information in the most appropriate way for the user.

[0169] Users can check notifications and track the location of moving objects in real time through the terminal interface. This allows for quick response measures. If a user detects an anomaly, they can feed that information back to the server via the terminal, providing an opportunity to smoothly notify security services or the police as needed.

[0170] For example, if suspicious activity is detected near a parked vehicle, the server immediately issues a warning, and the terminal adjusts the notification content according to the user's current emotional state. In this process, we utilize prompts input to a generative AI model such as, "We want to design a system that uses AI to identify abnormal behavior in parking lots and notifies users of alerts in an appropriate manner based on their emotional state. Please suggest what algorithms and devices we should use," providing an approach to optimize the effectiveness of notifications.

[0171] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0172] Step 1:

[0173] The server acquires video information in real time from surveillance cameras and sensors. It receives video data from the surveillance devices as input and processes this data using image analysis tools within the server. To identify suspicious movements or behaviors, it analyzes the information using AI algorithms and outputs data labeled as abnormal.

[0174] Step 2:

[0175] The server generates a warning signal based on the abnormal behavior detected by the AI. The input is the anomaly detection data from step 1, and the control function is used to create the warning signal. This signal includes the details and location information of the anomaly and is ready to be sent to the terminal. The output is sending the warning signal to the terminal using a secure communication protocol.

[0176] Step 3:

[0177] The terminal receives a warning signal sent from the server and notifies the user. The input is the warning signal from the server. The built-in emotion analysis function collects the user's biometric information (heart rate, etc.) and voice data and analyzes this information. The output is a notification adjusted according to the urgency level, which is displayed to the user via voice and visuals.

[0178] Step 4:

[0179] Users check notifications through their devices and take action based on detected anomalies. Input is notifications from the device. Output is that users use the provided information to assess the situation and contact security services or the police if necessary. Furthermore, the device allows for real-time location tracking of moving objects.

[0180] Step 5:

[0181] The user sends feedback to the server indicating that the abnormal situation has been resolved. The input is the feedback information from the user. As output, the server receives the feedback, saves it to a recording device, and prepares for the next system warning. This creates data that will be useful for improving the accuracy of future anomaly detection.

[0182] (Application Example 2)

[0183] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0184] While conventional monitoring systems are effective at detecting suspicious behavior and issuing alerts, they have the challenge of not being able to provide optimal responses that take into account the psychological aspects of users. In particular, there is a need to reduce the psychological burden on users in emergencies and to provide accurate information in a calm state.

[0185] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0186] In this invention, the server includes a monitoring device, means for acquiring video data by image analysis means and detecting abnormal behavior, means for detecting the user's emotional state by biometric information analysis means and adjusting the notification method, and means for periodically acquiring vehicle location information by a location tracking device and storing it in a database. This enables appropriate security measures that are in line with the user's emotional state.

[0187] A "monitoring device" is a device consisting of hardware and software that acquires video data in real time and detects abnormal behavior.

[0188] "Image analysis means" refers to AI-based technology that processes acquired video data and detects specific patterns.

[0189] "Abnormal behavior" refers to actions that deviate from normal behavioral patterns and should be considered suspicious or alarming.

[0190] A "control mechanism" is a function that issues commands to take appropriate action within the system when an abnormality is detected.

[0191] A "location tracking device" is a device that identifies the current location of a moving object, such as a vehicle, and provides location information.

[0192] A "database" is a system that plays a role in securely and efficiently storing and managing information acquired by a system.

[0193] "Biometric information analysis means" refers to technology that analyzes a user's biometric information, such as their voice and gestures, to recognize their emotional state.

[0194] "Emotional state" refers to the user's current psychological or emotional condition.

[0195] "Notification methods" refer to all means and processes used to communicate information or warnings to users.

[0196] In an embodiment of this invention, the monitoring system consists of three elements: a server, a terminal, and a user. The server acquires video data in real time using a monitoring device and detects abnormal behavior using image analysis means. It also has the function of periodically acquiring vehicle location information using a location tracking device and storing it in a database. The hardware used includes a monitoring camera, a location tracking device, and a server computer. The software used is an image and audio analysis system using Python or TensorFlow.

[0197] Meanwhile, the terminal receives warning notifications from the server in real time. At the same time, it analyzes the user's emotional state through biometric data analysis to determine the optimal notification method. Smartphones and tablets are commonly used as this terminal. Through this terminal, users can view the vehicle's location on a map and, if necessary, quickly report the incident to the police or security company. An intuitive interface is provided for this operation.

[0198] As a concrete example, a user could check the security status via a smartphone app when leaving a parking lot. When the user speaks to their smartphone saying, "Is it okay?", the device can analyze the user's anxious tone through emotion analysis and return a response emphasizing that the security system is functioning correctly. In implementing such a system, an example of a prompt message could be: "Detect suspicious activity from the surveillance camera footage. Analyze the user's voice emotion and adjust the notification method according to the urgency."

[0199] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0200] Step 1:

[0201] The server acquires video data from the monitoring device in real time. This data is input in the form of a video stream from the surveillance camera. The server sends this video data to an AI image analysis system to perform processing to detect abnormal behavior. Specifically, a deep learning model using TensorFlow analyzes the movement in the video and compares it with normal behavior patterns. When an anomaly is detected, that information is generated as a warning signal.

[0202] Step 2:

[0203] The server periodically acquires vehicle location information using a location tracking device. This location information is entered as GPS data, and the server stores this data in a database. The data is configured to be updated at specific time intervals, allowing for real-time tracking of location changes.

[0204] Step 3:

[0205] The terminal receives a warning signal from the server in real time. This signal is transmitted to the terminal as an alert when abnormal behavior is detected. The terminal's application prepares to display the notification content to the user and passes the data to the subsequent sentiment analysis process.

[0206] Step 4:

[0207] The device analyzes the user's emotional state using biometric data analysis. The input is voice data acquired through the smartphone's microphone. This voice data is analyzed by a voice emotion analysis model to determine the user's emotional state (e.g., emergency, normal, alert). Based on the output, the device adjusts notification volume, vibration patterns, and other parameters.

[0208] Step 5:

[0209] Users receive analyzed notifications through their devices. For example, if the device is analyzed as being in a high-stress state, it will emphasize voice and vibration and display a message prompting a quick response. Users can also check the vehicle's location on the device screen, allowing them to understand the situation with peace of mind.

[0210] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0211] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0212] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0213] [Second Embodiment]

[0214] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0215] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0216] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0217] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0218] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0219] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0220] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0221] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0222] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0223] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0224] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0225] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0226] As an example of how to implement the present invention, we will describe the design of a high-security parking lot using an AI monitoring system. This system consists of three main elements: a server, a terminal, and a user.

[0227] The server acquires video data in real time from multiple surveillance cameras in the parking lot and uses AI image analysis to detect suspicious behavior. For example, it can instantly recognize a person loitering unnaturally in the parking lot at night or a vehicle exhibiting unusual movements. When the server detects an anomaly, it immediately sends a warning signal and activates sensor lights and sirens to deter any abnormal behavior at that location.

[0228] The terminal receives warnings and notifications from the server in real time and provides them to the user. For example, when abnormal behavior is detected, the terminal sends a notification to the user's smartphone, informing them of the specific nature and location of the abnormality to prompt action. It is also possible to send confirmation feedback of the abnormality to the server through the terminal.

[0229] Users can check the vehicle's security status at any time through the terminal's interface. Furthermore, the vehicle's location information obtained by the location tracking device is displayed on the terminal, allowing users to understand the vehicle's current location and take appropriate action. For example, if a user receives an anomaly notification, they can check the vehicle's location information displayed on the map and quickly notify the police or security company, thereby minimizing theft damage.

[0230] Thus, this invention enhances security in parking lots, prevents vehicle theft, and enables a rapid response in emergencies.

[0231] The following describes the processing flow.

[0232] Step 1:

[0233] The server acquires video data in real time from surveillance cameras installed in the parking lot. It then starts processing the video captured by the surveillance cameras as input data.

[0234] Step 2:

[0235] The server inputs the acquired video data into an AI image analysis system to analyze whether there is any suspicious activity. For example, it can detect prolonged stays within a specific area or the movement of suspicious objects.

[0236] Step 3:

[0237] If the server detects abnormal behavior based on AI analysis, it will issue a warning signal, activate sensor lights in the parking lot, and sound a siren. This creates a deterrent effect against suspicious individuals at the scene.

[0238] Step 4:

[0239] The server sends information about the occurrence of an anomaly to the terminal. This information includes the type of anomaly, the location, and the time it occurred, providing the terminal with a specific alert.

[0240] Step 5:

[0241] The device receives an alert from the server and displays it to the user as a push notification. This notification includes detailed information about the problem, along with a message urging prompt action.

[0242] Step 6:

[0243] The user checks the notification on their device and presses the feedback button if necessary. This action sends feedback to the server indicating that an anomaly has been detected.

[0244] Step 7:

[0245] The server periodically acquires vehicle location information from the location tracking device and continuously monitors for any abnormalities.

[0246] Step 8:

[0247] The device receives location information from the server and displays the vehicle's current location on a map. The user reviews this information and decides whether or not to proceed with actual tracking.

[0248] Step 9:

[0249] Users can check their location and, if necessary, report it to the police or security company. The device interface provides contact information for reporting and supports a quick response.

[0250] (Example 1)

[0251] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0252] Conventional parking lot security systems have difficulty accurately detecting suspicious activity in real time, and also struggle to respond quickly after detection. Therefore, they have been unable to prevent vehicle theft and fraudulent activity. Furthermore, there have been insufficient means for users to easily track the location of their vehicles remotely.

[0253] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0254] In this invention, the server includes digital signal processing means, control means for issuing a warning signal when an anomaly is detected, and means for activating a notification device using light and sound. This enables high-speed and accurate detection of abnormal behavior and prompt issuance of warnings. In addition, it periodically acquires vehicle location information, making it easy for the user to understand the vehicle's current location.

[0255] A "monitoring device" is a device that monitors the environment within a parking lot and acquires image data through digital signal processing.

[0256] "Digital signal processing means" is a general term for methods and devices used to analyze image data and detect abnormal behavior.

[0257] A "warning signal" is a notification signal that is emitted when an abnormality is detected, and its purpose is to inform those around that an abnormality has occurred.

[0258] "Control means" refers to systems or mechanisms that control the transmission of warning signals or the operation of notification devices.

[0259] A "location tracking device" is a device that tracks the movement and current location of a vehicle and provides accurate location information.

[0260] A "storage device" is a device that stores acquired data and allows it to be retrieved as needed.

[0261] A "notification device" is a device that uses light or sound to warn those in the surrounding area when an abnormality is detected.

[0262] A "communication device" is a device or interface used to send and receive data between a server and a terminal.

[0263] A "mobile device" is a device that a user can carry and use to receive warnings and notifications.

[0264] An "information display device" is a device that visually displays the location and movement path of a vehicle.

[0265] This invention is a system for enhancing parking lot security and includes a monitoring device, digital signal processing means, control means, location tracking device, storage device, notification device, communication device, and mobile terminal. Embodiments of this system are described in detail below.

[0266] Server Role

[0267] The server acquires video data in real time from multiple monitoring devices. This process utilizes image processing software as a digital signal processing tool to detect suspicious behavior. Video analysis uses a multi-layer neural network to identify individuals or vehicles exhibiting abnormal behavior. For example, an AI model can be used to identify individuals exhibiting unnatural movements in a parking lot. Furthermore, if an anomaly is detected, the server uses control devices to send a warning signal and activate notification devices.

[0268] Terminal role

[0269] The terminal receives warning notifications sent from the server in real time and informs the user of the occurrence of anomalies. The terminal includes mobile devices, and users can receive anomaly notifications through a smartphone app. For example, the app may display a notification such as "Suspicious activity detected. Location: B3 area." Through the terminal, users can send information about confirmed anomalies as feedback to the server.

[0270] User roles

[0271] Users can manage the vehicle's security status through an application on their device. Furthermore, vehicle location information obtained from the location tracking device is displayed in real time on the device's information display, allowing users to understand the vehicle's current location and respond quickly as needed. For example, upon receiving an anomaly notification, it is recommended to check the vehicle's location on a map and report it to the police.

[0272] Examples of generative AI models and prompts

[0273] This system uses a generative AI model to automatically generate responses under specific conditions. An example of a prompt message is: "Generate a user notification message when suspicious activity is detected in the parking lot. The notification should include the detected area and recommended actions."

[0274] As described above, this invention significantly improves the safety of parking lots and makes it possible to prevent vehicle theft and fraudulent activities.

[0275] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0276] Step 1:

[0277] The server acquires video data in real time as input from the monitoring device. This data is the initial input of the program. The server performs digital signal processing for each frame and conducts preprocessing such as noise removal and motion extraction. As a result, high-quality video information is generated. The server passes the preprocessed data to the next analysis step.

[0278] Step 2:

[0279] The server inputs the preprocessed video data into the AI image analysis model. This model includes a generative AI model that detects behavior patterns indicating abnormal activities. Specifically, a multi-layer neural network is used to identify the movements of people and vehicles. As a result of the analysis, an output of anomaly detection is obtained, and it is determined whether there is any suspicious behavior. The server uses this result in the next control step.

[0280] Step 3:

[0281] Based on the analysis result of the AI model, when an anomaly is detected, the server outputs a warning signal. This signal activates the notification device via the control means. Specifically, it outputs a control command that causes the warning light to blink or the siren to sound. This action aims to have a deterrent effect on suspicious individuals.

[0282] Step 4:

[0283] The terminal receives the warning notification from the server as input data. This notification is output to the user's mobile terminal in real time. The terminal displays information about the specific content of the anomaly and the location where it occurred on the user interface, notifying the user visually and auditorily. This notification prompts the user to respond immediately.

[0284] Step 5:

[0285] The user receives the notification from the terminal as an input and checks the position of the vehicle using the map function of the application. By this operation, the current position of the vehicle and its movement route are output. If necessary, the user can take specific actions to contact the police or the administrator based on the map information. This enables prompt problem-solving.

[0286] (Application Example 1)

[0287] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0288] In the conventional security system, it was difficult to detect abnormal operations within the monitoring area in real time and respond promptly. In addition, since there were insufficient means for users to respond appropriately when an abnormality occurred, there was a need to enhance security. Furthermore, there were insufficient means for efficiently managing and tracking the position information of objects within the monitoring area.

[0289] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0290] In this invention, the server includes a monitoring device, means for acquiring video information within the monitoring area by image analysis means and detecting abnormal operations, a control unit for issuing warning information when an abnormality is detected, means for periodically acquiring the position information of an object by a position confirmation device and storing it in an information collection, means for analyzing the video within the monitoring area in real time and using an object detection model to identify suspicious operations, and means for transmitting warning information to an information exchange device using a communication system. This enables the rapid and accurate detection of abnormal operations within the monitoring area and immediate notification to the user. In addition, the position information of the object can be efficiently managed, making it easy for the user to track its movement.

[0291] The "monitoring device" is a device for acquiring video information within the monitoring area.

[0292] "Image analysis means" refers to a technology that analyzes acquired video information to detect suspicious or abnormal behavior.

[0293] "Abnormal behavior" refers to suspicious movements or actions that deviate from normal movements.

[0294] "Warning information" refers to information used to notify users when abnormal operation is detected.

[0295] A "control unit" is a device that performs control to issue warning information when abnormal operation is detected.

[0296] A "position confirmation device" is a device that acquires and provides information about the position of an object.

[0297] An "information collection" is a database used to store acquired location information and other data.

[0298] An "object detection model" is a machine learning algorithm used to identify objects and actions in video footage.

[0299] A "communication system" is a general term for network technologies and devices used to exchange information.

[0300] An "information exchange device" is a terminal device used to provide information to users.

[0301] In order to implement the present invention, it is necessary for each component of the monitoring system to work closely together to enhance security within the monitoring area.

[0302] First, the server uses monitoring equipment to acquire video information within the monitored area in real time. The video is analyzed by image analysis tools, and an AI model (such as an object detection algorithm like YOLOv5) is used to identify suspicious or abnormal behavior. The information obtained from this analysis is immediately generated as warning information.

[0303] Next, triggered by this warning information, the control unit issues a notice to the information exchange device, specifically the user's mobile terminal, using the communication system. This notice includes details of the abnormality and the situation at the scene to enable the user to respond promptly.

[0304] Furthermore, the position confirmation device periodically acquires the position information of objects within the monitoring area and stores it in the information collection. This position information is provided to the user via the communication system, enabling real-time tracking of the movement of the objects.

[0305] As a specific example, when a user uses an application on a smartphone and a suspicious person is detected near the entrance at night, a notification is immediately sent. At this time, the user can check the video on the application and take corresponding actions such as reporting to the security company or the police if necessary.

[0306] By using the generated AI model, high-precision identification of suspicious persons can be achieved even for unknown situations, significantly improving the security level. As an image analysis prompt sentence for supporting such a system, an instruction in the form of "Analyze the following video data and detect any suspicious human or vehicle movements: Link to the video data" is used.

[0307] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0308] Step 1:

[0309] The server acquires video information from the monitoring device. The input here is the raw video data transmitted from the monitoring device, and the output is the file stored on the server as this video data. In this process, the video stream is converted into a digital format and structured for analysis.

[0310] Step 2:

[0311] The server analyzes the acquired video information using a generating AI model. The input for this step is video data, and the output is information about detected suspicious or abnormal behavior. The AI ​​model (e.g., YOLOv5) is used to identify objects in the video quickly and accurately and to identify suspicious behavior.

[0312] Step 3:

[0313] The server generates warning information based on the detection results of abnormal operation and sends it to the terminal using the communication system. The input to this process is the abnormal operation information from step 2, and the output is the warning information sent to the terminal. In this process, a message with details of the abnormality is created and sent to the user's terminal in real time.

[0314] Step 4:

[0315] The terminal displays the received warning information to the user as a notification. The input for this step is the warning information received from the server, and the output is the notification displayed on the user's screen. This notification typically informs the user of an anomaly as a pop-up on the screen or a sound alert.

[0316] Step 5:

[0317] Users can access detailed information and take appropriate action through their device. The input consists of warning information and real-time video displayed on the device, while the output is the action the user takes (e.g., calling the police). Users check the situation and select the necessary action by operating the response buttons within the app.

[0318] Step 6:

[0319] The location tracking device periodically updates the location information of objects within the monitoring area and stores it in an information database. The input here is the object's current location, and the output is the location information stored in the database. This data is later used for location tracking and made accessible to users.

[0320] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0321] As an embodiment of the present invention, the following example describes a high-security parking lot configuration that combines an AI monitoring system with an emotion engine that detects the user's emotional state and appropriately adjusts alerts and information based on that state. This system includes the emotion engine in addition to three main elements: a server, a terminal, and a user.

[0322] The server acquires video data from surveillance cameras in real time and processes it using AI image analysis. This process detects suspicious movements and behaviors, and if abnormal behavior is confirmed, it issues a warning signal and transmits the information to the terminal. Furthermore, it periodically acquires the vehicle's location information using a location tracking device and records it in a database.

[0323] The device receives alerts from the server and notifies the user. During this process, the emotion engine analyzes the user's biometric information and voice input. For example, when a user gives a voice command, the system analyzes their emotional state and adjusts the alert volume and notification method according to the urgency. If the user is under high stress, the notification is delivered in a faster and clearer manner. Conversely, under normal circumstances, the standard notification settings are maintained.

[0324] Users can receive security status notifications tailored to their emotions through their device. For example, if they receive a notification while in an agitated state, the notification will include detailed instructions and reassuring messages, allowing them to quickly understand the situation and take appropriate action. Furthermore, the device's interface allows users to check the vehicle's location in real time, enabling smooth operation to contact the police or security company if necessary.

[0325] Thus, by combining a monitoring function with an emotion engine, the present invention can provide appropriate responses tailored to the user's psychological state, thereby creating an environment where users can use parking lots with greater peace of mind.

[0326] The following describes the processing flow.

[0327] Step 1:

[0328] The server acquires video data in real time from surveillance cameras installed in the parking lot. The AI ​​image analysis system then starts processing the video captured by the surveillance cameras as input data.

[0329] Step 2:

[0330] The server analyzes suspicious movements and behaviors using AI image analysis to detect abnormal behavior. For example, it may identify prolonged stays in a specific area or objects exhibiting sudden movements as abnormal.

[0331] Step 3:

[0332] If the server detects abnormal behavior, it immediately issues a warning signal, controls the sensor lights in the parking lot to illuminate them, and sounds a siren to alert on-site personnel.

[0333] Step 4:

[0334] The server sends information about the occurrence of an anomaly to the terminal, which includes the type of anomaly detected, the location, and the time of occurrence.

[0335] Step 5:

[0336] The device receives alerts from the server and uses an emotion engine to analyze the user's biometric information and voice input. It then acquires data to determine the user's emotional state.

[0337] Step 6:

[0338] Based on the results analyzed by the emotion engine, the device adjusts how alert notifications are delivered according to the user's emotional state. For example, if the user is agitated, the notification volume may be increased or more detailed explanations may be added.

[0339] Step 7:

[0340] The user checks the notification from the device and follows the instructions provided, contacting the police or security company if necessary. The device displays contact information to support quick reporting.

[0341] Step 8:

[0342] The server constantly monitors the vehicle's location based on data from the location tracking device, continuously checking for any abnormalities. Users can view the vehicle's current location on a map in real time via their device.

[0343] (Example 2)

[0344] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0345] Conventional parking lot monitoring systems were limited to detecting abnormal behavior and lacked the ability to provide appropriate responses that considered the user's psychological state. This meant that notifications received by users were not situation-specific, making prompt responses difficult. Furthermore, there was a lack of efficient means to manage and provide real-time location information of moving objects to users.

[0346] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0347] In this invention, the server includes a monitoring mechanism, means for acquiring video information within the parking lot using an image analysis method and detecting abnormal operation, a control function for issuing a warning signal when an abnormality is detected, means for periodically acquiring the location information of moving objects using location tracking technology and storing it in a storage device, and an emotion analysis function for detecting the user's emotional state and appropriately adjusting the notification method. This enables prompt responses with appropriate notifications and explanations according to the user's psychological state, as well as real-time tracking of the location of moving objects.

[0348] A "monitoring mechanism" refers to a device that includes cameras and sensors installed to acquire video information within the parking lot.

[0349] "Image analysis techniques" refer to algorithms and technologies used to identify suspicious behavior or anomalies from acquired video information.

[0350] "Abnormal behavior" refers to suspicious movements or actions that differ from normal behavior in a parking lot, and should be monitored.

[0351] A "control function" is a mechanism that controls the operation to issue a warning signal based on detected abnormalities.

[0352] "Location tracking technology" refers to technologies and methods for continuously acquiring and tracking the current location of a moving object.

[0353] "Storage device" refers to a database or storage device used to save acquired data and information.

[0354] "Emotional analysis function" is a technology used to analyze a user's psychological state based on their biometric information and voice, and to determine the appropriate notification method.

[0355] "Moving object" refers to vehicles or other moving objects being tracked within a parking lot.

[0356] A "display mechanism" refers to a screen or interface that provides information visually, making it easier for users to access the information.

[0357] This invention is a monitoring system designed to improve parking lot safety and includes three main components: a server, a terminal, and a user. The server acquires video information in real time from surveillance cameras and sensors and processes this information using AI image analysis means. In doing so, it uses software such as OpenCV and TensorFlow to detect suspicious movements and behaviors. When an anomaly is detected, the server generates a warning signal using its control function and transmits this information to the terminal using a secure communication protocol. The server also manages the system by periodically acquiring the location information of moving objects using location tracking technology and storing it in a storage device.

[0358] When the device receives a warning signal from the server, it uses its built-in emotion analysis function to analyze the user's biometric information and voice data. Biosensors and microphones assist in this analysis, and based on the analysis results, the device determines the urgency and adjusts the content and volume of the notification to deliver information in the most appropriate way for the user.

[0359] Users can check notifications and track the location of moving objects in real time through the terminal interface. This allows for quick response measures. If a user detects an anomaly, they can feed that information back to the server via the terminal, providing an opportunity to smoothly notify security services or the police as needed.

[0360] For example, if suspicious activity is detected near a parked vehicle, the server immediately issues a warning, and the terminal adjusts the notification content according to the user's current emotional state. In this process, we utilize prompts input to a generative AI model such as, "We want to design a system that uses AI to identify abnormal behavior in parking lots and notifies users of alerts in an appropriate manner based on their emotional state. Please suggest what algorithms and devices we should use," providing an approach to optimize the effectiveness of notifications.

[0361] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0362] Step 1:

[0363] The server acquires video information in real time from surveillance cameras and sensors. It receives video data from the surveillance devices as input and processes this data using image analysis tools within the server. To identify suspicious movements or behaviors, it analyzes the information using AI algorithms and outputs data labeled as abnormal.

[0364] Step 2:

[0365] The server generates a warning signal based on the abnormal behavior detected by the AI. The input is the anomaly detection data from step 1, and the control function is used to create the warning signal. This signal includes the details and location information of the anomaly and is ready to be sent to the terminal. The output is sending the warning signal to the terminal using a secure communication protocol.

[0366] Step 3:

[0367] The terminal receives a warning signal sent from the server and notifies the user. The input is the warning signal from the server. The built-in emotion analysis function collects the user's biometric information (heart rate, etc.) and voice data and analyzes this information. The output is a notification adjusted according to the urgency level, which is displayed to the user via voice and visuals.

[0368] Step 4:

[0369] Users check notifications through their devices and take action based on detected anomalies. Input is notifications from the device. Output is that users use the provided information to assess the situation and contact security services or the police if necessary. Furthermore, the device allows for real-time location tracking of moving objects.

[0370] Step 5:

[0371] The user sends feedback to the server indicating that the abnormal situation has been resolved. The input is the feedback information from the user. As output, the server receives the feedback, saves it to a recording device, and prepares for the next system warning. This creates data that will be useful for improving the accuracy of future anomaly detection.

[0372] (Application Example 2)

[0373] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0374] While conventional monitoring systems are effective at detecting suspicious behavior and issuing alerts, they have the challenge of not being able to provide optimal responses that take into account the psychological aspects of users. In particular, there is a need to reduce the psychological burden on users in emergencies and to provide accurate information in a calm state.

[0375] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0376] In this invention, the server includes a monitoring device, means for acquiring video data by image analysis means and detecting abnormal behavior, means for detecting the user's emotional state by biometric information analysis means and adjusting the notification method, and means for periodically acquiring vehicle location information by a location tracking device and storing it in a database. This enables appropriate security measures that are in line with the user's emotional state.

[0377] A "monitoring device" is a device consisting of hardware and software that acquires video data in real time and detects abnormal behavior.

[0378] "Image analysis means" refers to AI-based technology that processes acquired video data and detects specific patterns.

[0379] "Abnormal behavior" refers to actions that deviate from normal behavioral patterns and should be considered suspicious or alarming.

[0380] A "control mechanism" is a function that issues commands to take appropriate action within the system when an abnormality is detected.

[0381] A "location tracking device" is a device that identifies the current location of a moving object, such as a vehicle, and provides location information.

[0382] A "database" is a system that plays a role in securely and efficiently storing and managing information acquired by a system.

[0383] "Biometric information analysis means" refers to technology that analyzes a user's biometric information, such as their voice and gestures, to recognize their emotional state.

[0384] "Emotional state" refers to the user's current psychological or emotional condition.

[0385] "Notification methods" refer to all means and processes used to communicate information or warnings to users.

[0386] In an embodiment of this invention, the monitoring system consists of three elements: a server, a terminal, and a user. The server acquires video data in real time using a monitoring device and detects abnormal behavior using image analysis means. It also has the function of periodically acquiring vehicle location information using a location tracking device and storing it in a database. The hardware used includes a monitoring camera, a location tracking device, and a server computer. The software used is an image and audio analysis system using Python or TensorFlow.

[0387] Meanwhile, the terminal receives warning notifications from the server in real time. At the same time, it analyzes the user's emotional state through biometric data analysis to determine the optimal notification method. Smartphones and tablets are commonly used as this terminal. Through this terminal, users can view the vehicle's location on a map and, if necessary, quickly report the incident to the police or security company. An intuitive interface is provided for this operation.

[0388] As a concrete example, a user could check the security status via a smartphone app when leaving a parking lot. When the user speaks to their smartphone saying, "Is it okay?", the device can analyze the user's anxious tone through emotion analysis and return a response emphasizing that the security system is functioning correctly. In implementing such a system, an example of a prompt message could be: "Detect suspicious activity from the surveillance camera footage. Analyze the user's voice emotion and adjust the notification method according to the urgency."

[0389] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0390] Step 1:

[0391] The server acquires video data from the monitoring device in real time. This data is input in the form of a video stream from the surveillance camera. The server sends this video data to an AI image analysis system to perform processing to detect abnormal behavior. Specifically, a deep learning model using TensorFlow analyzes the movement in the video and compares it with normal behavior patterns. When an anomaly is detected, that information is generated as a warning signal.

[0392] Step 2:

[0393] The server periodically acquires vehicle location information using a location tracking device. This location information is entered as GPS data, and the server stores this data in a database. The data is configured to be updated at specific time intervals, allowing for real-time tracking of location changes.

[0394] Step 3:

[0395] The terminal receives a warning signal from the server in real time. This signal is transmitted to the terminal as an alert when abnormal behavior is detected. The terminal's application prepares to display the notification content to the user and passes the data to the subsequent sentiment analysis process.

[0396] Step 4:

[0397] The device analyzes the user's emotional state using biometric data analysis. The input is voice data acquired through the smartphone's microphone. This voice data is analyzed by a voice emotion analysis model to determine the user's emotional state (e.g., emergency, normal, alert). Based on the output, the device adjusts notification volume, vibration patterns, and other parameters.

[0398] Step 5:

[0399] Users receive analyzed notifications through their devices. For example, if the device is analyzed as being in a high-stress state, it will emphasize voice and vibration and display a message prompting a quick response. Users can also check the vehicle's location on the device screen, allowing them to understand the situation with peace of mind.

[0400] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0401] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0402] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0403] [Third Embodiment]

[0404] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0405] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0406] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0407] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0408] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0409] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0410] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0411] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0412] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0413] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0414] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0415] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0416] As an example of how to implement the present invention, we will describe the design of a high-security parking lot using an AI monitoring system. This system consists of three main elements: a server, a terminal, and a user.

[0417] The server acquires video data in real time from multiple surveillance cameras in the parking lot and uses AI image analysis to detect suspicious behavior. For example, it can instantly recognize a person loitering unnaturally in the parking lot at night or a vehicle exhibiting unusual movements. When the server detects an anomaly, it immediately sends a warning signal and activates sensor lights and sirens to deter any abnormal behavior at that location.

[0418] The terminal receives warnings and notifications from the server in real time and provides them to the user. For example, when abnormal behavior is detected, the terminal sends a notification to the user's smartphone, informing them of the specific nature and location of the abnormality to prompt action. It is also possible to send confirmation feedback of the abnormality to the server through the terminal.

[0419] Users can check the vehicle's security status at any time through the terminal's interface. Furthermore, the vehicle's location information obtained by the location tracking device is displayed on the terminal, allowing users to understand the vehicle's current location and take appropriate action. For example, if a user receives an anomaly notification, they can check the vehicle's location information displayed on the map and quickly notify the police or security company, thereby minimizing theft damage.

[0420] Thus, this invention enhances security in parking lots, prevents vehicle theft, and enables a rapid response in emergencies.

[0421] The following describes the processing flow.

[0422] Step 1:

[0423] The server acquires video data in real time from surveillance cameras installed in the parking lot. It then starts processing the video captured by the surveillance cameras as input data.

[0424] Step 2:

[0425] The server inputs the acquired video data into an AI image analysis system to analyze whether there is any suspicious activity. For example, it can detect prolonged stays within a specific area or the movement of suspicious objects.

[0426] Step 3:

[0427] If the server detects abnormal behavior based on AI analysis, it will issue a warning signal, activate sensor lights in the parking lot, and sound a siren. This creates a deterrent effect against suspicious individuals at the scene.

[0428] Step 4:

[0429] The server sends information about the occurrence of an anomaly to the terminal. This information includes the type of anomaly, the location, and the time it occurred, providing the terminal with a specific alert.

[0430] Step 5:

[0431] The device receives an alert from the server and displays it to the user as a push notification. This notification includes detailed information about the problem, along with a message urging prompt action.

[0432] Step 6:

[0433] The user checks the notification on their device and presses the feedback button if necessary. This action sends feedback to the server indicating that an anomaly has been detected.

[0434] Step 7:

[0435] The server periodically acquires vehicle location information from the location tracking device and continuously monitors for any abnormalities.

[0436] Step 8:

[0437] The device receives location information from the server and displays the vehicle's current location on a map. The user reviews this information and decides whether or not to proceed with actual tracking.

[0438] Step 9:

[0439] Users can check their location and, if necessary, report it to the police or security company. The device interface provides contact information for reporting and supports a quick response.

[0440] (Example 1)

[0441] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0442] Conventional parking lot security systems have difficulty accurately detecting suspicious activity in real time, and also struggle to respond quickly after detection. Therefore, they have been unable to prevent vehicle theft and fraudulent activity. Furthermore, there have been insufficient means for users to easily track the location of their vehicles remotely.

[0443] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0444] In this invention, the server includes digital signal processing means, control means for issuing a warning signal when an anomaly is detected, and means for activating a notification device using light and sound. This enables high-speed and accurate detection of abnormal behavior and prompt issuance of warnings. In addition, it periodically acquires vehicle location information, making it easy for the user to understand the vehicle's current location.

[0445] A "monitoring device" is a device that monitors the environment within a parking lot and acquires image data through digital signal processing.

[0446] "Digital signal processing means" is a general term for methods and devices used to analyze image data and detect abnormal behavior.

[0447] A "warning signal" is a notification signal that is emitted when an abnormality is detected, and its purpose is to inform those around that an abnormality has occurred.

[0448] "Control means" refers to systems or mechanisms that control the transmission of warning signals or the operation of notification devices.

[0449] A "location tracking device" is a device that tracks the movement and current location of a vehicle and provides accurate location information.

[0450] A "storage device" is a device that stores acquired data and allows it to be retrieved as needed.

[0451] A "notification device" is a device that uses light or sound to warn those in the surrounding area when an abnormality is detected.

[0452] A "communication device" is a device or interface used to send and receive data between a server and a terminal.

[0453] A "mobile device" is a device that a user can carry and use to receive warnings and notifications.

[0454] An "information display device" is a device that visually displays the location and movement path of a vehicle.

[0455] This invention is a system for enhancing parking lot security and includes a monitoring device, digital signal processing means, control means, location tracking device, storage device, notification device, communication device, and mobile terminal. Embodiments of this system are described in detail below.

[0456] Server Role

[0457] The server acquires video data in real time from multiple monitoring devices. This process utilizes image processing software as a digital signal processing tool to detect suspicious behavior. Video analysis uses a multi-layer neural network to identify individuals or vehicles exhibiting abnormal behavior. For example, an AI model can be used to identify individuals exhibiting unnatural movements in a parking lot. Furthermore, if an anomaly is detected, the server uses control devices to send a warning signal and activate notification devices.

[0458] Terminal role

[0459] The terminal receives warning notifications sent from the server in real time and informs the user of the occurrence of anomalies. The terminal includes mobile devices, and users can receive anomaly notifications through a smartphone app. For example, the app may display a notification such as "Suspicious activity detected. Location: B3 area." Through the terminal, users can send information about confirmed anomalies as feedback to the server.

[0460] User roles

[0461] Users can manage the vehicle's security status through an application on their device. Furthermore, vehicle location information obtained from the location tracking device is displayed in real time on the device's information display, allowing users to understand the vehicle's current location and respond quickly as needed. For example, upon receiving an anomaly notification, it is recommended to check the vehicle's location on a map and report it to the police.

[0462] Examples of generative AI models and prompts

[0463] This system uses a generative AI model to automatically generate responses under specific conditions. An example of a prompt message is: "Generate a user notification message when suspicious activity is detected in the parking lot. The notification should include the detected area and recommended actions."

[0464] As described above, this invention significantly improves the safety of parking lots and makes it possible to prevent vehicle theft and fraudulent activities.

[0465] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0466] Step 1:

[0467] The server acquires video data in real time from the monitoring device as input. This data serves as the initial input for the program. The server performs digital signal processing frame by frame, applying pre-processing such as noise reduction and motion extraction. This generates high-quality video information. The server then passes the pre-processed data to the next analysis step.

[0468] Step 2:

[0469] The server inputs pre-processed video data into an AI image analysis model. This model includes a generative AI model that detects behavioral patterns indicating abnormal activity. Specifically, it uses a multi-layer neural network to identify the movements of people and vehicles. The analysis yields an anomaly detection output, determining whether or not suspicious behavior is present. The server then uses this result in the next control step.

[0470] Step 3:

[0471] Based on the analysis results of the AI ​​model, the server outputs a warning signal if an anomaly is detected. This signal activates a notification device via a control mechanism. Specifically, it outputs control commands that cause warning lights to flash or sirens to sound. This action aims to deter suspicious individuals.

[0472] Step 4:

[0473] The terminal receives warning notifications from the server as input data. These notifications are output to the user's mobile device in real time. The terminal displays information about the specific nature of the anomaly and its location on the user interface, informing the user visually and audibly. This notification prompts the user to take immediate action.

[0474] Step 5:

[0475] The user receives notifications from their device as input and uses the app's map function to check the vehicle's location. This operation outputs the vehicle's current location and its travel route. If necessary, the user can take concrete action, such as contacting the police or administrators based on the map information. This enables quick problem resolution.

[0476] (Application Example 1)

[0477] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0478] Conventional security systems struggled to detect abnormal activity within monitored areas in real time and respond quickly. Furthermore, there was a lack of means for users to appropriately respond when an anomaly occurred, necessitating enhanced security. Additionally, there was a lack of efficient means to manage and track the location information of objects within monitored areas.

[0479] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0480] In this invention, the server includes a monitoring device, means for acquiring video information within a monitoring area using image analysis means and detecting abnormal operation, a control unit that issues warning information when an abnormality is detected, means for periodically acquiring the location information of objects using a location confirmation device and storing it in an information collection, means for analyzing video within the monitoring area in real time and using an object detection model to identify suspicious operation, and means for transmitting warning information to an information exchange device using a communication system. This makes it possible to quickly and accurately detect abnormal operation within a monitoring area and immediately notify users. In addition, it becomes possible to efficiently manage the location information of objects and make it easy for users to track their movements.

[0481] A "monitoring device" is a device used to acquire video information within a monitored area.

[0482] "Image analysis means" refers to a technology that analyzes acquired video information to detect suspicious or abnormal behavior.

[0483] "Abnormal behavior" refers to suspicious movements or actions that deviate from normal movements.

[0484] "Warning information" refers to information used to notify users when abnormal operation is detected.

[0485] A "control unit" is a device that performs control to issue warning information when abnormal operation is detected.

[0486] A "position confirmation device" is a device that acquires and provides information about the position of an object.

[0487] An "information collection" is a database used to store acquired location information and other data.

[0488] An "object detection model" is a machine learning algorithm used to identify objects and actions in video footage.

[0489] A "communication system" is a general term for network technologies and devices used to exchange information.

[0490] An "information exchange device" is a terminal device used to provide information to users.

[0491] In order to implement the present invention, it is necessary for each component of the monitoring system to work closely together to enhance security within the monitoring area.

[0492] First, the server uses monitoring equipment to acquire video information within the monitored area in real time. The video is analyzed by image analysis tools, and an AI model (such as an object detection algorithm like YOLOv5) is used to identify suspicious or abnormal behavior. The information obtained from this analysis is immediately generated as warning information.

[0493] Next, the control unit, triggered by this warning information, uses the communication system to send a notification to information exchange devices, specifically the user's mobile terminal. This notification includes details of the anomaly and the situation on site, enabling the user to respond quickly.

[0494] Furthermore, the location tracking device periodically acquires location information of objects within the monitoring area and stores it in an information database. This location information is provided to the user via a communication system, making it possible to track the movement of objects in real time.

[0495] As a concrete example, when a user uses the application on their smartphone, if a suspicious person is detected near the entrance at night, a notification is immediately sent. At this point, the user can view the video on the application and take appropriate action, such as contacting a security company or the police.

[0496] By using generative AI models, it is possible to achieve highly accurate identification of suspicious individuals even in unknown situations, significantly improving security levels. To support such systems, image analysis prompts are used in the format of "Analyze the following video data and detect any suspicious human or vehicle movements: [link to video data]".

[0497] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0498] Step 1:

[0499] The server acquires video information from the monitoring device. The input here is the raw video data transmitted from the monitoring device, and the output is a file stored on the server as this video data. This process converts the video stream into a digital format, creating a structure suitable for analysis.

[0500] Step 2:

[0501] The server analyzes the acquired video information using a generating AI model. The input for this step is video data, and the output is information about detected suspicious or abnormal behavior. The AI ​​model (e.g., YOLOv5) is used to identify objects in the video quickly and accurately and to identify suspicious behavior.

[0502] Step 3:

[0503] The server generates warning information based on the detection results of abnormal operation and sends it to the terminal using the communication system. The input to this process is the abnormal operation information from step 2, and the output is the warning information sent to the terminal. In this process, a message with details of the abnormality is created and sent to the user's terminal in real time.

[0504] Step 4:

[0505] The terminal displays the received warning information to the user as a notification. The input for this step is the warning information received from the server, and the output is the notification displayed on the user's screen. This notification typically informs the user of an anomaly as a pop-up on the screen or a sound alert.

[0506] Step 5:

[0507] Users can access detailed information and take appropriate action through their device. The input consists of warning information and real-time video displayed on the device, while the output is the action the user takes (e.g., calling the police). Users check the situation and select the necessary action by operating the response buttons within the app.

[0508] Step 6:

[0509] The location tracking device periodically updates the location information of objects within the monitoring area and stores it in an information database. The input here is the object's current location, and the output is the location information stored in the database. This data is later used for location tracking and made accessible to users.

[0510] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0511] As an embodiment of the present invention, the following example describes a high-security parking lot configuration that combines an AI monitoring system with an emotion engine that detects the user's emotional state and appropriately adjusts alerts and information based on that state. This system includes the emotion engine in addition to three main elements: a server, a terminal, and a user.

[0512] The server acquires video data from surveillance cameras in real time and processes it using AI image analysis. This process detects suspicious movements and behaviors, and if abnormal behavior is confirmed, it issues a warning signal and transmits the information to the terminal. Furthermore, it periodically acquires the vehicle's location information using a location tracking device and records it in a database.

[0513] The device receives alerts from the server and notifies the user. During this process, the emotion engine analyzes the user's biometric information and voice input. For example, when a user gives a voice command, the system analyzes their emotional state and adjusts the alert volume and notification method according to the urgency. If the user is under high stress, the notification is delivered in a faster and clearer manner. Conversely, under normal circumstances, the standard notification settings are maintained.

[0514] Users can receive security status notifications tailored to their emotions through their device. For example, if they receive a notification while in an agitated state, the notification will include detailed instructions and reassuring messages, allowing them to quickly understand the situation and take appropriate action. Furthermore, the device's interface allows users to check the vehicle's location in real time, enabling smooth operation to contact the police or security company if necessary.

[0515] Thus, by combining a monitoring function with an emotion engine, the present invention can provide appropriate responses tailored to the user's psychological state, thereby creating an environment where users can use parking lots with greater peace of mind.

[0516] The following describes the processing flow.

[0517] Step 1:

[0518] The server acquires video data in real time from surveillance cameras installed in the parking lot. The AI ​​image analysis system then starts processing the video captured by the surveillance cameras as input data.

[0519] Step 2:

[0520] The server analyzes suspicious movements and behaviors using AI image analysis to detect abnormal behavior. For example, it may identify prolonged stays in a specific area or objects exhibiting sudden movements as abnormal.

[0521] Step 3:

[0522] If the server detects abnormal behavior, it immediately issues a warning signal, controls the sensor lights in the parking lot to illuminate them, and sounds a siren to alert on-site personnel.

[0523] Step 4:

[0524] The server sends information about the occurrence of an anomaly to the terminal, which includes the type of anomaly detected, the location, and the time of occurrence.

[0525] Step 5:

[0526] The device receives alerts from the server and uses an emotion engine to analyze the user's biometric information and voice input. It then acquires data to determine the user's emotional state.

[0527] Step 6:

[0528] Based on the results analyzed by the emotion engine, the device adjusts how alert notifications are delivered according to the user's emotional state. For example, if the user is agitated, the notification volume may be increased or more detailed explanations may be added.

[0529] Step 7:

[0530] The user checks the notification from the device and follows the instructions provided, contacting the police or security company if necessary. The device displays contact information to support quick reporting.

[0531] Step 8:

[0532] The server constantly monitors the vehicle's location based on data from the location tracking device, continuously checking for any abnormalities. Users can view the vehicle's current location on a map in real time via their device.

[0533] (Example 2)

[0534] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0535] Conventional parking lot monitoring systems were limited to detecting abnormal behavior and lacked the ability to provide appropriate responses that considered the user's psychological state. This meant that notifications received by users were not situation-specific, making prompt responses difficult. Furthermore, there was a lack of efficient means to manage and provide real-time location information of moving objects to users.

[0536] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0537] In this invention, the server includes a monitoring mechanism, means for acquiring video information within the parking lot using an image analysis method and detecting abnormal operation, a control function for issuing a warning signal when an abnormality is detected, means for periodically acquiring the location information of moving objects using location tracking technology and storing it in a storage device, and an emotion analysis function for detecting the user's emotional state and appropriately adjusting the notification method. This enables prompt responses with appropriate notifications and explanations according to the user's psychological state, as well as real-time tracking of the location of moving objects.

[0538] A "monitoring mechanism" refers to a device that includes cameras and sensors installed to acquire video information within the parking lot.

[0539] "Image analysis techniques" refer to algorithms and technologies used to identify suspicious behavior or anomalies from acquired video information.

[0540] "Abnormal behavior" refers to suspicious movements or actions that differ from normal behavior in a parking lot, and should be monitored.

[0541] A "control function" is a mechanism that controls the operation to issue a warning signal based on detected abnormalities.

[0542] "Location tracking technology" refers to technologies and methods for continuously acquiring and tracking the current location of a moving object.

[0543] "Storage device" refers to a database or storage device used to save acquired data and information.

[0544] "Emotional analysis function" is a technology used to analyze a user's psychological state based on their biometric information and voice, and to determine the appropriate notification method.

[0545] "Moving object" refers to vehicles or other moving objects being tracked within a parking lot.

[0546] A "display mechanism" refers to a screen or interface that provides information visually, making it easier for users to access the information.

[0547] This invention is a monitoring system designed to improve parking lot safety and includes three main components: a server, a terminal, and a user. The server acquires video information in real time from surveillance cameras and sensors and processes this information using AI image analysis means. In doing so, it uses software such as OpenCV and TensorFlow to detect suspicious movements and behaviors. When an anomaly is detected, the server generates a warning signal using its control function and transmits this information to the terminal using a secure communication protocol. The server also manages the system by periodically acquiring the location information of moving objects using location tracking technology and storing it in a storage device.

[0548] When the device receives a warning signal from the server, it uses its built-in emotion analysis function to analyze the user's biometric information and voice data. Biosensors and microphones assist in this analysis, and based on the analysis results, the device determines the urgency and adjusts the content and volume of the notification to deliver information in the most appropriate way for the user.

[0549] Users can check notifications and track the location of moving objects in real time through the terminal interface. This allows for quick response measures. If a user detects an anomaly, they can feed that information back to the server via the terminal, providing an opportunity to smoothly notify security services or the police as needed.

[0550] For example, if suspicious activity is detected near a parked vehicle, the server immediately issues a warning, and the terminal adjusts the notification content according to the user's current emotional state. In this process, we utilize prompts input to a generative AI model such as, "We want to design a system that uses AI to identify abnormal behavior in parking lots and notifies users of alerts in an appropriate manner based on their emotional state. Please suggest what algorithms and devices we should use," providing an approach to optimize the effectiveness of notifications.

[0551] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0552] Step 1:

[0553] The server acquires video information in real time from surveillance cameras and sensors. It receives video data from the surveillance devices as input and processes this data using image analysis tools within the server. To identify suspicious movements or behaviors, it analyzes the information using AI algorithms and outputs data labeled as abnormal.

[0554] Step 2:

[0555] The server generates a warning signal based on the abnormal behavior detected by the AI. The input is the anomaly detection data from step 1, and the control function is used to create the warning signal. This signal includes the details and location information of the anomaly and is ready to be sent to the terminal. The output is sending the warning signal to the terminal using a secure communication protocol.

[0556] Step 3:

[0557] The terminal receives a warning signal sent from the server and notifies the user. The input is the warning signal from the server. The built-in emotion analysis function collects the user's biometric information (heart rate, etc.) and voice data and analyzes this information. The output is a notification adjusted according to the urgency level, which is displayed to the user via voice and visuals.

[0558] Step 4:

[0559] Users check notifications through their devices and take action based on detected anomalies. Input is notifications from the device. Output is that users use the provided information to assess the situation and contact security services or the police if necessary. Furthermore, the device allows for real-time location tracking of moving objects.

[0560] Step 5:

[0561] The user sends feedback to the server indicating that the abnormal situation has been resolved. The input is the feedback information from the user. As output, the server receives the feedback, saves it to a recording device, and prepares for the next system warning. This creates data that will be useful for improving the accuracy of future anomaly detection.

[0562] (Application Example 2)

[0563] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0564] While conventional monitoring systems are effective at detecting suspicious behavior and issuing alerts, they have the challenge of not being able to provide optimal responses that take into account the psychological aspects of users. In particular, there is a need to reduce the psychological burden on users in emergencies and to provide accurate information in a calm state.

[0565] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0566] In this invention, the server includes a monitoring device, means for acquiring video data by image analysis means and detecting abnormal behavior, means for detecting the user's emotional state by biometric information analysis means and adjusting the notification method, and means for periodically acquiring vehicle location information by a location tracking device and storing it in a database. This enables appropriate security measures that are in line with the user's emotional state.

[0567] A "monitoring device" is a device consisting of hardware and software that acquires video data in real time and detects abnormal behavior.

[0568] "Image analysis means" refers to AI-based technology that processes acquired video data and detects specific patterns.

[0569] "Abnormal behavior" refers to actions that deviate from normal behavioral patterns and should be considered suspicious or alarming.

[0570] A "control mechanism" is a function that issues commands to take appropriate action within the system when an abnormality is detected.

[0571] A "location tracking device" is a device that identifies the current location of a moving object, such as a vehicle, and provides location information.

[0572] A "database" is a system that plays a role in securely and efficiently storing and managing information acquired by a system.

[0573] "Biometric information analysis means" refers to technology that analyzes a user's biometric information, such as their voice and gestures, to recognize their emotional state.

[0574] "Emotional state" refers to the user's current psychological or emotional condition.

[0575] "Notification methods" refer to all means and processes used to communicate information or warnings to users.

[0576] In an embodiment of this invention, the monitoring system consists of three elements: a server, a terminal, and a user. The server acquires video data in real time using a monitoring device and detects abnormal behavior using image analysis means. It also has the function of periodically acquiring vehicle location information using a location tracking device and storing it in a database. The hardware used includes a monitoring camera, a location tracking device, and a server computer. The software used is an image and audio analysis system using Python or TensorFlow.

[0577] Meanwhile, the terminal receives warning notifications from the server in real time. At the same time, it analyzes the user's emotional state through biometric data analysis to determine the optimal notification method. Smartphones and tablets are commonly used as this terminal. Through this terminal, users can view the vehicle's location on a map and, if necessary, quickly report the incident to the police or security company. An intuitive interface is provided for this operation.

[0578] As a concrete example, a user could check the security status via a smartphone app when leaving a parking lot. When the user speaks to their smartphone saying, "Is it okay?", the device can analyze the user's anxious tone through emotion analysis and return a response emphasizing that the security system is functioning correctly. In implementing such a system, an example of a prompt message could be: "Detect suspicious activity from the surveillance camera footage. Analyze the user's voice emotion and adjust the notification method according to the urgency."

[0579] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0580] Step 1:

[0581] The server acquires video data from the monitoring device in real time. This data is input in the form of a video stream from the surveillance camera. The server sends this video data to an AI image analysis system to perform processing to detect abnormal behavior. Specifically, a deep learning model using TensorFlow analyzes the movement in the video and compares it with normal behavior patterns. When an anomaly is detected, that information is generated as a warning signal.

[0582] Step 2:

[0583] The server periodically acquires vehicle location information using a location tracking device. This location information is entered as GPS data, and the server stores this data in a database. The data is configured to be updated at specific time intervals, allowing for real-time tracking of location changes.

[0584] Step 3:

[0585] The terminal receives a warning signal from the server in real time. This signal is transmitted to the terminal as an alert when abnormal behavior is detected. The terminal's application prepares to display the notification content to the user and passes the data to the subsequent sentiment analysis process.

[0586] Step 4:

[0587] The device analyzes the user's emotional state using biometric data analysis. The input is voice data acquired through the smartphone's microphone. This voice data is analyzed by a voice emotion analysis model to determine the user's emotional state (e.g., emergency, normal, alert). Based on the output, the device adjusts notification volume, vibration patterns, and other parameters.

[0588] Step 5:

[0589] Users receive analyzed notifications through their devices. For example, if the device is analyzed as being in a high-stress state, it will emphasize voice and vibration and display a message prompting a quick response. Users can also check the vehicle's location on the device screen, allowing them to understand the situation with peace of mind.

[0590] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0591] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0592] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0593] [Fourth Embodiment]

[0594] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0595] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0596] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0597] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0598] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0599] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0600] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0601] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0602] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0603] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0604] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0605] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0606] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0607] As an example of how to implement the present invention, we will describe the design of a high-security parking lot using an AI monitoring system. This system consists of three main elements: a server, a terminal, and a user.

[0608] The server acquires video data in real time from multiple surveillance cameras in the parking lot and uses AI image analysis to detect suspicious behavior. For example, it can instantly recognize a person loitering unnaturally in the parking lot at night or a vehicle exhibiting unusual movements. When the server detects an anomaly, it immediately sends a warning signal and activates sensor lights and sirens to deter any abnormal behavior at that location.

[0609] The terminal receives warnings and notifications from the server in real time and provides them to the user. For example, when abnormal behavior is detected, the terminal sends a notification to the user's smartphone, informing them of the specific nature and location of the abnormality to prompt action. It is also possible to send confirmation feedback of the abnormality to the server through the terminal.

[0610] Users can check the vehicle's security status at any time through the terminal's interface. Furthermore, the vehicle's location information obtained by the location tracking device is displayed on the terminal, allowing users to understand the vehicle's current location and take appropriate action. For example, if a user receives an anomaly notification, they can check the vehicle's location information displayed on the map and quickly notify the police or security company, thereby minimizing theft damage.

[0611] Thus, this invention enhances security in parking lots, prevents vehicle theft, and enables a rapid response in emergencies.

[0612] The following describes the processing flow.

[0613] Step 1:

[0614] The server acquires video data in real time from surveillance cameras installed in the parking lot. It then starts processing the video captured by the surveillance cameras as input data.

[0615] Step 2:

[0616] The server inputs the acquired video data into an AI image analysis system to analyze whether there is any suspicious activity. For example, it can detect prolonged stays within a specific area or the movement of suspicious objects.

[0617] Step 3:

[0618] If the server detects abnormal behavior based on AI analysis, it will issue a warning signal, activate sensor lights in the parking lot, and sound a siren. This creates a deterrent effect against suspicious individuals at the scene.

[0619] Step 4:

[0620] The server sends information about the occurrence of an anomaly to the terminal. This information includes the type of anomaly, the location, and the time it occurred, providing the terminal with a specific alert.

[0621] Step 5:

[0622] The device receives an alert from the server and displays it to the user as a push notification. This notification includes detailed information about the problem, along with a message urging prompt action.

[0623] Step 6:

[0624] The user checks the notification on their device and presses the feedback button if necessary. This action sends feedback to the server indicating that an anomaly has been detected.

[0625] Step 7:

[0626] The server periodically acquires vehicle location information from the location tracking device and continuously monitors for any abnormalities.

[0627] Step 8:

[0628] The device receives location information from the server and displays the vehicle's current location on a map. The user reviews this information and decides whether or not to proceed with actual tracking.

[0629] Step 9:

[0630] Users can check their location and, if necessary, report it to the police or security company. The device interface provides contact information for reporting and supports a quick response.

[0631] (Example 1)

[0632] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0633] Conventional parking lot security systems have difficulty accurately detecting suspicious activity in real time, and also struggle to respond quickly after detection. Therefore, they have been unable to prevent vehicle theft and fraudulent activity. Furthermore, there have been insufficient means for users to easily track the location of their vehicles remotely.

[0634] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0635] In this invention, the server includes digital signal processing means, control means for issuing a warning signal when an anomaly is detected, and means for activating a notification device using light and sound. This enables high-speed and accurate detection of abnormal behavior and prompt issuance of warnings. In addition, it periodically acquires vehicle location information, making it easy for the user to understand the vehicle's current location.

[0636] A "monitoring device" is a device that monitors the environment within a parking lot and acquires image data through digital signal processing.

[0637] "Digital signal processing means" is a general term for methods and devices used to analyze image data and detect abnormal behavior.

[0638] A "warning signal" is a notification signal that is emitted when an abnormality is detected, and its purpose is to inform those around that an abnormality has occurred.

[0639] "Control means" refers to systems or mechanisms that control the transmission of warning signals or the operation of notification devices.

[0640] A "location tracking device" is a device that tracks the movement and current location of a vehicle and provides accurate location information.

[0641] A "storage device" is a device that stores acquired data and allows it to be retrieved as needed.

[0642] A "notification device" is a device that uses light or sound to warn those in the surrounding area when an abnormality is detected.

[0643] A "communication device" is a device or interface used to send and receive data between a server and a terminal.

[0644] A "mobile device" is a device that a user can carry and use to receive warnings and notifications.

[0645] An "information display device" is a device that visually displays the location and movement path of a vehicle.

[0646] This invention is a system for enhancing parking lot security and includes a monitoring device, digital signal processing means, control means, location tracking device, storage device, notification device, communication device, and mobile terminal. Embodiments of this system are described in detail below.

[0647] Server Role

[0648] The server acquires video data in real time from multiple monitoring devices. This process utilizes image processing software as a digital signal processing tool to detect suspicious behavior. Video analysis uses a multi-layer neural network to identify individuals or vehicles exhibiting abnormal behavior. For example, an AI model can be used to identify individuals exhibiting unnatural movements in a parking lot. Furthermore, if an anomaly is detected, the server uses control devices to send a warning signal and activate notification devices.

[0649] Terminal role

[0650] The terminal receives warning notifications sent from the server in real time and informs the user of the occurrence of anomalies. The terminal includes mobile devices, and users can receive anomaly notifications through a smartphone app. For example, the app may display a notification such as "Suspicious activity detected. Location: B3 area." Through the terminal, users can send information about confirmed anomalies as feedback to the server.

[0651] User roles

[0652] Users can manage the vehicle's security status through an application on their device. Furthermore, vehicle location information obtained from the location tracking device is displayed in real time on the device's information display, allowing users to understand the vehicle's current location and respond quickly as needed. For example, upon receiving an anomaly notification, it is recommended to check the vehicle's location on a map and report it to the police.

[0653] Examples of generative AI models and prompts

[0654] This system uses a generative AI model to automatically generate responses under specific conditions. An example of a prompt message is: "Generate a user notification message when suspicious activity is detected in the parking lot. The notification should include the detected area and recommended actions."

[0655] As described above, this invention significantly improves the safety of parking lots and makes it possible to prevent vehicle theft and fraudulent activities.

[0656] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0657] Step 1:

[0658] The server acquires video data in real time from the monitoring device as input. This data serves as the initial input for the program. The server performs digital signal processing frame by frame, applying pre-processing such as noise reduction and motion extraction. This generates high-quality video information. The server then passes the pre-processed data to the next analysis step.

[0659] Step 2:

[0660] The server inputs pre-processed video data into an AI image analysis model. This model includes a generative AI model that detects behavioral patterns indicating abnormal activity. Specifically, it uses a multi-layer neural network to identify the movements of people and vehicles. The analysis yields an anomaly detection output, determining whether or not suspicious behavior is present. The server then uses this result in the next control step.

[0661] Step 3:

[0662] Based on the analysis results of the AI ​​model, the server outputs a warning signal if an anomaly is detected. This signal activates a notification device via a control mechanism. Specifically, it outputs control commands that cause warning lights to flash or sirens to sound. This action aims to deter suspicious individuals.

[0663] Step 4:

[0664] The terminal receives warning notifications from the server as input data. These notifications are output to the user's mobile device in real time. The terminal displays information about the specific nature of the anomaly and its location on the user interface, informing the user visually and audibly. This notification prompts the user to take immediate action.

[0665] Step 5:

[0666] The user receives notifications from their device as input and uses the app's map function to check the vehicle's location. This operation outputs the vehicle's current location and its travel route. If necessary, the user can take concrete action, such as contacting the police or administrators based on the map information. This enables quick problem resolution.

[0667] (Application Example 1)

[0668] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0669] Conventional security systems struggled to detect abnormal activity within monitored areas in real time and respond quickly. Furthermore, there was a lack of means for users to appropriately respond when an anomaly occurred, necessitating enhanced security. Additionally, there was a lack of efficient means to manage and track the location information of objects within monitored areas.

[0670] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0671] In this invention, the server includes a monitoring device, means for acquiring video information within a monitoring area using image analysis means and detecting abnormal operation, a control unit that issues warning information when an abnormality is detected, means for periodically acquiring the location information of objects using a location confirmation device and storing it in an information collection, means for analyzing video within the monitoring area in real time and using an object detection model to identify suspicious operation, and means for transmitting warning information to an information exchange device using a communication system. This makes it possible to quickly and accurately detect abnormal operation within a monitoring area and immediately notify users. In addition, it becomes possible to efficiently manage the location information of objects and make it easy for users to track their movements.

[0672] A "monitoring device" is a device used to acquire video information within a monitored area.

[0673] "Image analysis means" refers to a technology that analyzes acquired video information to detect suspicious or abnormal behavior.

[0674] "Abnormal behavior" refers to suspicious movements or actions that deviate from normal movements.

[0675] "Warning information" refers to information used to notify users when abnormal operation is detected.

[0676] A "control unit" is a device that performs control to issue warning information when abnormal operation is detected.

[0677] A "position confirmation device" is a device that acquires and provides information about the position of an object.

[0678] An "information collection" is a database used to store acquired location information and other data.

[0679] An "object detection model" is a machine learning algorithm used to identify objects and actions in video footage.

[0680] A "communication system" is a general term for network technologies and devices used to exchange information.

[0681] An "information exchange device" is a terminal device used to provide information to users.

[0682] In order to implement the present invention, it is necessary for each component of the monitoring system to work closely together to enhance security within the monitoring area.

[0683] First, the server uses monitoring equipment to acquire video information within the monitored area in real time. The video is analyzed by image analysis tools, and an AI model (such as an object detection algorithm like YOLOv5) is used to identify suspicious or abnormal behavior. The information obtained from this analysis is immediately generated as warning information.

[0684] Next, the control unit, triggered by this warning information, uses the communication system to send a notification to information exchange devices, specifically the user's mobile terminal. This notification includes details of the anomaly and the situation on site, enabling the user to respond quickly.

[0685] Furthermore, the location tracking device periodically acquires location information of objects within the monitoring area and stores it in an information database. This location information is provided to the user via a communication system, making it possible to track the movement of objects in real time.

[0686] As a concrete example, when a user uses the application on their smartphone, if a suspicious person is detected near the entrance at night, a notification is immediately sent. At this point, the user can view the video on the application and take appropriate action, such as contacting a security company or the police.

[0687] By using generative AI models, it is possible to achieve highly accurate identification of suspicious individuals even in unknown situations, significantly improving security levels. To support such systems, image analysis prompts are used in the format of "Analyze the following video data and detect any suspicious human or vehicle movements: [link to video data]".

[0688] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0689] Step 1:

[0690] The server acquires video information from the monitoring device. The input here is the raw video data transmitted from the monitoring device, and the output is a file stored on the server as this video data. This process converts the video stream into a digital format, creating a structure suitable for analysis.

[0691] Step 2:

[0692] The server analyzes the acquired video information using a generating AI model. The input for this step is video data, and the output is information about detected suspicious or abnormal behavior. The AI ​​model (e.g., YOLOv5) is used to identify objects in the video quickly and accurately and to identify suspicious behavior.

[0693] Step 3:

[0694] The server generates warning information based on the detection results of abnormal operation and sends it to the terminal using the communication system. The input to this process is the abnormal operation information from step 2, and the output is the warning information sent to the terminal. In this process, a message with details of the abnormality is created and sent to the user's terminal in real time.

[0695] Step 4:

[0696] The terminal displays the received warning information to the user as a notification. The input for this step is the warning information received from the server, and the output is the notification displayed on the user's screen. This notification typically informs the user of an anomaly as a pop-up on the screen or a sound alert.

[0697] Step 5:

[0698] Users can access detailed information and take appropriate action through their device. The input consists of warning information and real-time video displayed on the device, while the output is the action the user takes (e.g., calling the police). Users check the situation and select the necessary action by operating the response buttons within the app.

[0699] Step 6:

[0700] The location tracking device periodically updates the location information of objects within the monitoring area and stores it in an information database. The input here is the object's current location, and the output is the location information stored in the database. This data is later used for location tracking and made accessible to users.

[0701] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0702] As an embodiment of the present invention, the following example describes a high-security parking lot configuration that combines an AI monitoring system with an emotion engine that detects the user's emotional state and appropriately adjusts alerts and information based on that state. This system includes the emotion engine in addition to three main elements: a server, a terminal, and a user.

[0703] The server acquires video data from surveillance cameras in real time and processes it using AI image analysis. This process detects suspicious movements and behaviors, and if abnormal behavior is confirmed, it issues a warning signal and transmits the information to the terminal. Furthermore, it periodically acquires the vehicle's location information using a location tracking device and records it in a database.

[0704] The device receives alerts from the server and notifies the user. During this process, the emotion engine analyzes the user's biometric information and voice input. For example, when a user gives a voice command, the system analyzes their emotional state and adjusts the alert volume and notification method according to the urgency. If the user is under high stress, the notification is delivered in a faster and clearer manner. Conversely, under normal circumstances, the standard notification settings are maintained.

[0705] Users can receive security status notifications tailored to their emotions through their device. For example, if they receive a notification while in an agitated state, the notification will include detailed instructions and reassuring messages, allowing them to quickly understand the situation and take appropriate action. Furthermore, the device's interface allows users to check the vehicle's location in real time, enabling smooth operation to contact the police or security company if necessary.

[0706] Thus, by combining a monitoring function with an emotion engine, the present invention can provide appropriate responses tailored to the user's psychological state, thereby creating an environment where users can use parking lots with greater peace of mind.

[0707] The following describes the processing flow.

[0708] Step 1:

[0709] The server acquires video data in real time from surveillance cameras installed in the parking lot. The AI ​​image analysis system then starts processing the video captured by the surveillance cameras as input data.

[0710] Step 2:

[0711] The server analyzes suspicious movements and behaviors using AI image analysis to detect abnormal behavior. For example, it may identify prolonged stays in a specific area or objects exhibiting sudden movements as abnormal.

[0712] Step 3:

[0713] If the server detects abnormal behavior, it immediately issues a warning signal, controls the sensor lights in the parking lot to illuminate them, and sounds a siren to alert on-site personnel.

[0714] Step 4:

[0715] The server sends information about the occurrence of an anomaly to the terminal, which includes the type of anomaly detected, the location, and the time of occurrence.

[0716] Step 5:

[0717] The device receives alerts from the server and uses an emotion engine to analyze the user's biometric information and voice input. It then acquires data to determine the user's emotional state.

[0718] Step 6:

[0719] Based on the results analyzed by the emotion engine, the device adjusts how alert notifications are delivered according to the user's emotional state. For example, if the user is agitated, the notification volume may be increased or more detailed explanations may be added.

[0720] Step 7:

[0721] The user checks the notification from the device and follows the instructions provided, contacting the police or security company if necessary. The device displays contact information to support quick reporting.

[0722] Step 8:

[0723] The server constantly monitors the vehicle's location based on data from the location tracking device, continuously checking for any abnormalities. Users can view the vehicle's current location on a map in real time via their device.

[0724] (Example 2)

[0725] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0726] Conventional parking lot monitoring systems were limited to detecting abnormal behavior and lacked the ability to provide appropriate responses that considered the user's psychological state. This meant that notifications received by users were not situation-specific, making prompt responses difficult. Furthermore, there was a lack of efficient means to manage and provide real-time location information of moving objects to users.

[0727] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0728] In this invention, the server includes a monitoring mechanism, means for acquiring video information within the parking lot using an image analysis method and detecting abnormal operation, a control function for issuing a warning signal when an abnormality is detected, means for periodically acquiring the location information of moving objects using location tracking technology and storing it in a storage device, and an emotion analysis function for detecting the user's emotional state and appropriately adjusting the notification method. This enables prompt responses with appropriate notifications and explanations according to the user's psychological state, as well as real-time tracking of the location of moving objects.

[0729] A "monitoring mechanism" refers to a device that includes cameras and sensors installed to acquire video information within the parking lot.

[0730] "Image analysis techniques" refer to algorithms and technologies used to identify suspicious behavior or anomalies from acquired video information.

[0731] "Abnormal behavior" refers to suspicious movements or actions that differ from normal behavior in a parking lot, and should be monitored.

[0732] A "control function" is a mechanism that controls the operation to issue a warning signal based on detected abnormalities.

[0733] "Location tracking technology" refers to technologies and methods for continuously acquiring and tracking the current location of a moving object.

[0734] "Storage device" refers to a database or storage device used to save acquired data and information.

[0735] "Emotional analysis function" is a technology used to analyze a user's psychological state based on their biometric information and voice, and to determine the appropriate notification method.

[0736] "Moving object" refers to vehicles or other moving objects being tracked within a parking lot.

[0737] A "display mechanism" refers to a screen or interface that provides information visually, making it easier for users to access the information.

[0738] This invention is a monitoring system designed to improve parking lot safety and includes three main components: a server, a terminal, and a user. The server acquires video information in real time from surveillance cameras and sensors and processes this information using AI image analysis means. In doing so, it uses software such as OpenCV and TensorFlow to detect suspicious movements and behaviors. When an anomaly is detected, the server generates a warning signal using its control function and transmits this information to the terminal using a secure communication protocol. The server also manages the system by periodically acquiring the location information of moving objects using location tracking technology and storing it in a storage device.

[0739] When the device receives a warning signal from the server, it uses its built-in emotion analysis function to analyze the user's biometric information and voice data. Biosensors and microphones assist in this analysis, and based on the analysis results, the device determines the urgency and adjusts the content and volume of the notification to deliver information in the most appropriate way for the user.

[0740] Users can check notifications and track the location of moving objects in real time through the terminal interface. This allows for quick response measures. If a user detects an anomaly, they can feed that information back to the server via the terminal, providing an opportunity to smoothly notify security services or the police as needed.

[0741] For example, if suspicious activity is detected near a parked vehicle, the server immediately issues a warning, and the terminal adjusts the notification content according to the user's current emotional state. In this process, we utilize prompts input to a generative AI model such as, "We want to design a system that uses AI to identify abnormal behavior in parking lots and notifies users of alerts in an appropriate manner based on their emotional state. Please suggest what algorithms and devices we should use," providing an approach to optimize the effectiveness of notifications.

[0742] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0743] Step 1:

[0744] The server acquires video information in real time from surveillance cameras and sensors. It receives video data from the surveillance devices as input and processes this data using image analysis tools within the server. To identify suspicious movements or behaviors, it analyzes the information using AI algorithms and outputs data labeled as abnormal.

[0745] Step 2:

[0746] The server generates a warning signal based on the abnormal behavior detected by the AI. The input is the anomaly detection data from step 1, and the control function is used to create the warning signal. This signal includes the details and location information of the anomaly and is ready to be sent to the terminal. The output is sending the warning signal to the terminal using a secure communication protocol.

[0747] Step 3:

[0748] The terminal receives a warning signal sent from the server and notifies the user. The input is the warning signal from the server. The built-in emotion analysis function collects the user's biometric information (heart rate, etc.) and voice data and analyzes this information. The output is a notification adjusted according to the urgency level, which is displayed to the user via voice and visuals.

[0749] Step 4:

[0750] Users check notifications through their devices and take action based on detected anomalies. Input is notifications from the device. Output is that users use the provided information to assess the situation and contact security services or the police if necessary. Furthermore, the device allows for real-time location tracking of moving objects.

[0751] Step 5:

[0752] The user sends feedback to the server indicating that the abnormal situation has been resolved. The input is the feedback information from the user. As output, the server receives the feedback, saves it to a recording device, and prepares for the next system warning. This creates data that will be useful for improving the accuracy of future anomaly detection.

[0753] (Application Example 2)

[0754] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0755] While conventional monitoring systems are effective at detecting suspicious behavior and issuing alerts, they have the challenge of not being able to provide optimal responses that take into account the psychological aspects of users. In particular, there is a need to reduce the psychological burden on users in emergencies and to provide accurate information in a calm state.

[0756] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0757] In this invention, the server includes a monitoring device, means for acquiring video data by image analysis means and detecting abnormal behavior, means for detecting the user's emotional state by biometric information analysis means and adjusting the notification method, and means for periodically acquiring vehicle location information by a location tracking device and storing it in a database. This enables appropriate security measures that are in line with the user's emotional state.

[0758] A "monitoring device" is a device consisting of hardware and software that acquires video data in real time and detects abnormal behavior.

[0759] "Image analysis means" refers to AI-based technology that processes acquired video data and detects specific patterns.

[0760] "Abnormal behavior" refers to actions that deviate from normal behavioral patterns and should be considered suspicious or alarming.

[0761] A "control mechanism" is a function that issues commands to take appropriate action within the system when an abnormality is detected.

[0762] A "location tracking device" is a device that identifies the current location of a moving object, such as a vehicle, and provides location information.

[0763] A "database" is a system that plays a role in securely and efficiently storing and managing information acquired by a system.

[0764] "Biometric information analysis means" refers to technology that analyzes a user's biometric information, such as their voice and gestures, to recognize their emotional state.

[0765] "Emotional state" refers to the user's current psychological or emotional condition.

[0766] "Notification methods" refer to all means and processes used to communicate information or warnings to users.

[0767] In an embodiment of this invention, the monitoring system consists of three elements: a server, a terminal, and a user. The server acquires video data in real time using a monitoring device and detects abnormal behavior using image analysis means. It also has the function of periodically acquiring vehicle location information using a location tracking device and storing it in a database. The hardware used includes a monitoring camera, a location tracking device, and a server computer. The software used is an image and audio analysis system using Python or TensorFlow.

[0768] Meanwhile, the terminal receives warning notifications from the server in real time. At the same time, it analyzes the user's emotional state through biometric data analysis to determine the optimal notification method. Smartphones and tablets are commonly used as this terminal. Through this terminal, users can view the vehicle's location on a map and, if necessary, quickly report the incident to the police or security company. An intuitive interface is provided for this operation.

[0769] As a concrete example, a user could check the security status via a smartphone app when leaving a parking lot. When the user speaks to their smartphone saying, "Is it okay?", the device can analyze the user's anxious tone through emotion analysis and return a response emphasizing that the security system is functioning correctly. In implementing such a system, an example of a prompt message could be: "Detect suspicious activity from the surveillance camera footage. Analyze the user's voice emotion and adjust the notification method according to the urgency."

[0770] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0771] Step 1:

[0772] The server acquires video data from the monitoring device in real time. This data is input in the form of a video stream from the surveillance camera. The server sends this video data to an AI image analysis system to perform processing to detect abnormal behavior. Specifically, a deep learning model using TensorFlow analyzes the movement in the video and compares it with normal behavior patterns. When an anomaly is detected, that information is generated as a warning signal.

[0773] Step 2:

[0774] The server periodically acquires vehicle location information using a location tracking device. This location information is entered as GPS data, and the server stores this data in a database. The data is configured to be updated at specific time intervals, allowing for real-time tracking of location changes.

[0775] Step 3:

[0776] The terminal receives a warning signal from the server in real time. This signal is transmitted to the terminal as an alert when abnormal behavior is detected. The terminal's application prepares to display the notification content to the user and passes the data to the subsequent sentiment analysis process.

[0777] Step 4:

[0778] The device analyzes the user's emotional state using biometric data analysis. The input is voice data acquired through the smartphone's microphone. This voice data is analyzed by a voice emotion analysis model to determine the user's emotional state (e.g., emergency, normal, alert). Based on the output, the device adjusts notification volume, vibration patterns, and other parameters.

[0779] Step 5:

[0780] Users receive analyzed notifications through their devices. For example, if the device is analyzed as being in a high-stress state, it will emphasize voice and vibration and display a message prompting a quick response. Users can also check the vehicle's location on the device screen, allowing them to understand the situation with peace of mind.

[0781] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0782] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0783] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0784] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0785] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0786] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0787] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0788] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0789] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0790] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0791] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0792] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0793] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0794] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0795] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0796] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0797] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0798] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0799] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0800] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0801] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0802] The following is further disclosed regarding the embodiments described above.

[0803] (Claim 1)

[0804] A monitoring device is provided, and means are used to acquire video data of the parking lot using image analysis means and to detect abnormal behavior.

[0805] A control means that issues a warning signal when an abnormality is detected,

[0806] A means of periodically acquiring vehicle location information using a location tracking device and storing it in a database,

[0807] A system that includes this.

[0808] (Claim 2)

[0809] A means of receiving warning notifications from the server device to the terminal device in real time and notifying the user of the occurrence of an anomaly,

[0810] The system according to claim 1, further comprising means for a user to send feedback to a server indicating that an anomaly has been identified.

[0811] (Claim 3)

[0812] The system according to claim 1, further comprising a display means that visualizes the location information of a vehicle obtained from a location tracking device on a map, enabling a user to track the movement of the vehicle.

[0813] "Example 1"

[0814] (Claim 1)

[0815] A monitoring device is provided, and means for acquiring image data within the parking lot using digital signal processing means and detecting abnormal behavior,

[0816] A control means that issues a warning signal when an abnormality is detected,

[0817] A means for periodically acquiring vehicle location information using a location tracking device and storing it in a storage device,

[0818] A means of activating a notification device using light and sound when a warning signal is issued,

[0819] A system that includes this.

[0820] (Claim 2)

[0821] A means of receiving warning notifications from communication devices on mobile terminals in real time and informing users of the occurrence of an anomaly,

[0822] The system according to claim 1, further comprising means for a user to transmit information confirming an anomaly to a communication device.

[0823] (Claim 3)

[0824] The system according to claim 1, further comprising a display means that visualizes vehicle location information obtained from a location tracking device on an information display device, enabling a user to track the movement of the vehicle.

[0825] "Application Example 1"

[0826] (Claim 1)

[0827] A monitoring device is provided, and means are used to acquire video information within the monitoring area using image analysis means and to detect abnormal operation.

[0828] A control unit that issues warning information when an abnormality is detected,

[0829] A means for periodically acquiring the position information of an object using a position confirmation device and storing it in an information collection,

[0830] A means of using an object detection model to analyze video footage within a surveillance area in real time and identify suspicious activity,

[0831] A means of transmitting warning information to an information exchange device using a communication system,

[0832] A system that includes this.

[0833] (Claim 2)

[0834] A means of receiving warning information from a communication system in real time to information exchange equipment and notifying users of the occurrence of an anomaly,

[0835] The system according to claim 1, further comprising means for a user to transmit an abnormality confirmation response to a communication system.

[0836] (Claim 3)

[0837] The system according to claim 1, further comprising a display means that visualizes the location information of an object obtained from a location confirmation device on a map, enabling a user to track the movement of the object.

[0838] "Example 2 of combining an emotion engine"

[0839] (Claim 1)

[0840] A monitoring mechanism is provided, which acquires video information within the parking lot using image analysis techniques and has means to detect abnormal operation.

[0841] A control function that issues a warning signal when an abnormality is detected,

[0842] A means for periodically acquiring the location information of a moving object using location tracking technology and storing it in a storage device,

[0843] A sentiment analysis function that detects the user's emotional state and adjusts the notification method appropriately,

[0844] A system that includes this.

[0845] (Claim 2)

[0846] A means of receiving warning notifications from the server device to the terminal device in real time and notifying the user of the occurrence of an anomaly,

[0847] A means of analyzing the user's biometric information and voice using emotion analysis functionality and adjusting notifications according to urgency,

[0848] The system according to claim 1, further comprising means for a user to send feedback to a server indicating that an anomaly has been identified.

[0849] (Claim 3)

[0850] The system according to claim 1, comprising a display mechanism that visualizes the location information of a moving object obtained from location tracking technology on a map, enabling the user to track the movement of the moving object.

[0851] "Application example 2 when combining with an emotional engine"

[0852] (Claim 1)

[0853] A monitoring device is provided, and means are used to acquire video data of the parking lot using image analysis means and to detect abnormal behavior.

[0854] A control means that issues a warning signal when an abnormality is detected,

[0855] A means of periodically acquiring vehicle location information using a location tracking device and storing it in a database,

[0856] A means for detecting the user's emotional state using biometric information analysis and adjusting the notification method,

[0857] A system that includes this.

[0858] (Claim 2)

[0859] A means of receiving warning notifications from the server device to the terminal device in real time and notifying the user of the occurrence of an anomaly,

[0860] A means of adjusting the volume and notification method of alerts based on the user's emotional state,

[0861] The system according to claim 1, further comprising means for a user to send feedback to a server indicating that an anomaly has been identified.

[0862] (Claim 3)

[0863] The system has a display means that visualizes the vehicle's location information obtained from a location tracking device on a map, allowing the user to track the vehicle's movement.

[0864] The system according to claim 1, further comprising means for personalizing security notification content according to the user's emotional state. [Explanation of Symbols]

[0865] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A monitoring device is provided, and means are used to acquire video information within the monitoring area using image analysis means and to detect abnormal operation. A control unit that issues warning information when an abnormality is detected, A means for periodically acquiring the position information of an object using a position confirmation device and storing it in an information collection, A means of using an object detection model to analyze video footage within a surveillance area in real time and identify suspicious activity, A means of transmitting warning information to an information exchange device using a communication system, A system that includes this.

2. A means of receiving warning information from a communication system in real time to information exchange equipment and notifying users of the occurrence of an anomaly, The system according to claim 1, further comprising means for a user to transmit an abnormality confirmation response to a communication system.

3. The system according to claim 1, further comprising a display means that visualizes the location information of an object obtained from a location confirmation device on a map, enabling a user to track the movement of the object.

Images