System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting image data in real time from a camera installed in a traffic signal, a public building, or a vehicle; means for analyzing the image data and extracting feature information of the vehicle; means for collating the feature information with a crime database and a vehicle inspection database stored in advance; and means for notifying police or a fire department when an abnormality is detected.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Automobile crimes (theft, illegal modifications, forged license plate numbers, hit-and-runs, etc.) have been increasing in recent years, creating a need for rapid and accurate detection of these crimes and for reporting them to the relevant authorities. However, current surveillance systems face the problem of difficulty in detecting criminal vehicles and illegal acts in real time and responding appropriately. In particular, there is a need for constant surveillance over a wide area and immediate analysis of information provided by users, but no system exists that can achieve this. The present invention aims to solve these problems and enable rapid and effective responses to automobile crimes. [Means for solving the problem]

[0005] This invention provides a means for collecting image data in real time from cameras installed at traffic lights, public buildings, and vehicles; a means for analyzing the image data to extract vehicle characteristics; a means for comparing the characteristics with pre-stored crime databases and vehicle inspection databases; and a means for notifying the police or fire department if an abnormality is detected. It also provides a means for receiving and analyzing image data captured by users, which is also compared with the crime database and includes a function for immediately notifying the police or fire department if an abnormality is detected. This allows the system to achieve both continuous monitoring and ad hoc information provision, enabling rapid and accurate response. In particular, connecting the cameras to devices equipped with artificial intelligence and automatically analyzing the data can significantly improve the accuracy and speed of crime detection.

[0006] A "camera" is a device that has the function of capturing images and saving or transmitting them as image data.

[0007] "Image data" is data that represents in digital form images captured by a camera.

[0008] "Real-time" refers to data being processed and analyzed as it is generated.

[0009] "Analysis" is the process of analyzing acquired data and extracting useful information from it.

[0010] "Vehicle characteristic information" is data including vehicle type, license plate, color, shape, and other identifiable vehicle attribute information.

[0011] A "crime database" is a database that accumulates information about crimes that have occurred in the past and is used for investigations and analysis.

[0012] A "vehicle inspection database" is a database that stores information about vehicles obtained through official inspections such as vehicle inspections.

[0013] "Matching" is the process of comparing acquired data with existing database information.

[0014] "Abnormal" refers to a state that is not normal or out of the ordinary, and specifically indicates suspicion of vehicle theft or illegal activity.

[0015] "Notification" is the act of conveying information to relevant authorities when an abnormality is detected.

[0016] "Police" is a public institution that protects public safety and enforces the law.

[0017] A "fire department" is a public institution that provides fire protection and emergency medical care.

[0018] "User" means any person or entity that uses the System to provide or receive information. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention relates to a system that detects automobile crimes by collecting image data using cameras installed on traffic lights, public buildings, and vehicles, and analyzing the collected image data in real time, and then notifies the relevant authorities. Below, we will create a program for this system and explain its processing with specific examples.

[0041] Implementation of continuous monitoring

[0042] 1. Data Collection

[0043] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[0044] For example, a camera attached to a traffic light at an intersection captures video of passing vehicles.

[0045] 2. Image Preprocessing

[0046] The server performs pre-processing on the received image data, including noise removal and frame-by-frame image segmentation.

[0047] This process removes unnecessary information from the image data, making it suitable for analysis.

[0048] 3. Feature Extraction

[0049] The server extracts vehicle feature information from the preprocessed image data, specifically identifying the vehicle model, license plate number, and driving situation using object detection algorithms (e.g., YOLO, SSD) and OCR technology.

[0050] For example, the license plate number of a particular vehicle can be read from an image captured by a camera and the information stored in a database.

[0051] 4. Database Verification

[0052] The server compares the extracted vehicle license plates with a criminal database and a vehicle inspection information database, and this comparison detects any anomalies.

[0053] For example, it checks whether the license plate being read is on a list of stolen vehicles.

[0054] 5. Anomaly detection and notification

[0055] If the server detects an abnormality based on the comparison results, it notifies the police or fire department, including the vehicle's characteristics and location information.

[0056] For example, if a stolen vehicle is detected, the information and current location will be sent to the police in real time.

[0057] This series of processes enables a rapid response when a car crime is detected during road surveillance.

[0058] Emergency confirmation implementation

[0059] 1. Data Reception

[0060] The server receives videos and photos taken by users via email or messaging apps.

[0061] For example, a user may take a video of a hit-and-run incident and send the video to a server via email.

[0062] 2. Data storage

[0063] The server temporarily stores the received data, including metadata such as sender information and the date and time of receipt.

[0064] 3. Image Preprocessing

[0065] The server preprocesses the stored data, making each frame individually analyzable, by removing noise and adjusting the resolution.

[0066] 4. Feature Extraction

[0067] The server extracts vehicle feature information from the pre-processed data in real time, again using object detection algorithms and OCR technology.

[0068] 5. Urgent Analysis and Notification

[0069] The server compares the extracted characteristic information with a crime database, and if it detects an abnormality, it notifies the police or fire department of the results.

[0070] For example, if the license plate of a photographed vehicle is found to be on a list of hit-and-run vehicles, the information will be immediately notified to the police.

[0071] By using the above means, the present invention is a system that realizes early detection and rapid response to automobile crimes through both constant monitoring and emergency confirmation.

[0072] The processing flow will be explained below.

[0073] Continuous monitoring process steps

[0074] Step 1:

[0075] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[0076] Step 2:

[0077] The server performs preprocessing on the received image data. In this step, it removes noise from the image and divides the image into frames. This process removes unnecessary information from the image data and makes it suitable for analysis.

[0078] Step 3:

[0079] The server extracts vehicle feature information from the preprocessed image data, using object detection algorithms (e.g., YOLO, SSD) and OCR technology to identify the vehicle type, license plate number, and driving situation. For example, it reads the license plate number of a specific vehicle from the image captured by the camera.

[0080] Step 4:

[0081] The server compares the extracted vehicle license plate information with a criminal database and a vehicle inspection information database, and this comparison detects any anomalies.

[0082] Step 5:

[0083] The server determines an anomaly based on the matching result, for example, if a license plate matching a stolen vehicle list is detected.

[0084] Step 6:

[0085] If an abnormality is detected, the server immediately notifies the police or fire department. The notification includes vehicle characteristics (such as license plate, model, and location information), allowing the relevant authorities to respond quickly.

[0086] Processing steps for urgent confirmation

[0087] Step 1:

[0088] The server receives videos or photos taken by users via email or messaging apps. For example, a user may take a video of a hit-and-run accident and send the video via email.

[0089] Step 2:

[0090] The server temporarily stores the received data, which also includes metadata such as sender information and the date and time of receipt.

[0091] Step 3:

[0092] The server preprocesses the stored data, which involves removing noise and adjusting the resolution so that each frame can be analyzed individually.

[0093] Step 4:

[0094] The server extracts vehicle feature information from the pre-processed data in real time, again using object detection algorithms and OCR technology.

[0095] Step 5:

[0096] The server compares the extracted feature information with a crime database to detect anomalies, such as when the license plate of the photographed vehicle is included in a list of hit-and-run vehicles.

[0097] Step 6:

[0098] If the server detects an abnormality, it immediately notifies the police or fire department. The notification includes analyzed characteristic information and sender information, allowing the relevant authorities to respond quickly.

[0099] Example 1

[0100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] In modern society, improving the safety and efficiency of transportation infrastructure is an important issue. Rapid detection and response to vehicle crimes is particularly important, but conventional surveillance systems have struggled to accurately detect abnormalities in real time and notify relevant authorities. Furthermore, there is a lack of means for users to efficiently analyze data captured in emergency situations and respond quickly. To address these issues, a system that combines advanced image processing and database matching technologies is needed.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0103] In this invention, the server includes means for collecting visible data in real time from optical devices installed on traffic lights, public buildings, and vehicles, means for preprocessing the visible data to make it analyzable, means for extracting characteristic information about the vehicle from the visible data, means for comparing the characteristic information with a pre-stored database, and means for notifying relevant authorities when an abnormality is detected, thereby enabling early detection and rapid response to automobile crimes.

[0104] The server also includes a means for receiving visible data captured by a user, a means for preprocessing the received visible data to make it analyzable, a means for extracting characteristic information of the vehicle from the visible data, a means for comparing the characteristic information with a pre-stored database, a means for notifying relevant organizations when an abnormality is detected, and a means for managing the analysis results sent to the relevant organizations by the notifying means, thereby enabling emergency information from users to be processed quickly and accurately.

[0105] Furthermore, by connecting the optical equipment to a device equipped with a machine learning algorithm and having the means to automatically analyze the visible data, high-precision analysis can be performed in real time, improving the accuracy of detecting vehicle crimes and enabling rapid response.

[0106] "Optical equipment" refers to cameras and sensors used to acquire image and video data.

[0107] "Visible data" refers to image data or video data acquired by optical instruments, and includes visual information to be analyzed.

[0108] "Preprocessing" refers to a series of processing operations performed to prepare visible data for analysis, including noise removal, image segmentation, and resolution adjustment.

[0109] "Feature information" refers to significant information extracted from visible data, specifically vehicle model, license plate number, driving conditions, etc.

[0110] A "database" is a computer system that systematically stores information for later retrieval or collation, and includes crime databases and vehicle inspection databases.

[0111] A "machine learning algorithm" is an algorithm that automatically learns patterns and rules from data and uses that knowledge to analyze new data.

[0112] "Relevant agencies" are agencies to which notification will be sent if an abnormality such as a car crime is detected, specifically the police and fire department.

[0113] "Means of notification" refers to the communication method used to notify relevant authorities of an abnormality, and includes SMS, email, voice calls, etc.

[0114] "Receiving means" refers to the method or function by which the server receives the visible data sent by the user.

[0115] "Management means" refers to the method or function for recording and storing the analysis results sent to the relevant authorities by the notification means, and for use in future reference and analysis.

[0116] This invention is a system that collects visible data in real time from optical devices installed on traffic lights, public buildings, and vehicles, preprocesses this data to extract vehicle characteristic information, compares it with a crime database, and notifies relevant authorities if an abnormality is detected. The following hardware and software are used to implement this invention.

[0117] Hardware and software used

[0118] 1. Optical equipment

[0119] Cameras and sensors are used as optical instruments. For example, surveillance cameras are installed to capture data in real time 24 hours a day, 365 days a year.

[0120] 2. Server

[0121] The server preprocesses the received visible data, extracts feature information, compares it with a database, and notifies users when an anomaly is detected. The following software is used on the server:

[0122] "OpenCV": Used for preprocessing the visual data, which includes denoising and frame-by-frame segmentation.

[0123] "YOLO" or "SSD": Used as an object detection algorithm to extract vehicle feature information from visual data.

[0124] "Tesseract OCR": Uses text recognition technology to analyze the characters on license plates.

[0125] "Elasticsearch" or "MySQL": Used to match the extracted feature information with the database.

[0126] "Twilio API": Used to notify relevant authorities (e.g., police and fire department) when an anomaly is detected.

[0127] 3. Communication Network

[0128] A network for transmitting data from cameras and sensors to a server. Data is transferred via the Internet or a dedicated data transfer network.

[0129] System operation example

[0130] For example, consider a surveillance camera installed at a traffic light at an intersection. This camera captures video data of passing vehicles in real time. This data is then sent to a server via a communication network.

[0131] 1. Data Collection

[0132] The server collects visible data received from optical devices, in this case video data transmitted from surveillance cameras.

[0133] 2. Image Preprocessing

[0134] The server preprocesses the collected visual data, for example, using OpenCV to perform image processing such as noise removal and frame segmentation, thereby creating data suitable for analysis.

[0135] 3. Feature Extraction

[0136] The server uses object detection algorithms such as YOLO and SSD to extract vehicle feature information from the preprocessed data, then uses Tesseract OCR to read the license plate characters and store them in a database.

[0137] 4. Database Verification

[0138] The server compares the extracted feature information with databases stored in Elasticsearch or MySQL, thereby detecting anomalies by comparing them with criminal databases and vehicle inspection information databases.

[0139] 5. Anomaly detection and notification

[0140] If the server detects an abnormality, it will use the Twilio API to notify the police, fire department, and other relevant authorities. This notification will include information about the vehicle's characteristics and current location, enabling a prompt response.

[0141] Prompt Sentence Examples

[0142] Prompt: Please explain the processing procedure of this system in natural language. Please specify what hardware or software is used to process or calculate data. Please also specify the specific name of the hardware or software used.

[0143] 1. The server receives real-time visual data from optical devices installed on traffic lights, public buildings, and vehicles, for example, dedicated surveillance cameras.

[0144] 2. The server preprocesses the received visual data. In this step, it uses OpenCV to remove noise and segment the image for each frame.

[0145] 3. The server extracts vehicle feature information from the preprocessed visual data using object detection algorithms such as YOLO and SSD, as well as Tesseract OCR.

[0146] 4. The server compares the extracted vehicle feature information with the database, using Elasticsearch and MySQL for this comparison.

[0147] 5. If the server detects an anomaly based on the matching results, it notifies the relevant authorities using the "Twilio API."

[0148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0149] Program processing steps

[0150] Step 1:

[0151] The server collects visible data in real time from optical devices installed in traffic lights, public buildings, and vehicles. The input data from these optical devices includes images and videos. The input data is stored as is and then moved on to the next processing step. Specifically, surveillance cameras capture image data in real time 24 hours a day, 365 days a year, and send it to the server via the Internet. The received data is then stored in the server's storage.

[0152] Step 2:

[0153] The server preprocesses the collected visual data to make it ready for analysis. This process uses image processing techniques to remove noise and segment each frame. The input is the image data collected in step 1, and the output is the preprocessed image data. Specific examples include using OpenCV to remove noise from the image data and adjust the image resolution as needed. This ensures that the data used for analysis is of high quality.

[0154] Step 3:

[0155] The server extracts vehicle feature information from the preprocessed visual data. This process uses an object detection algorithm to identify features such as the vehicle model and license plate. The input is the image data preprocessed in step 2, and the output is vehicle feature information. Specifically, vehicles are detected using "YOLO" or "SSD," and text information from license plates is extracted using "Tesseract OCR." The extracted information is temporarily stored in the server's memory.

[0156] Step 4:

[0157] The server compares the extracted vehicle characteristics information with a database. This comparison checks for matching data in criminal databases or vehicle inspection databases. The input is the vehicle characteristics information extracted in step 3, and the output is the results of an anomaly inspection. For example, a query is run against a database stored in Elasticsearch or MySQL to match license plates and vehicle characteristics.

[0158] Step 5:

[0159] If the server detects an abnormality based on the matching results, it will take measures to notify the relevant authorities. The notification content includes vehicle characteristics and location information. The input is the abnormality inspection results obtained in step 4, and the output is the notified information. Specifically, it uses the "Twilio API" to send real-time notifications to the police and fire department via email, SMS, voice calls, etc. It may also use the "Google Maps API" to obtain location information, allowing the relevant authorities to respond quickly.

[0160] Step 6:

[0161] A user takes a photo of an accident scene and sends the image data to the server. This data is sent to the server via email or a messaging app. The input is the image data sent by the user, and the output is the data received by the server. For example, this includes the user taking a photo of the accident scene and sending the image to the server via email.

[0162] Step 7:

[0163] The server preprocesses the image data received from the user to make it ready for analysis. The input is the image data received in step 6, and the output is the preprocessed image data. As a specific example, "OpenCV" is used as part of the preprocessing to remove noise and divide the image frames.

[0164] Step 8:

[0165] The server extracts vehicle feature information from the preprocessed data. The input is the image data preprocessed in step 7, and the output is the vehicle feature information. Specific operations include detecting vehicles using the YOLO model and reading license plates using Tesseract OCR.

[0166] Step 9:

[0167] The server compares the extracted feature information with the database and notifies the relevant authorities if an abnormality is detected. The input is the vehicle feature information extracted in step 8, and the output is the notification result. As a specific example, a query is executed against the database, and if an abnormality is detected, the relevant authorities are notified using the "Twilio API." This process enables a fast and accurate response even in emergencies.

[0168] (Application example 1)

[0169] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0170] The problem that this invention aims to solve is to detect automobile crimes in parking lots and on roads in real time and respond promptly and appropriately. Conventional technologies have had the problem that it takes time from the time a crime occurs until it is reported, making it difficult to respond quickly. In addition, there was a lack of means to efficiently identify vehicles illegally entering parking lots or stolen vehicles, which placed a heavy burden on managers and police.

[0171] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0172] In this invention, the server includes means for collecting image data in real time from imaging devices installed in traffic signals, public facilities, and vehicles, means for analyzing the image data and extracting characteristic information about the vehicle, means for comparing the characteristic information with a pre-stored crime database and vehicle inspection database, means for notifying an administrator and the police when an abnormality is detected, and means for installing a predetermined application on a smart device and displaying images and notifications in real time when an abnormality is detected, thereby enabling fast and efficient detection and management of automobile crimes in parking lots and on roads.

[0173] A "signal device" is a device that emits a signal, and includes traffic lights and the like for controlling traffic flow.

[0174] "Public facilities" are facilities that are available to the general public, such as city halls, schools, and train stations.

[0175] An "imaging device" is a device for capturing images or videos, and includes cameras and video cameras.

[0176] "Real-time" refers to processing or responding to an event as it occurs.

[0177] "Image data" refers to digital data of images or videos captured by an imaging device.

[0178] "Characteristic information" is information that indicates specific attributes or characteristics extracted from analyzed data, such as license plates and vehicle models.

[0179] A "crime database" is a database that stores information related to past crimes.

[0180] A "vehicle inspection database" is a database that stores vehicle registration information and inspection history.

[0181] "Matching" refers to comparing one piece of data with another to see if they match.

[0182] "Abnormal" refers to an unusual or unexpected condition, such as the discovery of a stolen vehicle.

[0183] "Notification" means the communication of specific information to another person.

[0184] "Administrator" refers to a person responsible for operating and monitoring a system or facility.

[0185] "Police" means an agency that enforces the law to maintain public order and safety.

[0186] "Smart devices" refer to electronic devices with advanced functions, including smartphones and head-mounted displays.

[0187] An "application" is a software program that provides a particular function or service.

[0188] The system for implementing this invention collects image data in real time from imaging devices installed in traffic signals, public facilities, and vehicles, analyzes the image data to extract characteristic information about the vehicle, compares it with a crime database and a vehicle inspection database, and notifies administrators and the police if an abnormality is detected. It also includes displaying this information in real time on a smart device with a specified application installed.

[0189] Hardware

[0190] The system includes the following hardware:

[0191] Imaging devices (cameras): Installed on traffic signals, public facilities, and vehicles.

[0192] Server (data processing server): Receives image data in real time and performs analysis.

[0193] Smart Devices: Includes smartphones and head-mounted displays (HMDs), which receive notifications and display footage in real time.

[0194] software

[0195] The software used is as follows:

[0196] YOLOv5: Used as an object detection algorithm.

[0197] OpenCV: A library for image processing.

[0198] Requests: A library for making HTTP requests.

[0199] Python: The primary development language in which the entire program is written.

[0200] Data processing and calculation

[0201] After receiving image data from the imaging device in real time, the server performs the following processes:

[0202] 1. Preprocessing of image data: noise removal and resolution adjustment.

[0203] 2. Feature extraction: YOLOv5 is used to extract vehicle features, specifically identifying license plates and vehicle types.

[0204] 3. Database matching: The extracted feature information is matched with the crime database and vehicle inspection database to detect suspicious vehicles.

[0205] 4. Notification: If an abnormality is detected, a notification will be sent to the smart device and detailed information will be forwarded to the administrator and police.

[0206] Example

[0207] As an example, we provide a parking lot security monitor. When the parking lot manager launches the "Parking Lot Security Monitor" app on their smartphone, real-time video footage of the parking lot is displayed. If a suspicious vehicle is detected, details are displayed in the smartphone's notification center. In addition, by using a head-mounted display, the situation inside the parking lot can be directly viewed.

[0208] Prompt Sentence Examples

[0209] "Create a Python program that uses YOLOv5 to extract vehicle characteristics from parking lot surveillance cameras in real time, and notify the manager and police if there are any abnormalities based on the matching results."

[0210] The system enables fast and efficient detection and management of car crime in parking lots and on roads.

[0211] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0212] Step 1:

[0213] The server collects image data in real time from image capture devices installed on traffic signals, public facilities, and vehicles. As input, it receives image data transmitted from each image capture device. As output, it stores the raw image data in memory. This data is used for subsequent processing. Specifically, the server retrieves data from the URL of the camera stream.

[0214] Step 2:

[0215] The server preprocesses the received image data. It uses raw image data as input. It gets preprocessed image data as output. This preprocessing includes noise removal and resolution adjustment. Specifically, it uses the OpenCV library to remove noise and adjust the resolution of the image.

[0216] Step 3:

[0217] The server extracts vehicle feature information from the preprocessed image data. The preprocessed image data is used as input. Feature information such as license plates and vehicle types is obtained as output. Specifically, object detection is performed using YOLOv5, and license plate recognition is performed.

[0218] Step 4:

[0219] The server compares the extracted feature information with the pre-stored crime database and vehicle inspection database. The feature information and database information are used as input. The comparison result is obtained as output. Specifically, the server sends an HTTP request to query the database.

[0220] Step 5:

[0221] If the server detects an anomaly based on the matching results, it notifies the administrator and the police. It uses the matching results as input, and generates and sends a notification message as output. Specifically, it implements a function to send an alert via the notification system.

[0222] Step 6:

[0223] When an anomaly is detected, the server installs a specific application on the smart device and displays images and notifications in real time. The server uses the anomaly detection information as input. The server sends a notification to the smart device as output. Specifically, the server displays an alert in the notification center of the smartphone or head-mounted display.

[0224] Step 7:

[0225] The server manages the collected and processed data. It uses feature information, matching results, and notification history as input. It obtains analysis results as output, which are stored in a database. Specific operations include recording all data to a central server, making it available for later analysis and review.

[0226] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0227] This invention relates to a system that detects automobile crimes and notifies relevant authorities by combining an emotion engine with a system that optimizes alertness and response based on the user's emotion information. Below, we will create a program for this system and explain its processing with specific examples.

[0228] Implementation of continuous monitoring

[0229] 1. Data Collection

[0230] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[0231] For example, a camera attached to a traffic light at an intersection captures video of passing vehicles.

[0232] 2. Image Preprocessing

[0233] The server performs pre-processing on the received image data, including noise removal and frame-by-frame image segmentation.

[0234] This removes unnecessary information from the image data, making it suitable for analysis.

[0235] 3. Feature Extraction

[0236] The server extracts vehicle feature information from the pre-processed image data, using object detection algorithms (e.g., YOLO, SSD) and OCR technology to identify the vehicle model, license plate number, and driving situation.

[0237] For example, the license plate number of a particular vehicle can be read from an image captured by a camera.

[0238] 4. Database Verification

[0239] The server compares the extracted vehicle license plate information with the crime database and vehicle inspection information database, and any anomalies are detected through the comparison process.

[0240] For example, it checks whether the license plate being read is on a list of stolen vehicles.

[0241] 5. Anomaly Detection

[0242] The server determines an anomaly based on the matching result, for example, if a license plate matching a stolen vehicle list is detected.

[0243] 6. Notice and Information

[0244] If an abnormality is detected, the server notifies the police or fire department, and the notification includes information about the vehicle's characteristics and location.

[0245] For example, if a stolen vehicle is detected, the information and current location will be sent to the police in real time.

[0246] Emergency confirmation implementation

[0247] 1. Data Reception

[0248] The server receives videos and photos taken by users via email or messaging apps, and simultaneously collects their emotional information.

[0249] For example, a user can film the scene of a hit-and-run accident and send the video along with emotional information from that time via email.

[0250] 2. Data storage

[0251] The server temporarily stores the received data, including metadata such as sender information and the date and time of receipt.

[0252] 3. Image Preprocessing

[0253] The server preprocesses the stored data, which involves removing noise and adjusting the resolution, making each frame individually analyzable.

[0254] 4. Feature Extraction

[0255] The server extracts vehicle feature information from the preprocessed data in real time using object detection algorithms and OCR technology, and the emotion engine analyzes the user's emotion information.

[0256] For example, the license plate of a photographed vehicle is analyzed, and at the same time, the user's emotional information is analyzed.

[0257] 5. Emergency analysis

[0258] The server compares the extracted feature information with a criminal database to detect abnormalities, prioritizes the analysis results based on the user's emotional information, and determines how to respond based on the level of urgency.

[0259] For example, if the license plate of a photographed vehicle matches a list of hit-and-run vehicles, and the user feels extreme shock or fear, an immediate alert will be sent to the police.

[0260] 6. Notification of Results and Response

[0261] If the server detects an abnormality, it immediately notifies the police or fire department. The notification includes analyzed feature information and the user's emotional information. The urgency and detail of the notification are adjusted based on the emotional information.

[0262] For example, if more detailed information or a faster response is required based on the user's emotional information, this will be communicated to the police.

[0263] The system of this invention achieves early detection and rapid response to car crimes by using a dual approach of continuous monitoring and emergency confirmation. Furthermore, by combining it with an emotion engine, the urgency and detail of the analysis results are optimized, aiming to enable faster and more appropriate responses from relevant authorities.

[0264] The processing flow will be explained below.

[0265] Continuous monitoring process steps

[0266] Step 1:

[0267] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[0268] Step 2:

[0269] The server performs preprocessing on the received image data. In this step, it removes noise from the image and divides the image into frames. This process removes unnecessary information from the image data and makes it suitable for analysis.

[0270] Step 3:

[0271] The server extracts vehicle feature information from the preprocessed image data, using object detection algorithms (e.g., YOLO, SSD) and OCR technology to identify the vehicle type, license plate number, and driving situation. For example, it reads the license plate number of a specific vehicle from the image captured by the camera.

[0272] Step 4:

[0273] The server compares the extracted vehicle license plate information with a criminal database and a vehicle inspection information database, and any anomalies are detected through this comparison.

[0274] Step 5:

[0275] The server determines an anomaly based on the matching result, for example, if a license plate matching a stolen vehicle list is detected.

[0276] Step 6:

[0277] If an abnormality is detected, the server immediately notifies the police or fire department. The notification includes vehicle characteristics (such as license plate, model, and location information), allowing the relevant authorities to respond quickly.

[0278] Processing steps for urgent confirmation

[0279] Step 1:

[0280] The server receives videos or photos taken by users via email or messaging apps, and simultaneously collects the user's emotional information. For example, a user may take a photo of a hit-and-run accident scene and send the video and their emotional information via email.

[0281] Step 2:

[0282] The server temporarily stores the received data, including metadata such as sender information and the date and time of receipt.

[0283] Step 3:

[0284] The server preprocesses the stored data, a step that involves removing noise and adjusting resolution so that each frame can be analyzed individually.

[0285] Step 4:

[0286] The server extracts vehicle feature information from the preprocessed data in real time, again using object detection algorithms and OCR technology. The emotion engine then analyzes the user's emotional information. For example, the server analyzes the license plate of the vehicle in the photo and simultaneously analyzes the user's emotional information.

[0287] Step 5:

[0288] The server compares the extracted feature information with a crime database to detect any abnormalities. It prioritizes the analysis results based on the user's emotional information and determines how to respond based on the level of urgency. For example, if the license plate of a photographed vehicle matches a list of hit-and-run vehicles, and the user feels extreme shock or fear, it will immediately send a report to the police.

[0289] Step 6:

[0290] If an abnormality is detected, the server immediately notifies the police or fire department. The notification content includes analyzed feature information and the user's emotional information. The urgency and level of detail of the notification are adjusted based on the emotional information. For example, if more detailed information or a faster response is required based on the user's emotional information, this information is communicated to the police. The system of the present invention achieves early detection and rapid response of car crimes using a dual approach of continuous monitoring and emergency confirmation. Furthermore, by combining an emotion engine, the urgency and level of detail of the analysis results are optimized, aiming to enable faster and more appropriate responses from relevant authorities.

[0291] Example 2

[0292] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0293] Conventional vehicle crime monitoring systems are capable of collecting and analyzing image data in real time, but lack the ability to prioritize emergency responses based on user emotional information. This makes it difficult for relevant authorities to respond appropriately and quickly when a crime is detected. Furthermore, when analyzing data based on emergency calls from users, it is difficult to determine the reliability and urgency of the data, which can cause delays in situations where a rapid response is required. A new system is needed to solve these problems.

[0294] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting image data in real time from sensors installed on traffic lights, public buildings, and vehicles, means for preprocessing the image data and extracting vehicle feature information, means for comparing the feature information with a pre-stored database, means for collecting user emotion information and setting the urgency level of the analysis results, and means for notifying relevant authorities when an abnormality is detected. This enables relevant authorities to respond appropriately and quickly when a crime is detected.

[0295] A "traffic light" is a device installed to control traffic on a road and displays signals to vehicles and pedestrians.

[0296] A "public building" is a building used for public purposes and functions as an administrative agency or public facility.

[0297] A "sensor" is a device that detects physical changes and outputs them as electrical signals.

[0298] "Image data" is a digital representation of optical information acquired by a camera or sensor.

[0299] "Preprocessing" refers to processing carried out before data analysis, such as removing noise and shaping data.

[0300] "Vehicle characteristic information" is information necessary for identifying and classifying vehicles, and includes license plates, vehicle models, colors, etc.

[0301] A "database" is a systematically organized collection of data that allows for efficient data retrieval and updating.

[0302] "User emotion information" is digital data that expresses the emotion felt by the user in a particular situation.

[0303] The "urgency of the analysis results" is an index that indicates the priority and urgency of the response based on the analyzed data.

[0304] "Related agencies" refers to administrative agencies and public facilities that respond to crimes and emergencies, including police and fire departments.

[0305] This invention is a surveillance system for detecting automobile crimes and notifying relevant authorities, with the aim of optimizing the level of vigilance and the urgency of response by using user emotional information in particular. This system detects crimes by collecting and analyzing data from cameras and sensors installed in traffic lights, public buildings, and vehicles. Furthermore, it analyzes emotional information based on emergency calls from users and notifies relevant authorities based on that information.

[0306] Hardware and Software

[0307] Hardware

[0308] Cameras and sensors: installed on traffic lights, public buildings, and vehicles. They collect image data in real time.

[0309] Server: A high-performance computing device for data collection, pre-processing, analysis, and notification.

[0310] Cloud storage: Used for temporary data storage and backup.

[0311] software

[0312] Image analysis algorithms: Object detection algorithms (YOLO, SSD, etc.) and OCR technology are used.

[0313] Databases: Crime database and vehicle inspection database.

[0314] Emotion engine: Used to analyze the user's emotional information and set the level of urgency.

[0315] Notification system: Communication software that notifies relevant authorities when an abnormality is detected.

[0316] Data flow

[0317] Data collection

[0318] The server collects image data in real time from cameras and sensors installed in traffic lights, public buildings, and vehicles. For example, a camera installed at an intersection captures images of a specific vehicle and sends the data to the server.

[0319] Pretreatment

[0320] The server preprocesses the received image data. In this step, noise is removed and the image is divided into frames to prepare it for analysis. For example, unnecessary parts are removed from the captured image to extract only the vehicle.

[0321] Analysis and Feature Extraction

[0322] The server extracts vehicle feature information from the preprocessed image data, using object detection algorithms such as YOLO, SSD, and OCR technology, analyzing features such as license plates, car models, and colors.

[0323] Database Matching

[0324] The server compares the extracted feature information with a criminal database and a vehicle inspection database, allowing it to detect anomalies, such as whether the license plate is included in a list of stolen vehicles.

[0325] Sentiment analysis and urgency setting

[0326] The server collects and analyzes emotional information along with video and image data based on emergency calls from users. The emotion engine sets the urgency level of the analysis results based on the user's emotional information. For example, if the user feels extreme shock or fear, the urgency level is set high.

[0327] notification

[0328] If an abnormality is detected, the server notifies the relevant authorities. The notification content includes vehicle characteristics, location information, and user emotion information. For example, if a stolen vehicle is detected, the police will be notified in real time along with its location information.

[0329] Specific examples

[0330] For example, if a camera installed at an intersection detects what appears to be a stolen vehicle, the image data is sent to a server. The server preprocesses the image, extracts features, and compares them with a database. If it matches a stolen vehicle, the server immediately notifies the police. Also, if a user witnesses a hit-and-run incident, the video is sent to the server along with emotional information. The server uses an emotion engine to set the urgency of the analysis results to a high level and promptly notify the police.

[0331] Prompt Sentence Examples

[0332] Here are some examples of prompts to input to the generative AI model:

[0333] "Please explain the process of a system that detects hit-and-run incidents and notifies the police based on user-recorded vehicle video and its emotional information."

[0334] This invention realizes early detection and rapid response to car crimes using a dual approach of continuous monitoring and emergency confirmation. In addition, the introduction of an emotion engine allows for faster and more appropriate notification to relevant authorities, contributing to improved safety for society as a whole.

[0335] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0336] Step 1: Data collection

[0337] Input: Real-time image data captured by cameras and sensors.

[0338] How it works: The server collects image data in real time from cameras and sensors installed on traffic lights, public buildings, and vehicles. This data is sent to the server via the internet or dedicated lines.

[0339] Output: Raw image data stored in the server storage.

[0340] Step 2: Image preprocessing

[0341] Input: Raw image data stored in storage.

[0342] How it works: The server applies a noise reduction filter to the received image data, splits the image into frames, cuts out the background and unnecessary parts, and formats the image in a way that is suitable for analysis.

[0343] Output: Preprocessed and clean image data.

[0344] Step 3: Feature extraction

[0345] Input: Preprocessed and clean image data.

[0346] How it works: The server uses an object detection algorithm (e.g., YOLO, SSD) to identify vehicles and OCR technology to read license plates, specifically identifying the make, color, and number plate characters in the image.

[0347] Output: Vehicle characteristics information (model, color, license plate, etc.).

[0348] Step 4: Database Matching

[0349] Input: Extracted vehicle feature information.

[0350] Operation: The server compares the extracted vehicle characteristics with the crime database and vehicle inspection database to determine whether the vehicle is a stolen or illegal vehicle.

[0351] Output: Matching result (whether anomalies are found or not).

[0352] Step 5: Anomaly detection

[0353] Input: Match results obtained from database matching.

[0354] How it works: The server determines whether something is an anomaly based on the match. For example, if the license plate matches a list of stolen vehicles, it will flag it as an anomaly.

[0355] Output: Anomaly detection flag and its detailed information.

[0356] Step 6: Notification and Information

[0357] Input: Anomaly detection flag and its details.

[0358] Operation: If an abnormality is detected, the server notifies the relevant authorities (police or fire department). The notification includes vehicle characteristics and location information. For example, the location information of a stolen vehicle is sent to the police in real time.

[0359] Output: Notification information sent to the appropriate authorities.

[0360] ---

[0361] The specific operation of each step is clarified and the processing flow from input to output is shown, making it easier to understand the operation of the entire system.

[0362] (Application example 2)

[0363] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0364] While early detection and rapid response to vehicle crimes are required, current systems are unable to consider the user's emotional state or level of urgency. This can result in a lack of accuracy in crime detection and the effectiveness of emergency notifications. Furthermore, there is no way to effectively utilize information provided by users via direct photography, making it difficult to accurately detect crimes in real time and respond quickly.

[0365] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting image data in real time from cameras installed on traffic lights, public buildings, and vehicles, means for analyzing the image data and extracting characteristic information about the vehicle, means for comparing the characteristic information with a pre-stored crime database and vehicle inspection database, means for notifying the police or fire department when an abnormality is detected, means for a user to capture video of the surroundings using a mobile communication device and collect emotional information, and means for adjusting the urgency of the analysis results based on the emotional information of the user. This makes it possible to adjust the urgency taking into account the emotional state of the user, thereby achieving a faster and more appropriate response.

[0366] A "traffic light" is a device installed to control traffic flow and issue instructions to vehicles and pedestrians.

[0367] "Public building" means a building open to the public, including government agencies and facilities providing public services.

[0368] "Vehicle" means a mechanical device, usually an engine-powered means of transportation, designed to transport people or goods.

[0369] A "camera" is an optical device for capturing video or images, and includes a lens and an image sensor.

[0370] "Image data" is data that digitally represents visual information acquired by a device such as a camera.

[0371] "User" refers to an individual or entity that operates a system or device and provides information.

[0372] "Mobile communication device" means a portable device that can send and receive data using wireless communication, including smartphones and tablets.

[0373] "Emotional information" is data that represents an individual's emotional state and is information collected based on voice analysis and facial expression recognition.

[0374] A "crime database" is a database for recording and managing information on crimes that have occurred in the past.

[0375] The "vehicle inspection database" is a database that stores and manages vehicle inspection history and registration information.

[0376] "Artificial intelligence" refers to computer systems that mimic or replicate human intelligence, including software that performs data analysis and pattern recognition.

[0377] "Analysis Results" refers to the conclusions or information obtained after processing and analyzing collected data.

[0378] "Police" means a government agency responsible for maintaining public safety and enforcing the law.

[0379] "Fire department" refers to a public institution that carries out fire prevention and extinguishing and emergency rescue activities.

[0380] "Notification" is the act or process of making specific information known to another person or system.

[0381] "Real-time" refers to a situation where data and information are processed and analyzed immediately, and results are provided without delay.

[0382] The present invention relates to a system for detecting motor vehicle crime and notifying appropriate authorities, optimizing vigilance and response based on the emotional state of the user. Specific embodiments for implementing the invention are described in detail herein.

[0383] System Configuration

[0384] The system of the present invention consists of the following main components:

[0385] 1. Data Collection Module

[0386] Cameras: Capture image data in real time from cameras installed on traffic lights, public buildings, and vehicles.

[0387] Mobile communication device: The user uses a mobile communication device (smartphone) to capture images of the surroundings and collect emotional information.

[0388] GPS: Location information is obtained using the GPS module built into the mobile communication device.

[0389] 2. Data Processing Module

[0390] Image preprocessing: Using a library such as OpenCV, image data acquired from the camera is denoised and segmented into individual frames.

[0391] Feature extraction: Object detection is performed using TensorFlow to extract vehicle feature information, and Tesseract is used for OCR processing to read license plate information.

[0392] Emotion analysis: Using artificial intelligence such as Watson Tone Analyzer, emotional information is analyzed from the voice data provided by the user.

[0393] 3. Database Matching Module

[0394] Database matching: The extracted vehicle feature information is matched with pre-stored crime databases and vehicle inspection databases.

[0395] 4. Notification Module

[0396] Notification system: Uses Firebase Cloud Messaging to notify police and fire departments when an abnormality is detected.

[0397] Example

[0398] For example, when a user takes a photo of a suspicious vehicle with a mobile communication device, the video and audio data are sent to a server. The server first extracts the vehicle's characteristics from the video and determines whether it is a target vehicle. At the same time, it analyzes the user's emotional information from the audio data and adjusts the alert level as necessary. If an abnormality is detected, the server immediately notifies the police or fire department and provides the vehicle's characteristics, location, and the user's emotional information.

[0399] Examples of prompts include:

[0400] If you notice a suspicious vehicle nearby, capture video and audio of the vehicle using your smartphone camera and microphone. The application will detect signs of crime from the video and audio and quickly notify the appropriate authorities based on the user's emotional state.

[0401] In this way, the present invention provides a system that effectively utilizes the video and emotional information provided by the user and optimizes the level of urgency and detail, thereby enabling a quick and appropriate response.

[0402] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0403] Step 1: Data collection

[0404] The user takes a video of the surroundings using the camera on their mobile communication device (smartphone). The camera captures video data in real time, and simultaneously collects audio data using the microphone. Location information is also obtained using the GPS module. The input data consists of video data, audio data, and location information. This data is sent to the server and used for subsequent analysis.

[0405] Step 2: Image preprocessing

[0406] The server performs preprocessing on the received video data. In preprocessing, OpenCV is used to remove noise and segment the image frame by frame. The input is the video data acquired in step 1, and the output is image data with noise removed and suitable for analysis. This process ensures that the subsequent feature extraction process can be performed accurately and efficiently.

[0407] Step 3: Feature extraction

[0408] The server extracts vehicle feature information from the preprocessed image data. It uses TensorFlow to perform object detection and identify the vehicle's position and shape. It also uses Tesseract to perform OCR processing of license plates. The input is the preprocessed image data, and the output is vehicle feature information and license plate information. This allows for vehicle identification.

[0409] Step 4: Sentiment Analysis

[0410] The server analyzes the voice data collected by the user when taking a photo. It uses artificial intelligence such as Watson Tone Analyzer to obtain the user's emotional information. The input is the voice data obtained in step 1, and the output is the analyzed emotional information of the user. This allows the emotion the user was feeling at the time to be detected.

[0411] Step 5: Database Matching

[0412] The server compares the extracted vehicle characteristic information and license plate information with the crime database and vehicle inspection database. This comparison determines whether the vehicle is a stolen or illegal vehicle. The input is the vehicle characteristic information and license plate information obtained in step 3, and the output is the comparison result. This allows anomalies to be detected.

[0413] Step 6: Adjusting urgency

[0414] The server integrates the matching results with the user's emotional information and adjusts the urgency of the anomaly. If the urgency is high based on the emotional information, it increases the priority of notification to the police and fire department. The input is the emotional information acquired in step 4 and the matching results obtained in step 5, and the output is the adjusted urgency information. This optimizes the urgency.

[0415] Step 7: Notification

[0416] The server notifies the police and fire department of the adjusted urgency information and details of the abnormality. It uses a notification system such as Firebase Cloud Messaging to transmit the information to the relevant agencies in real time. The input is the urgency information and details of the abnormality obtained in step 6, and the output is a notification message to the police and fire department. This enables a rapid response.

[0417] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0418] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0419] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0420] [Second embodiment]

[0421] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0422] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0423] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0424] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0425] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0426] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0427] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0428] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0429] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0430] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0431] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0432] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0433] The present invention relates to a system that detects automobile crimes by collecting image data using cameras installed on traffic lights, public buildings, and vehicles, and analyzing the collected image data in real time, and then notifies the relevant authorities. Below, we will create a program for this system and explain its processing with specific examples.

[0434] Implementation of continuous monitoring

[0435] 1. Data Collection

[0436] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[0437] For example, a camera attached to a traffic light at an intersection captures video of passing vehicles.

[0438] 2. Image Preprocessing

[0439] The server performs pre-processing on the received image data, including noise removal and frame-by-frame image segmentation.

[0440] This process removes unnecessary information from the image data, making it suitable for analysis.

[0441] 3. Feature Extraction

[0442] The server extracts vehicle feature information from the preprocessed image data, specifically identifying the vehicle model, license plate number, and driving situation using object detection algorithms (e.g., YOLO, SSD) and OCR technology.

[0443] For example, the license plate number of a particular vehicle can be read from an image captured by a camera and the information stored in a database.

[0444] 4. Database Verification

[0445] The server compares the extracted vehicle license plates with a criminal database and a vehicle inspection information database, and this comparison detects any anomalies.

[0446] For example, it checks whether the license plate being read is on a list of stolen vehicles.

[0447] 5. Anomaly detection and notification

[0448] If the server detects an abnormality based on the comparison results, it notifies the police or fire department, including the vehicle's characteristics and location information.

[0449] For example, if a stolen vehicle is detected, the information and current location will be sent to the police in real time.

[0450] This series of processes enables a rapid response when a car crime is detected during road surveillance.

[0451] Emergency confirmation implementation

[0452] 1. Data Reception

[0453] The server receives videos and photos taken by users via email or messaging apps.

[0454] For example, a user may take a video of a hit-and-run incident and send the video to a server via email.

[0455] 2. Data storage

[0456] The server temporarily stores the received data, including metadata such as sender information and the date and time of receipt.

[0457] 3. Image Preprocessing

[0458] The server preprocesses the stored data, making each frame individually analyzable, by removing noise and adjusting the resolution.

[0459] 4. Feature Extraction

[0460] The server extracts vehicle feature information from the pre-processed data in real time, again using object detection algorithms and OCR technology.

[0461] 5. Urgent Analysis and Notification

[0462] The server compares the extracted characteristic information with a crime database, and if it detects an abnormality, it notifies the police or fire department of the results.

[0463] For example, if the license plate of a photographed vehicle is found to be on a list of hit-and-run vehicles, the information will be immediately notified to the police.

[0464] By using the above means, the present invention is a system that realizes early detection and rapid response to automobile crimes through both constant monitoring and emergency confirmation.

[0465] The processing flow will be explained below.

[0466] Continuous monitoring process steps

[0467] Step 1:

[0468] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[0469] Step 2:

[0470] The server performs preprocessing on the received image data. In this step, it removes noise from the image and divides the image into frames. This process removes unnecessary information from the image data and makes it suitable for analysis.

[0471] Step 3:

[0472] The server extracts vehicle feature information from the preprocessed image data, using object detection algorithms (e.g., YOLO, SSD) and OCR technology to identify the vehicle type, license plate number, and driving situation. For example, it reads the license plate number of a specific vehicle from the image captured by the camera.

[0473] Step 4:

[0474] The server compares the extracted vehicle license plate information with a criminal database and a vehicle inspection information database, and this comparison detects any anomalies.

[0475] Step 5:

[0476] The server determines an anomaly based on the matching result, for example, if a license plate matching a stolen vehicle list is detected.

[0477] Step 6:

[0478] If an abnormality is detected, the server immediately notifies the police or fire department. The notification includes vehicle characteristics (such as license plate, model, and location information), allowing the relevant authorities to respond quickly.

[0479] Processing steps for urgent confirmation

[0480] Step 1:

[0481] The server receives videos or photos taken by users via email or messaging apps. For example, a user may take a video of a hit-and-run accident and send the video via email.

[0482] Step 2:

[0483] The server temporarily stores the received data, which also includes metadata such as sender information and the date and time of receipt.

[0484] Step 3:

[0485] The server preprocesses the stored data, which involves removing noise and adjusting the resolution so that each frame can be analyzed individually.

[0486] Step 4:

[0487] The server extracts vehicle feature information from the pre-processed data in real time, again using object detection algorithms and OCR technology.

[0488] Step 5:

[0489] The server compares the extracted feature information with a crime database to detect anomalies, such as when the license plate of the photographed vehicle is included in a list of hit-and-run vehicles.

[0490] Step 6:

[0491] If the server detects an abnormality, it immediately notifies the police or fire department. The notification includes analyzed characteristic information and sender information, allowing the relevant authorities to respond quickly.

[0492] Example 1

[0493] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0494] In modern society, improving the safety and efficiency of transportation infrastructure is an important issue. Rapid detection and response to vehicle crimes is particularly important, but conventional surveillance systems have struggled to accurately detect abnormalities in real time and notify relevant authorities. Furthermore, there is a lack of means for users to efficiently analyze data captured in emergency situations and respond quickly. To address these issues, a system that combines advanced image processing and database matching technologies is needed.

[0495] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0496] In this invention, the server includes means for collecting visible data in real time from optical devices installed on traffic lights, public buildings, and vehicles, means for preprocessing the visible data to make it analyzable, means for extracting characteristic information about the vehicle from the visible data, means for comparing the characteristic information with a pre-stored database, and means for notifying relevant authorities when an abnormality is detected, thereby enabling early detection and rapid response to automobile crimes.

[0497] The server also includes a means for receiving visible data captured by a user, a means for preprocessing the received visible data to make it analyzable, a means for extracting characteristic information of the vehicle from the visible data, a means for comparing the characteristic information with a pre-stored database, a means for notifying relevant organizations when an abnormality is detected, and a means for managing the analysis results sent to the relevant organizations by the notifying means, thereby enabling emergency information from users to be processed quickly and accurately.

[0498] Furthermore, by connecting the optical equipment to a device equipped with a machine learning algorithm and having the means to automatically analyze the visible data, high-precision analysis can be performed in real time, improving the accuracy of detecting vehicle crimes and enabling rapid response.

[0499] "Optical equipment" refers to cameras and sensors used to acquire image and video data.

[0500] "Visible data" refers to image data or video data acquired by optical instruments, and includes visual information to be analyzed.

[0501] "Preprocessing" refers to a series of processing operations performed to prepare visible data for analysis, including noise removal, image segmentation, and resolution adjustment.

[0502] "Feature information" refers to significant information extracted from visible data, specifically vehicle model, license plate number, driving conditions, etc.

[0503] A "database" is a computer system that systematically stores information for later retrieval or collation, and includes crime databases and vehicle inspection databases.

[0504] A "machine learning algorithm" is an algorithm that automatically learns patterns and rules from data and uses that knowledge to analyze new data.

[0505] "Relevant agencies" are agencies to which notification will be sent if an abnormality such as a car crime is detected, specifically the police and fire department.

[0506] "Means of notification" refers to the communication method used to notify relevant authorities of an abnormality, and includes SMS, email, voice calls, etc.

[0507] "Receiving means" refers to the method or function by which the server receives the visible data sent by the user.

[0508] "Management means" refers to the method or function for recording and storing the analysis results sent to the relevant authorities by the notification means, and for use in future reference and analysis.

[0509] This invention is a system that collects visible data in real time from optical devices installed on traffic lights, public buildings, and vehicles, preprocesses this data to extract vehicle characteristic information, compares it with a crime database, and notifies relevant authorities if an abnormality is detected. The following hardware and software are used to implement this invention.

[0510] Hardware and software used

[0511] 1. Optical equipment

[0512] Cameras and sensors are used as optical instruments. For example, surveillance cameras are installed to capture data in real time 24 hours a day, 365 days a year.

[0513] 2. Server

[0514] The server preprocesses the received visible data, extracts feature information, compares it with a database, and notifies users when an anomaly is detected. The following software is used on the server:

[0515] "OpenCV": Used for preprocessing the visual data, which includes denoising and frame-by-frame segmentation.

[0516] "YOLO" or "SSD": Used as an object detection algorithm to extract vehicle feature information from visual data.

[0517] "Tesseract OCR": Uses text recognition technology to analyze the characters on license plates.

[0518] "Elasticsearch" or "MySQL": Used to match the extracted feature information with the database.

[0519] "Twilio API": Used to notify relevant authorities (e.g., police and fire department) when an anomaly is detected.

[0520] 3. Communication Network

[0521] A network for transmitting data from cameras and sensors to a server. Data is transferred via the Internet or a dedicated data transfer network.

[0522] System operation example

[0523] For example, consider a surveillance camera installed at a traffic light at an intersection. This camera captures video data of passing vehicles in real time. This data is then sent to a server via a communication network.

[0524] 1. Data Collection

[0525] The server collects visible data received from optical devices, in this case video data transmitted from surveillance cameras.

[0526] 2. Image Preprocessing

[0527] The server preprocesses the collected visual data, for example, using OpenCV to perform image processing such as noise removal and frame segmentation, thereby creating data suitable for analysis.

[0528] 3. Feature Extraction

[0529] The server uses object detection algorithms such as YOLO and SSD to extract vehicle feature information from the preprocessed data, then uses Tesseract OCR to read the license plate characters and store them in a database.

[0530] 4. Database Verification

[0531] The server compares the extracted feature information with databases stored in Elasticsearch or MySQL, thereby detecting anomalies by comparing them with criminal databases and vehicle inspection information databases.

[0532] 5. Anomaly detection and notification

[0533] If the server detects an abnormality, it will use the Twilio API to notify the police, fire department, and other relevant authorities. This notification will include information about the vehicle's characteristics and current location, enabling a prompt response.

[0534] Prompt Sentence Examples

[0535] Prompt: Please explain the processing procedure of this system in natural language. Please specify what hardware or software is used to process or calculate data. Please also specify the specific name of the hardware or software used.

[0536] 1. The server receives real-time visual data from optical devices installed on traffic lights, public buildings, and vehicles, for example, dedicated surveillance cameras.

[0537] 2. The server preprocesses the received visual data. In this step, it uses OpenCV to remove noise and segment the image for each frame.

[0538] 3. The server extracts vehicle feature information from the preprocessed visual data using object detection algorithms such as YOLO and SSD, as well as Tesseract OCR.

[0539] 4. The server compares the extracted vehicle feature information with the database, using Elasticsearch and MySQL for this comparison.

[0540] 5. If the server detects an anomaly based on the matching results, it notifies the relevant authorities using the "Twilio API."

[0541] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0542] Program processing steps

[0543] Step 1:

[0544] The server collects visible data in real time from optical devices installed in traffic lights, public buildings, and vehicles. The input data from these optical devices includes images and videos. The input data is stored as is and then moved on to the next processing step. Specifically, surveillance cameras capture image data in real time 24 hours a day, 365 days a year, and send it to the server via the Internet. The received data is then stored in the server's storage.

[0545] Step 2:

[0546] The server preprocesses the collected visual data to make it ready for analysis. This process uses image processing techniques to remove noise and segment each frame. The input is the image data collected in step 1, and the output is the preprocessed image data. Specific examples include using OpenCV to remove noise from the image data and adjust the image resolution as needed. This ensures that the data used for analysis is of high quality.

[0547] Step 3:

[0548] The server extracts vehicle feature information from the preprocessed visual data. This process uses an object detection algorithm to identify features such as the vehicle model and license plate. The input is the image data preprocessed in step 2, and the output is vehicle feature information. Specifically, vehicles are detected using "YOLO" or "SSD," and text information from license plates is extracted using "Tesseract OCR." The extracted information is temporarily stored in the server's memory.

[0549] Step 4:

[0550] The server compares the extracted vehicle characteristics information with a database. This comparison checks for matching data in criminal databases or vehicle inspection databases. The input is the vehicle characteristics information extracted in step 3, and the output is the results of an anomaly inspection. For example, a query is run against a database stored in Elasticsearch or MySQL to match license plates and vehicle characteristics.

[0551] Step 5:

[0552] If the server detects an abnormality based on the matching results, it will take measures to notify the relevant authorities. The notification content includes vehicle characteristics and location information. The input is the abnormality inspection results obtained in step 4, and the output is the notified information. Specifically, it uses the "Twilio API" to send real-time notifications to the police and fire department via email, SMS, voice calls, etc. It may also use the "Google Maps API" to obtain location information, allowing the relevant authorities to respond quickly.

[0553] Step 6:

[0554] A user takes a photo of an accident scene and sends the image data to the server. This data is sent to the server via email or a messaging app. The input is the image data sent by the user, and the output is the data received by the server. For example, this includes the user taking a photo of the accident scene and sending the image to the server via email.

[0555] Step 7:

[0556] The server preprocesses the image data received from the user to make it ready for analysis. The input is the image data received in step 6, and the output is the preprocessed image data. As a specific example, "OpenCV" is used as part of the preprocessing to remove noise and divide the image frames.

[0557] Step 8:

[0558] The server extracts vehicle feature information from the preprocessed data. The input is the image data preprocessed in step 7, and the output is the vehicle feature information. Specific operations include detecting vehicles using the YOLO model and reading license plates using Tesseract OCR.

[0559] Step 9:

[0560] The server compares the extracted feature information with the database and notifies the relevant authorities if an abnormality is detected. The input is the vehicle feature information extracted in step 8, and the output is the notification result. As a specific example, a query is executed against the database, and if an abnormality is detected, the relevant authorities are notified using the "Twilio API." This process enables a fast and accurate response even in emergencies.

[0561] (Application example 1)

[0562] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0563] The problem that this invention aims to solve is to detect automobile crimes in parking lots and on roads in real time and respond promptly and appropriately. Conventional technologies have had the problem that it takes time from the time a crime occurs until it is reported, making it difficult to respond quickly. In addition, there was a lack of means to efficiently identify vehicles illegally entering parking lots or stolen vehicles, which placed a heavy burden on managers and police.

[0564] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0565] In this invention, the server includes means for collecting image data in real time from imaging devices installed in traffic signals, public facilities, and vehicles, means for analyzing the image data and extracting characteristic information about the vehicle, means for comparing the characteristic information with a pre-stored crime database and vehicle inspection database, means for notifying an administrator and the police when an abnormality is detected, and means for installing a predetermined application on a smart device and displaying images and notifications in real time when an abnormality is detected, thereby enabling fast and efficient detection and management of automobile crimes in parking lots and on roads.

[0566] A "signal device" is a device that emits a signal, and includes traffic lights and the like for controlling traffic flow.

[0567] "Public facilities" are facilities that are available to the general public, such as city halls, schools, and train stations.

[0568] An "imaging device" is a device for capturing images or videos, and includes cameras and video cameras.

[0569] "Real-time" refers to processing or responding to an event as it occurs.

[0570] "Image data" refers to digital data of images or videos captured by an imaging device.

[0571] "Characteristic information" is information that indicates specific attributes or characteristics extracted from analyzed data, such as license plates and vehicle models.

[0572] A "crime database" is a database that stores information related to past crimes.

[0573] A "vehicle inspection database" is a database that stores vehicle registration information and inspection history.

[0574] "Matching" refers to comparing one piece of data with another to see if they match.

[0575] "Abnormal" refers to an unusual or unexpected condition, such as the discovery of a stolen vehicle.

[0576] "Notification" means the communication of specific information to another person.

[0577] "Administrator" refers to a person responsible for operating and monitoring a system or facility.

[0578] "Police" means an agency that enforces the law to maintain public order and safety.

[0579] "Smart devices" refer to electronic devices with advanced functions, including smartphones and head-mounted displays.

[0580] An "application" is a software program that provides a particular function or service.

[0581] The system for implementing this invention collects image data in real time from imaging devices installed in traffic signals, public facilities, and vehicles, analyzes the image data to extract characteristic information about the vehicle, compares it with a crime database and a vehicle inspection database, and notifies administrators and the police if an abnormality is detected. It also includes displaying this information in real time on a smart device with a specified application installed.

[0582] Hardware

[0583] The system includes the following hardware:

[0584] Imaging devices (cameras): Installed on traffic signals, public facilities, and vehicles.

[0585] Server (data processing server): Receives image data in real time and performs analysis.

[0586] Smart Devices: Includes smartphones and head-mounted displays (HMDs), which receive notifications and display footage in real time.

[0587] software

[0588] The software used is as follows:

[0589] YOLOv5: Used as an object detection algorithm.

[0590] OpenCV: A library for image processing.

[0591] Requests: A library for making HTTP requests.

[0592] Python: The primary development language in which the entire program is written.

[0593] Data processing and calculation

[0594] After receiving image data from the imaging device in real time, the server performs the following processes:

[0595] 1. Preprocessing of image data: noise removal and resolution adjustment.

[0596] 2. Feature extraction: YOLOv5 is used to extract vehicle features, specifically identifying license plates and vehicle types.

[0597] 3. Database matching: The extracted feature information is matched with the crime database and vehicle inspection database to detect suspicious vehicles.

[0598] 4. Notification: If an abnormality is detected, a notification will be sent to the smart device and detailed information will be forwarded to the administrator and police.

[0599] Example

[0600] As an example, we provide a parking lot security monitor. When the parking lot manager launches the "Parking Lot Security Monitor" app on their smartphone, real-time video footage of the parking lot is displayed. If a suspicious vehicle is detected, details are displayed in the smartphone's notification center. In addition, by using a head-mounted display, the situation inside the parking lot can be directly viewed.

[0601] Prompt Sentence Examples

[0602] "Create a Python program that uses YOLOv5 to extract vehicle characteristics from parking lot surveillance cameras in real time, and notify the manager and police if there are any abnormalities based on the matching results."

[0603] The system enables fast and efficient detection and management of car crime in parking lots and on roads.

[0604] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0605] Step 1:

[0606] The server collects image data in real time from image capture devices installed on traffic signals, public facilities, and vehicles. As input, it receives image data transmitted from each image capture device. As output, it stores the raw image data in memory. This data is used for subsequent processing. Specifically, the server retrieves data from the URL of the camera stream.

[0607] Step 2:

[0608] The server preprocesses the received image data. It uses raw image data as input. It gets preprocessed image data as output. This preprocessing includes noise removal and resolution adjustment. Specifically, it uses the OpenCV library to remove noise and adjust the resolution of the image.

[0609] Step 3:

[0610] The server extracts vehicle feature information from the preprocessed image data. The preprocessed image data is used as input. Feature information such as license plates and vehicle types is obtained as output. Specifically, object detection is performed using YOLOv5, and license plate recognition is performed.

[0611] Step 4:

[0612] The server compares the extracted feature information with the pre-stored crime database and vehicle inspection database. The feature information and database information are used as input. The comparison result is obtained as output. Specifically, the server sends an HTTP request to query the database.

[0613] Step 5:

[0614] If the server detects an anomaly based on the matching results, it notifies the administrator and the police. It uses the matching results as input, and generates and sends a notification message as output. Specifically, it implements a function to send an alert via the notification system.

[0615] Step 6:

[0616] When an anomaly is detected, the server installs a specific application on the smart device and displays images and notifications in real time. The server uses the anomaly detection information as input. The server sends a notification to the smart device as output. Specifically, the server displays an alert in the notification center of the smartphone or head-mounted display.

[0617] Step 7:

[0618] The server manages the collected and processed data. It uses feature information, matching results, and notification history as input. It obtains analysis results as output, which are stored in a database. Specific operations include recording all data to a central server, making it available for later analysis and review.

[0619] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0620] This invention relates to a system that detects automobile crimes and notifies relevant authorities by combining an emotion engine with a system that optimizes alertness and response based on the user's emotion information. Below, we will create a program for this system and explain its processing with specific examples.

[0621] Implementation of continuous monitoring

[0622] 1. Data Collection

[0623] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[0624] For example, a camera attached to a traffic light at an intersection captures video of passing vehicles.

[0625] 2. Image Preprocessing

[0626] The server performs pre-processing on the received image data, including noise removal and frame-by-frame image segmentation.

[0627] This removes unnecessary information from the image data, making it suitable for analysis.

[0628] 3. Feature Extraction

[0629] The server extracts vehicle feature information from the pre-processed image data, using object detection algorithms (e.g., YOLO, SSD) and OCR technology to identify the vehicle model, license plate number, and driving situation.

[0630] For example, the license plate number of a particular vehicle can be read from an image captured by a camera.

[0631] 4. Database Verification

[0632] The server compares the extracted vehicle license plate information with the crime database and vehicle inspection information database, and any anomalies are detected through the comparison process.

[0633] For example, it checks whether the license plate being read is on a list of stolen vehicles.

[0634] 5. Anomaly Detection

[0635] The server determines an anomaly based on the matching result, for example, if a license plate matching a stolen vehicle list is detected.

[0636] 6. Notice and Information

[0637] If an abnormality is detected, the server notifies the police or fire department, and the notification includes information about the vehicle's characteristics and location.

[0638] For example, if a stolen vehicle is detected, the information and current location will be sent to the police in real time.

[0639] Emergency confirmation implementation

[0640] 1. Data Reception

[0641] The server receives videos and photos taken by users via email or messaging apps, and simultaneously collects their emotional information.

[0642] For example, a user can film the scene of a hit-and-run accident and send the video along with emotional information from that time via email.

[0643] 2. Data storage

[0644] The server temporarily stores the received data, including metadata such as sender information and the date and time of receipt.

[0645] 3. Image Preprocessing

[0646] The server preprocesses the stored data, which involves removing noise and adjusting the resolution, making each frame individually analyzable.

[0647] 4. Feature Extraction

[0648] The server extracts vehicle feature information from the preprocessed data in real time using object detection algorithms and OCR technology, and the emotion engine analyzes the user's emotion information.

[0649] For example, the license plate of a photographed vehicle is analyzed, and at the same time, the user's emotional information is analyzed.

[0650] 5. Emergency analysis

[0651] The server compares the extracted feature information with a criminal database to detect abnormalities, prioritizes the analysis results based on the user's emotional information, and determines how to respond based on the level of urgency.

[0652] For example, if the license plate of a photographed vehicle matches a list of hit-and-run vehicles, and the user feels extreme shock or fear, an immediate alert will be sent to the police.

[0653] 6. Notification of Results and Response

[0654] If the server detects an abnormality, it immediately notifies the police or fire department. The notification includes analyzed feature information and the user's emotional information. The urgency and detail of the notification are adjusted based on the emotional information.

[0655] For example, if more detailed information or a faster response is required based on the user's emotional information, this will be communicated to the police.

[0656] The system of this invention achieves early detection and rapid response to car crimes by using a dual approach of continuous monitoring and emergency confirmation. Furthermore, by combining it with an emotion engine, the urgency and detail of the analysis results are optimized, aiming to enable faster and more appropriate responses from relevant authorities.

[0657] The processing flow will be explained below.

[0658] Continuous monitoring process steps

[0659] Step 1:

[0660] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[0661] Step 2:

[0662] The server performs preprocessing on the received image data. In this step, it removes noise from the image and divides the image into frames. This process removes unnecessary information from the image data and makes it suitable for analysis.

[0663] Step 3:

[0664] The server extracts vehicle feature information from the preprocessed image data, using object detection algorithms (e.g., YOLO, SSD) and OCR technology to identify the vehicle type, license plate number, and driving situation. For example, it reads the license plate number of a specific vehicle from the image captured by the camera.

[0665] Step 4:

[0666] The server compares the extracted vehicle license plate information with a criminal database and a vehicle inspection information database, and any anomalies are detected through this comparison.

[0667] Step 5:

[0668] The server determines an anomaly based on the matching result, for example, if a license plate matching a stolen vehicle list is detected.

[0669] Step 6:

[0670] If an abnormality is detected, the server immediately notifies the police or fire department. The notification includes vehicle characteristics (such as license plate, model, and location information), allowing the relevant authorities to respond quickly.

[0671] Processing steps for urgent confirmation

[0672] Step 1:

[0673] The server receives videos or photos taken by users via email or messaging apps, and simultaneously collects the user's emotional information. For example, a user may take a photo of a hit-and-run accident scene and send the video and their emotional information via email.

[0674] Step 2:

[0675] The server temporarily stores the received data, including metadata such as sender information and the date and time of receipt.

[0676] Step 3:

[0677] The server preprocesses the stored data, a step that involves removing noise and adjusting resolution so that each frame can be analyzed individually.

[0678] Step 4:

[0679] The server extracts vehicle feature information from the preprocessed data in real time, again using object detection algorithms and OCR technology. The emotion engine then analyzes the user's emotional information. For example, the server analyzes the license plate of the vehicle in the photo and simultaneously analyzes the user's emotional information.

[0680] Step 5:

[0681] The server compares the extracted feature information with a crime database to detect any abnormalities. It prioritizes the analysis results based on the user's emotional information and determines how to respond based on the level of urgency. For example, if the license plate of a photographed vehicle matches a list of hit-and-run vehicles, and the user feels extreme shock or fear, it will immediately send a report to the police.

[0682] Step 6:

[0683] If an abnormality is detected, the server immediately notifies the police or fire department. The notification content includes analyzed feature information and the user's emotional information. The urgency and level of detail of the notification are adjusted based on the emotional information. For example, if more detailed information or a faster response is required based on the user's emotional information, this information is communicated to the police. The system of the present invention achieves early detection and rapid response of car crimes using a dual approach of continuous monitoring and emergency confirmation. Furthermore, by combining an emotion engine, the urgency and level of detail of the analysis results are optimized, aiming to enable faster and more appropriate responses from relevant authorities.

[0684] Example 2

[0685] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0686] Conventional vehicle crime monitoring systems are capable of collecting and analyzing image data in real time, but lack the ability to prioritize emergency responses based on user emotional information. This makes it difficult for relevant authorities to respond appropriately and quickly when a crime is detected. Furthermore, when analyzing data based on emergency calls from users, it is difficult to determine the reliability and urgency of the data, which can cause delays in situations where a rapid response is required. A new system is needed to solve these problems.

[0687] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting image data in real time from sensors installed on traffic lights, public buildings, and vehicles, means for preprocessing the image data and extracting vehicle feature information, means for comparing the feature information with a pre-stored database, means for collecting user emotion information and setting the urgency level of the analysis results, and means for notifying relevant authorities when an abnormality is detected. This enables relevant authorities to respond appropriately and quickly when a crime is detected.

[0688] A "traffic light" is a device installed to control traffic on a road and displays signals to vehicles and pedestrians.

[0689] A "public building" is a building used for public purposes and functions as an administrative agency or public facility.

[0690] A "sensor" is a device that detects physical changes and outputs them as electrical signals.

[0691] "Image data" is a digital representation of optical information acquired by a camera or sensor.

[0692] "Preprocessing" refers to processing carried out before data analysis, such as removing noise and shaping data.

[0693] "Vehicle characteristic information" is information necessary for identifying and classifying vehicles, and includes license plates, vehicle models, colors, etc.

[0694] A "database" is a systematically organized collection of data that allows for efficient data retrieval and updating.

[0695] "User emotion information" is digital data that expresses the emotion felt by the user in a particular situation.

[0696] The "urgency of the analysis results" is an index that indicates the priority and urgency of the response based on the analyzed data.

[0697] "Related agencies" refers to administrative agencies and public facilities that respond to crimes and emergencies, including police and fire departments.

[0698] This invention is a surveillance system for detecting automobile crimes and notifying relevant authorities, with the aim of optimizing the level of vigilance and the urgency of response by using user emotional information in particular. This system detects crimes by collecting and analyzing data from cameras and sensors installed in traffic lights, public buildings, and vehicles. Furthermore, it analyzes emotional information based on emergency calls from users and notifies relevant authorities based on that information.

[0699] Hardware and Software

[0700] Hardware

[0701] Cameras and sensors: installed on traffic lights, public buildings, and vehicles. They collect image data in real time.

[0702] Server: A high-performance computing device for data collection, pre-processing, analysis, and notification.

[0703] Cloud storage: Used for temporary data storage and backup.

[0704] software

[0705] Image analysis algorithms: Object detection algorithms (YOLO, SSD, etc.) and OCR technology are used.

[0706] Databases: Crime database and vehicle inspection database.

[0707] Emotion engine: Used to analyze the user's emotional information and set the level of urgency.

[0708] Notification system: Communication software that notifies relevant authorities when an abnormality is detected.

[0709] Data flow

[0710] Data collection

[0711] The server collects image data in real time from cameras and sensors installed in traffic lights, public buildings, and vehicles. For example, a camera installed at an intersection captures images of a specific vehicle and sends the data to the server.

[0712] Pretreatment

[0713] The server preprocesses the received image data. In this step, noise is removed and the image is divided into frames to prepare it for analysis. For example, unnecessary parts are removed from the captured image to extract only the vehicle.

[0714] Analysis and Feature Extraction

[0715] The server extracts vehicle feature information from the preprocessed image data, using object detection algorithms such as YOLO, SSD, and OCR technology, analyzing features such as license plates, car models, and colors.

[0716] Database Matching

[0717] The server compares the extracted feature information with a criminal database and a vehicle inspection database, allowing it to detect anomalies, such as whether the license plate is included in a list of stolen vehicles.

[0718] Sentiment analysis and urgency setting

[0719] The server collects and analyzes emotional information along with video and image data based on emergency calls from users. The emotion engine sets the urgency level of the analysis results based on the user's emotional information. For example, if the user feels extreme shock or fear, the urgency level is set high.

[0720] notification

[0721] If an abnormality is detected, the server notifies the relevant authorities. The notification content includes vehicle characteristics, location information, and user emotion information. For example, if a stolen vehicle is detected, the police will be notified in real time along with its location information.

[0722] Specific examples

[0723] For example, if a camera installed at an intersection detects what appears to be a stolen vehicle, the image data is sent to a server. The server preprocesses the image, extracts features, and compares them with a database. If it matches a stolen vehicle, the server immediately notifies the police. Also, if a user witnesses a hit-and-run incident, the video is sent to the server along with emotional information. The server uses an emotion engine to set the urgency of the analysis results to a high level and promptly notify the police.

[0724] Prompt Sentence Examples

[0725] Here are some examples of prompts to input to the generative AI model:

[0726] "Please explain the process of a system that detects hit-and-run incidents and notifies the police based on user-recorded vehicle video and its emotional information."

[0727] This invention realizes early detection and rapid response to car crimes using a dual approach of continuous monitoring and emergency confirmation. In addition, the introduction of an emotion engine allows for faster and more appropriate notification to relevant authorities, contributing to improved safety for society as a whole.

[0728] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0729] Step 1: Data collection

[0730] Input: Real-time image data captured by cameras and sensors.

[0731] How it works: The server collects image data in real time from cameras and sensors installed on traffic lights, public buildings, and vehicles. This data is sent to the server via the internet or dedicated lines.

[0732] Output: Raw image data stored in the server storage.

[0733] Step 2: Image preprocessing

[0734] Input: Raw image data stored in storage.

[0735] How it works: The server applies a noise reduction filter to the received image data, splits the image into frames, cuts out the background and unnecessary parts, and formats the image in a way that is suitable for analysis.

[0736] Output: Preprocessed and clean image data.

[0737] Step 3: Feature extraction

[0738] Input: Preprocessed and clean image data.

[0739] How it works: The server uses an object detection algorithm (e.g., YOLO, SSD) to identify vehicles and OCR technology to read license plates, specifically identifying the make, color, and number plate characters in the image.

[0740] Output: Vehicle characteristics information (model, color, license plate, etc.).

[0741] Step 4: Database Matching

[0742] Input: Extracted vehicle feature information.

[0743] Operation: The server compares the extracted vehicle characteristics with the crime database and vehicle inspection database to determine whether the vehicle is a stolen or illegal vehicle.

[0744] Output: Matching result (whether anomalies are found or not).

[0745] Step 5: Anomaly detection

[0746] Input: Match results obtained from database matching.

[0747] How it works: The server determines whether something is an anomaly based on the match. For example, if the license plate matches a list of stolen vehicles, it will flag it as an anomaly.

[0748] Output: Anomaly detection flag and its detailed information.

[0749] Step 6: Notification and Information

[0750] Input: Anomaly detection flag and its details.

[0751] Operation: If an abnormality is detected, the server notifies the relevant authorities (police or fire department). The notification includes vehicle characteristics and location information. For example, the location information of a stolen vehicle is sent to the police in real time.

[0752] Output: Notification information sent to the appropriate authorities.

[0753] ---

[0754] The specific operation of each step is clarified and the processing flow from input to output is shown, making it easier to understand the operation of the entire system.

[0755] (Application example 2)

[0756] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0757] While early detection and rapid response to vehicle crimes are required, current systems are unable to consider the user's emotional state or level of urgency. This can result in a lack of accuracy in crime detection and the effectiveness of emergency notifications. Furthermore, there is no way to effectively utilize information provided by users via direct photography, making it difficult to accurately detect crimes in real time and respond quickly.

[0758] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting image data in real time from cameras installed on traffic lights, public buildings, and vehicles, means for analyzing the image data and extracting characteristic information about the vehicle, means for comparing the characteristic information with a pre-stored crime database and vehicle inspection database, means for notifying the police or fire department when an abnormality is detected, means for a user to capture video of the surroundings using a mobile communication device and collect emotional information, and means for adjusting the urgency of the analysis results based on the emotional information of the user. This makes it possible to adjust the urgency taking into account the emotional state of the user, thereby achieving a faster and more appropriate response.

[0759] A "traffic light" is a device installed to control traffic flow and issue instructions to vehicles and pedestrians.

[0760] "Public building" means a building open to the public, including government agencies and facilities providing public services.

[0761] "Vehicle" means a mechanical device, usually an engine-powered means of transportation, designed to transport people or goods.

[0762] A "camera" is an optical device for capturing video or images, and includes a lens and an image sensor.

[0763] "Image data" is data that digitally represents visual information acquired by a device such as a camera.

[0764] "User" refers to an individual or entity that operates a system or device and provides information.

[0765] "Mobile communication device" means a portable device that can send and receive data using wireless communication, including smartphones and tablets.

[0766] "Emotional information" is data that represents an individual's emotional state and is information collected based on voice analysis and facial expression recognition.

[0767] A "crime database" is a database for recording and managing information on crimes that have occurred in the past.

[0768] The "vehicle inspection database" is a database that stores and manages vehicle inspection history and registration information.

[0769] "Artificial intelligence" refers to computer systems that mimic or replicate human intelligence, including software that performs data analysis and pattern recognition.

[0770] "Analysis Results" refers to the conclusions or information obtained after processing and analyzing collected data.

[0771] "Police" means a government agency responsible for maintaining public safety and enforcing the law.

[0772] "Fire department" refers to a public institution that carries out fire prevention and extinguishing and emergency rescue activities.

[0773] "Notification" is the act or process of making specific information known to another person or system.

[0774] "Real-time" refers to a situation where data and information are processed and analyzed immediately, and results are provided without delay.

[0775] The present invention relates to a system for detecting motor vehicle crime and notifying appropriate authorities, optimizing vigilance and response based on the emotional state of the user. Specific embodiments for implementing the invention are described in detail herein.

[0776] System Configuration

[0777] The system of the present invention consists of the following main components:

[0778] 1. Data Collection Module

[0779] Cameras: Capture image data in real time from cameras installed on traffic lights, public buildings, and vehicles.

[0780] Mobile communication device: The user uses a mobile communication device (smartphone) to capture images of the surroundings and collect emotional information.

[0781] GPS: Location information is obtained using the GPS module built into the mobile communication device.

[0782] 2. Data Processing Module

[0783] Image preprocessing: Using a library such as OpenCV, image data acquired from the camera is denoised and segmented into individual frames.

[0784] Feature extraction: Object detection is performed using TensorFlow to extract vehicle feature information, and Tesseract is used for OCR processing to read license plate information.

[0785] Emotion analysis: Using artificial intelligence such as Watson Tone Analyzer, emotional information is analyzed from the voice data provided by the user.

[0786] 3. Database Matching Module

[0787] Database matching: The extracted vehicle feature information is matched with pre-stored crime databases and vehicle inspection databases.

[0788] 4. Notification Module

[0789] Notification system: Uses Firebase Cloud Messaging to notify police and fire departments when an abnormality is detected.

[0790] Example

[0791] For example, when a user takes a photo of a suspicious vehicle with a mobile communication device, the video and audio data are sent to a server. The server first extracts the vehicle's characteristics from the video and determines whether it is a target vehicle. At the same time, it analyzes the user's emotional information from the audio data and adjusts the alert level as necessary. If an abnormality is detected, the server immediately notifies the police or fire department and provides the vehicle's characteristics, location, and the user's emotional information.

[0792] Examples of prompts include:

[0793] If you notice a suspicious vehicle nearby, capture video and audio of the vehicle using your smartphone camera and microphone. The application will detect signs of crime from the video and audio and quickly notify the appropriate authorities based on the user's emotional state.

[0794] In this way, the present invention provides a system that effectively utilizes the video and emotional information provided by the user and optimizes the level of urgency and detail, thereby enabling a quick and appropriate response.

[0795] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0796] Step 1: Data collection

[0797] The user takes a video of the surroundings using the camera on their mobile communication device (smartphone). The camera captures video data in real time, and simultaneously collects audio data using the microphone. Location information is also obtained using the GPS module. The input data consists of video data, audio data, and location information. This data is sent to the server and used for subsequent analysis.

[0798] Step 2: Image preprocessing

[0799] The server performs preprocessing on the received video data. In preprocessing, OpenCV is used to remove noise and segment the image frame by frame. The input is the video data acquired in step 1, and the output is image data with noise removed and suitable for analysis. This process ensures that the subsequent feature extraction process can be performed accurately and efficiently.

[0800] Step 3: Feature extraction

[0801] The server extracts vehicle feature information from the preprocessed image data. It uses TensorFlow to perform object detection and identify the vehicle's position and shape. It also uses Tesseract to perform OCR processing of license plates. The input is the preprocessed image data, and the output is vehicle feature information and license plate information. This allows for vehicle identification.

[0802] Step 4: Sentiment Analysis

[0803] The server analyzes the voice data collected by the user when taking a photo. It uses artificial intelligence such as Watson Tone Analyzer to obtain the user's emotional information. The input is the voice data obtained in step 1, and the output is the analyzed emotional information of the user. This allows the emotion the user was feeling at the time to be detected.

[0804] Step 5: Database Matching

[0805] The server compares the extracted vehicle characteristic information and license plate information with the crime database and vehicle inspection database. This comparison determines whether the vehicle is a stolen or illegal vehicle. The input is the vehicle characteristic information and license plate information obtained in step 3, and the output is the comparison result. This allows anomalies to be detected.

[0806] Step 6: Adjusting urgency

[0807] The server integrates the matching results with the user's emotional information and adjusts the urgency of the anomaly. If the urgency is high based on the emotional information, it increases the priority of notification to the police and fire department. The input is the emotional information acquired in step 4 and the matching results obtained in step 5, and the output is the adjusted urgency information. This optimizes the urgency.

[0808] Step 7: Notification

[0809] The server notifies the police and fire department of the adjusted urgency information and details of the abnormality. It uses a notification system such as Firebase Cloud Messaging to transmit the information to the relevant agencies in real time. The input is the urgency information and details of the abnormality obtained in step 6, and the output is a notification message to the police and fire department. This enables a rapid response.

[0810] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0811] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0812] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0813] [Third embodiment]

[0814] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0815] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0816] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0817] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0818] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0819] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0820] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0821] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0822] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0823] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0824] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0825] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0826] The present invention relates to a system that detects automobile crimes by collecting image data using cameras installed on traffic lights, public buildings, and vehicles, and analyzing the collected image data in real time, and then notifies the relevant authorities. Below, we will create a program for this system and explain its processing with specific examples.

[0827] Implementation of continuous monitoring

[0828] 1. Data Collection

[0829] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[0830] For example, a camera attached to a traffic light at an intersection captures video of passing vehicles.

[0831] 2. Image Preprocessing

[0832] The server performs pre-processing on the received image data, including noise removal and frame-by-frame image segmentation.

[0833] This process removes unnecessary information from the image data, making it suitable for analysis.

[0834] 3. Feature Extraction

[0835] The server extracts vehicle feature information from the preprocessed image data, specifically identifying the vehicle model, license plate number, and driving situation using object detection algorithms (e.g., YOLO, SSD) and OCR technology.

[0836] For example, the license plate number of a particular vehicle can be read from an image captured by a camera and the information stored in a database.

[0837] 4. Database Verification

[0838] The server compares the extracted vehicle license plates with a criminal database and a vehicle inspection information database, and this comparison detects any anomalies.

[0839] For example, it checks whether the license plate being read is on a list of stolen vehicles.

[0840] 5. Anomaly detection and notification

[0841] If the server detects an abnormality based on the comparison results, it notifies the police or fire department, including the vehicle's characteristics and location information.

[0842] For example, if a stolen vehicle is detected, the information and current location will be sent to the police in real time.

[0843] This series of processes enables a rapid response when a car crime is detected during road surveillance.

[0844] Emergency confirmation implementation

[0845] 1. Data Reception

[0846] The server receives videos and photos taken by users via email or messaging apps.

[0847] For example, a user may take a video of a hit-and-run incident and send the video to a server via email.

[0848] 2. Data storage

[0849] The server temporarily stores the received data, including metadata such as sender information and the date and time of receipt.

[0850] 3. Image Preprocessing

[0851] The server preprocesses the stored data, making each frame individually analyzable, by removing noise and adjusting the resolution.

[0852] 4. Feature Extraction

[0853] The server extracts vehicle feature information from the pre-processed data in real time, again using object detection algorithms and OCR technology.

[0854] 5. Urgent Analysis and Notification

[0855] The server compares the extracted characteristic information with a crime database, and if it detects an abnormality, it notifies the police or fire department of the results.

[0856] For example, if the license plate of a photographed vehicle is found to be on a list of hit-and-run vehicles, the information will be immediately notified to the police.

[0857] By using the above means, the present invention is a system that realizes early detection and rapid response to automobile crimes through both constant monitoring and emergency confirmation.

[0858] The processing flow will be explained below.

[0859] Continuous monitoring process steps

[0860] Step 1:

[0861] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[0862] Step 2:

[0863] The server performs preprocessing on the received image data. In this step, it removes noise from the image and divides the image into frames. This process removes unnecessary information from the image data and makes it suitable for analysis.

[0864] Step 3:

[0865] The server extracts vehicle feature information from the preprocessed image data, using object detection algorithms (e.g., YOLO, SSD) and OCR technology to identify the vehicle type, license plate number, and driving situation. For example, it reads the license plate number of a specific vehicle from the image captured by the camera.

[0866] Step 4:

[0867] The server compares the extracted vehicle license plate information with a criminal database and a vehicle inspection information database, and this comparison detects any anomalies.

[0868] Step 5:

[0869] The server determines an anomaly based on the matching result, for example, if a license plate matching a stolen vehicle list is detected.

[0870] Step 6:

[0871] If an abnormality is detected, the server immediately notifies the police or fire department. The notification includes vehicle characteristics (such as license plate, model, and location information), allowing the relevant authorities to respond quickly.

[0872] Processing steps for urgent confirmation

[0873] Step 1:

[0874] The server receives videos or photos taken by users via email or messaging apps. For example, a user may take a video of a hit-and-run accident and send the video via email.

[0875] Step 2:

[0876] The server temporarily stores the received data, which also includes metadata such as sender information and the date and time of receipt.

[0877] Step 3:

[0878] The server preprocesses the stored data, which involves removing noise and adjusting the resolution so that each frame can be analyzed individually.

[0879] Step 4:

[0880] The server extracts vehicle feature information from the pre-processed data in real time, again using object detection algorithms and OCR technology.

[0881] Step 5:

[0882] The server compares the extracted feature information with a crime database to detect anomalies, such as when the license plate of the photographed vehicle is included in a list of hit-and-run vehicles.

[0883] Step 6:

[0884] If the server detects an abnormality, it immediately notifies the police or fire department. The notification includes analyzed characteristic information and sender information, allowing the relevant authorities to respond quickly.

[0885] Example 1

[0886] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0887] In modern society, improving the safety and efficiency of transportation infrastructure is an important issue. Rapid detection and response to vehicle crimes is particularly important, but conventional surveillance systems have struggled to accurately detect abnormalities in real time and notify relevant authorities. Furthermore, there is a lack of means for users to efficiently analyze data captured in emergency situations and respond quickly. To address these issues, a system that combines advanced image processing and database matching technologies is needed.

[0888] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0889] In this invention, the server includes means for collecting visible data in real time from optical devices installed on traffic lights, public buildings, and vehicles, means for preprocessing the visible data to make it analyzable, means for extracting characteristic information about the vehicle from the visible data, means for comparing the characteristic information with a pre-stored database, and means for notifying relevant authorities when an abnormality is detected, thereby enabling early detection and rapid response to automobile crimes.

[0890] The server also includes a means for receiving visible data captured by a user, a means for preprocessing the received visible data to make it analyzable, a means for extracting characteristic information of the vehicle from the visible data, a means for comparing the characteristic information with a pre-stored database, a means for notifying relevant organizations when an abnormality is detected, and a means for managing the analysis results sent to the relevant organizations by the notifying means, thereby enabling emergency information from users to be processed quickly and accurately.

[0891] Furthermore, by connecting the optical equipment to a device equipped with a machine learning algorithm and having the means to automatically analyze the visible data, high-precision analysis can be performed in real time, improving the accuracy of detecting vehicle crimes and enabling rapid response.

[0892] "Optical equipment" refers to cameras and sensors used to acquire image and video data.

[0893] "Visible data" refers to image data or video data acquired by optical instruments, and includes visual information to be analyzed.

[0894] "Preprocessing" refers to a series of processing operations performed to prepare visible data for analysis, including noise removal, image segmentation, and resolution adjustment.

[0895] "Feature information" refers to significant information extracted from visible data, specifically vehicle model, license plate number, driving conditions, etc.

[0896] A "database" is a computer system that systematically stores information for later retrieval or collation, and includes crime databases and vehicle inspection databases.

[0897] A "machine learning algorithm" is an algorithm that automatically learns patterns and rules from data and uses that knowledge to analyze new data.

[0898] "Relevant agencies" are agencies to which notification will be sent if an abnormality such as a car crime is detected, specifically the police and fire department.

[0899] "Means of notification" refers to the communication method used to notify relevant authorities of an abnormality, and includes SMS, email, voice calls, etc.

[0900] "Receiving means" refers to the method or function by which the server receives the visible data sent by the user.

[0901] "Management means" refers to the method or function for recording and storing the analysis results sent to the relevant authorities by the notification means, and for use in future reference and analysis.

[0902] This invention is a system that collects visible data in real time from optical devices installed on traffic lights, public buildings, and vehicles, preprocesses this data to extract vehicle characteristic information, compares it with a crime database, and notifies relevant authorities if an abnormality is detected. The following hardware and software are used to implement this invention.

[0903] Hardware and software used

[0904] 1. Optical equipment

[0905] Cameras and sensors are used as optical instruments. For example, surveillance cameras are installed to capture data in real time 24 hours a day, 365 days a year.

[0906] 2. Server

[0907] The server preprocesses the received visible data, extracts feature information, compares it with a database, and notifies users when an anomaly is detected. The following software is used on the server:

[0908] "OpenCV": Used for preprocessing the visual data, which includes denoising and frame-by-frame segmentation.

[0909] "YOLO" or "SSD": Used as an object detection algorithm to extract vehicle feature information from visual data.

[0910] "Tesseract OCR": Uses text recognition technology to analyze the characters on license plates.

[0911] "Elasticsearch" or "MySQL": Used to match the extracted feature information with the database.

[0912] "Twilio API": Used to notify relevant authorities (e.g., police and fire department) when an anomaly is detected.

[0913] 3. Communication Network

[0914] A network for transmitting data from cameras and sensors to a server. Data is transferred via the Internet or a dedicated data transfer network.

[0915] System operation example

[0916] For example, consider a surveillance camera installed at a traffic light at an intersection. This camera captures video data of passing vehicles in real time. This data is then sent to a server via a communication network.

[0917] 1. Data Collection

[0918] The server collects visible data received from optical devices, in this case video data transmitted from surveillance cameras.

[0919] 2. Image Preprocessing

[0920] The server preprocesses the collected visual data, for example, using OpenCV to perform image processing such as noise removal and frame segmentation, thereby creating data suitable for analysis.

[0921] 3. Feature Extraction

[0922] The server uses object detection algorithms such as YOLO and SSD to extract vehicle feature information from the preprocessed data, then uses Tesseract OCR to read the license plate characters and store them in a database.

[0923] 4. Database Verification

[0924] The server compares the extracted feature information with databases stored in Elasticsearch or MySQL, thereby detecting anomalies by comparing them with criminal databases and vehicle inspection information databases.

[0925] 5. Anomaly detection and notification

[0926] If the server detects an abnormality, it will use the Twilio API to notify the police, fire department, and other relevant authorities. This notification will include information about the vehicle's characteristics and current location, enabling a prompt response.

[0927] Prompt Sentence Examples

[0928] Prompt: Please explain the processing procedure of this system in natural language. Please specify what hardware or software is used to process or calculate data. Please also specify the specific name of the hardware or software used.

[0929] 1. The server receives real-time visual data from optical devices installed on traffic lights, public buildings, and vehicles, for example, dedicated surveillance cameras.

[0930] 2. The server preprocesses the received visual data. In this step, it uses OpenCV to remove noise and segment the image for each frame.

[0931] 3. The server extracts vehicle feature information from the preprocessed visual data using object detection algorithms such as YOLO and SSD, as well as Tesseract OCR.

[0932] 4. The server compares the extracted vehicle feature information with the database, using Elasticsearch and MySQL for this comparison.

[0933] 5. If the server detects an anomaly based on the matching results, it notifies the relevant authorities using the "Twilio API."

[0934] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0935] Program processing steps

[0936] Step 1:

[0937] The server collects visible data in real time from optical devices installed in traffic lights, public buildings, and vehicles. The input data from these optical devices includes images and videos. The input data is stored as is and then moved on to the next processing step. Specifically, surveillance cameras capture image data in real time 24 hours a day, 365 days a year, and send it to the server via the Internet. The received data is then stored in the server's storage.

[0938] Step 2:

[0939] The server preprocesses the collected visual data to make it ready for analysis. This process uses image processing techniques to remove noise and segment each frame. The input is the image data collected in step 1, and the output is the preprocessed image data. Specific examples include using OpenCV to remove noise from the image data and adjust the image resolution as needed. This ensures that the data used for analysis is of high quality.

[0940] Step 3:

[0941] The server extracts vehicle feature information from the preprocessed visual data. This process uses an object detection algorithm to identify features such as the vehicle model and license plate. The input is the image data preprocessed in step 2, and the output is vehicle feature information. Specifically, vehicles are detected using "YOLO" or "SSD," and text information from license plates is extracted using "Tesseract OCR." The extracted information is temporarily stored in the server's memory.

[0942] Step 4:

[0943] The server compares the extracted vehicle characteristics information with a database. This comparison checks for matching data in criminal databases or vehicle inspection databases. The input is the vehicle characteristics information extracted in step 3, and the output is the results of an anomaly inspection. For example, a query is run against a database stored in Elasticsearch or MySQL to match license plates and vehicle characteristics.

[0944] Step 5:

[0945] If the server detects an abnormality based on the matching results, it will take measures to notify the relevant authorities. The notification content includes vehicle characteristics and location information. The input is the abnormality inspection results obtained in step 4, and the output is the notified information. Specifically, it uses the "Twilio API" to send real-time notifications to the police and fire department via email, SMS, voice calls, etc. It may also use the "Google Maps API" to obtain location information, allowing the relevant authorities to respond quickly.

[0946] Step 6:

[0947] A user takes a photo of an accident scene and sends the image data to the server. This data is sent to the server via email or a messaging app. The input is the image data sent by the user, and the output is the data received by the server. For example, this includes the user taking a photo of the accident scene and sending the image to the server via email.

[0948] Step 7:

[0949] The server preprocesses the image data received from the user to make it ready for analysis. The input is the image data received in step 6, and the output is the preprocessed image data. As a specific example, "OpenCV" is used as part of the preprocessing to remove noise and divide the image frames.

[0950] Step 8:

[0951] The server extracts vehicle feature information from the preprocessed data. The input is the image data preprocessed in step 7, and the output is the vehicle feature information. Specific operations include detecting vehicles using the YOLO model and reading license plates using Tesseract OCR.

[0952] Step 9:

[0953] The server compares the extracted feature information with the database and notifies the relevant authorities if an abnormality is detected. The input is the vehicle feature information extracted in step 8, and the output is the notification result. As a specific example, a query is executed against the database, and if an abnormality is detected, the relevant authorities are notified using the "Twilio API." This process enables a fast and accurate response even in emergencies.

[0954] (Application example 1)

[0955] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0956] The problem that this invention aims to solve is to detect automobile crimes in parking lots and on roads in real time and respond promptly and appropriately. Conventional technologies have had the problem that it takes time from the time a crime occurs until it is reported, making it difficult to respond quickly. In addition, there was a lack of means to efficiently identify vehicles illegally entering parking lots or stolen vehicles, which placed a heavy burden on managers and police.

[0957] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0958] In this invention, the server includes means for collecting image data in real time from imaging devices installed in traffic signals, public facilities, and vehicles, means for analyzing the image data and extracting characteristic information about the vehicle, means for comparing the characteristic information with a pre-stored crime database and vehicle inspection database, means for notifying an administrator and the police when an abnormality is detected, and means for installing a predetermined application on a smart device and displaying images and notifications in real time when an abnormality is detected, thereby enabling fast and efficient detection and management of automobile crimes in parking lots and on roads.

[0959] A "signal device" is a device that emits a signal, and includes traffic lights and the like for controlling traffic flow.

[0960] "Public facilities" are facilities that are available to the general public, such as city halls, schools, and train stations.

[0961] An "imaging device" is a device for capturing images or videos, and includes cameras and video cameras.

[0962] "Real-time" refers to processing or responding to an event as it occurs.

[0963] "Image data" refers to digital data of images or videos captured by an imaging device.

[0964] "Characteristic information" is information that indicates specific attributes or characteristics extracted from analyzed data, such as license plates and vehicle models.

[0965] A "crime database" is a database that stores information related to past crimes.

[0966] A "vehicle inspection database" is a database that stores vehicle registration information and inspection history.

[0967] "Matching" refers to comparing one piece of data with another to see if they match.

[0968] "Abnormal" refers to an unusual or unexpected condition, such as the discovery of a stolen vehicle.

[0969] "Notification" means the communication of specific information to another person.

[0970] "Administrator" refers to a person responsible for operating and monitoring a system or facility.

[0971] "Police" means an agency that enforces the law to maintain public order and safety.

[0972] "Smart devices" refer to electronic devices with advanced functions, including smartphones and head-mounted displays.

[0973] An "application" is a software program that provides a particular function or service.

[0974] The system for implementing this invention collects image data in real time from imaging devices installed in traffic signals, public facilities, and vehicles, analyzes the image data to extract characteristic information about the vehicle, compares it with a crime database and a vehicle inspection database, and notifies administrators and the police if an abnormality is detected. It also includes displaying this information in real time on a smart device with a specified application installed.

[0975] Hardware

[0976] The system includes the following hardware:

[0977] Imaging devices (cameras): Installed on traffic signals, public facilities, and vehicles.

[0978] Server (data processing server): Receives image data in real time and performs analysis.

[0979] Smart Devices: Includes smartphones and head-mounted displays (HMDs), which receive notifications and display footage in real time.

[0980] software

[0981] The software used is as follows:

[0982] YOLOv5: Used as an object detection algorithm.

[0983] OpenCV: A library for image processing.

[0984] Requests: A library for making HTTP requests.

[0985] Python: The primary development language in which the entire program is written.

[0986] Data processing and calculation

[0987] After receiving image data from the imaging device in real time, the server performs the following processes:

[0988] 1. Preprocessing of image data: noise removal and resolution adjustment.

[0989] 2. Feature extraction: YOLOv5 is used to extract vehicle features, specifically identifying license plates and vehicle types.

[0990] 3. Database matching: The extracted feature information is matched with the crime database and vehicle inspection database to detect suspicious vehicles.

[0991] 4. Notification: If an abnormality is detected, a notification will be sent to the smart device and detailed information will be forwarded to the administrator and police.

[0992] Example

[0993] As an example, we provide a parking lot security monitor. When the parking lot manager launches the "Parking Lot Security Monitor" app on their smartphone, real-time video footage of the parking lot is displayed. If a suspicious vehicle is detected, details are displayed in the smartphone's notification center. In addition, by using a head-mounted display, the situation inside the parking lot can be directly viewed.

[0994] Prompt Sentence Examples

[0995] "Create a Python program that uses YOLOv5 to extract vehicle characteristics from parking lot surveillance cameras in real time, and notify the manager and police if there are any abnormalities based on the matching results."

[0996] The system enables fast and efficient detection and management of car crime in parking lots and on roads.

[0997] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0998] Step 1:

[0999] The server collects image data in real time from image capture devices installed on traffic signals, public facilities, and vehicles. As input, it receives image data transmitted from each image capture device. As output, it stores the raw image data in memory. This data is used for subsequent processing. Specifically, the server retrieves data from the URL of the camera stream.

[1000] Step 2:

[1001] The server preprocesses the received image data. It uses raw image data as input. It gets preprocessed image data as output. This preprocessing includes noise removal and resolution adjustment. Specifically, it uses the OpenCV library to remove noise and adjust the resolution of the image.

[1002] Step 3:

[1003] The server extracts vehicle feature information from the preprocessed image data. The preprocessed image data is used as input. Feature information such as license plates and vehicle types is obtained as output. Specifically, object detection is performed using YOLOv5, and license plate recognition is performed.

[1004] Step 4:

[1005] The server compares the extracted feature information with the pre-stored crime database and vehicle inspection database. The feature information and database information are used as input. The comparison result is obtained as output. Specifically, the server sends an HTTP request to query the database.

[1006] Step 5:

[1007] If the server detects an anomaly based on the matching results, it notifies the administrator and the police. It uses the matching results as input, and generates and sends a notification message as output. Specifically, it implements a function to send an alert via the notification system.

[1008] Step 6:

[1009] When an anomaly is detected, the server installs a specific application on the smart device and displays images and notifications in real time. The server uses the anomaly detection information as input. The server sends a notification to the smart device as output. Specifically, the server displays an alert in the notification center of the smartphone or head-mounted display.

[1010] Step 7:

[1011] The server manages the collected and processed data. It uses feature information, matching results, and notification history as input. It obtains analysis results as output, which are stored in a database. Specific operations include recording all data to a central server, making it available for later analysis and review.

[1012] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1013] This invention relates to a system that detects automobile crimes and notifies relevant authorities by combining an emotion engine with a system that optimizes alertness and response based on the user's emotion information. Below, we will create a program for this system and explain its processing with specific examples.

[1014] Implementation of continuous monitoring

[1015] 1. Data Collection

[1016] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[1017] For example, a camera attached to a traffic light at an intersection captures video of passing vehicles.

[1018] 2. Image Preprocessing

[1019] The server performs pre-processing on the received image data, including noise removal and frame-by-frame image segmentation.

[1020] This removes unnecessary information from the image data, making it suitable for analysis.

[1021] 3. Feature Extraction

[1022] The server extracts vehicle feature information from the pre-processed image data, using object detection algorithms (e.g., YOLO, SSD) and OCR technology to identify the vehicle model, license plate number, and driving situation.

[1023] For example, the license plate number of a particular vehicle can be read from an image captured by a camera.

[1024] 4. Database Verification

[1025] The server compares the extracted vehicle license plate information with the crime database and vehicle inspection information database, and any anomalies are detected through the comparison process.

[1026] For example, it checks whether the license plate being read is on a list of stolen vehicles.

[1027] 5. Anomaly Detection

[1028] The server determines an anomaly based on the matching result, for example, if a license plate matching a stolen vehicle list is detected.

[1029] 6. Notice and Information

[1030] If an abnormality is detected, the server notifies the police or fire department, and the notification includes information about the vehicle's characteristics and location.

[1031] For example, if a stolen vehicle is detected, the information and current location will be sent to the police in real time.

[1032] Emergency confirmation implementation

[1033] 1. Data Reception

[1034] The server receives videos and photos taken by users via email or messaging apps, and simultaneously collects their emotional information.

[1035] For example, a user can film the scene of a hit-and-run accident and send the video along with emotional information from that time via email.

[1036] 2. Data storage

[1037] The server temporarily stores the received data, including metadata such as sender information and the date and time of receipt.

[1038] 3. Image Preprocessing

[1039] The server preprocesses the stored data, which involves removing noise and adjusting the resolution, making each frame individually analyzable.

[1040] 4. Feature Extraction

[1041] The server extracts vehicle feature information from the preprocessed data in real time using object detection algorithms and OCR technology, and the emotion engine analyzes the user's emotion information.

[1042] For example, the license plate of a photographed vehicle is analyzed, and at the same time, the user's emotional information is analyzed.

[1043] 5. Emergency analysis

[1044] The server compares the extracted feature information with a criminal database to detect abnormalities, prioritizes the analysis results based on the user's emotional information, and determines how to respond based on the level of urgency.

[1045] For example, if the license plate of a photographed vehicle matches a list of hit-and-run vehicles, and the user feels extreme shock or fear, an immediate alert will be sent to the police.

[1046] 6. Notification of Results and Response

[1047] If the server detects an abnormality, it immediately notifies the police or fire department. The notification includes analyzed feature information and the user's emotional information. The urgency and detail of the notification are adjusted based on the emotional information.

[1048] For example, if more detailed information or a faster response is required based on the user's emotional information, this will be communicated to the police.

[1049] The system of this invention achieves early detection and rapid response to car crimes by using a dual approach of continuous monitoring and emergency confirmation. Furthermore, by combining it with an emotion engine, the urgency and detail of the analysis results are optimized, aiming to enable faster and more appropriate responses from relevant authorities.

[1050] The processing flow will be explained below.

[1051] Continuous monitoring process steps

[1052] Step 1:

[1053] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[1054] Step 2:

[1055] The server performs preprocessing on the received image data. In this step, it removes noise from the image and divides the image into frames. This process removes unnecessary information from the image data and makes it suitable for analysis.

[1056] Step 3:

[1057] The server extracts vehicle feature information from the preprocessed image data, using object detection algorithms (e.g., YOLO, SSD) and OCR technology to identify the vehicle type, license plate number, and driving situation. For example, it reads the license plate number of a specific vehicle from the image captured by the camera.

[1058] Step 4:

[1059] The server compares the extracted vehicle license plate information with a criminal database and a vehicle inspection information database, and any anomalies are detected through this comparison.

[1060] Step 5:

[1061] The server determines an anomaly based on the matching result, for example, if a license plate matching a stolen vehicle list is detected.

[1062] Step 6:

[1063] If an abnormality is detected, the server immediately notifies the police or fire department. The notification includes vehicle characteristics (such as license plate, model, and location information), allowing the relevant authorities to respond quickly.

[1064] Processing steps for urgent confirmation

[1065] Step 1:

[1066] The server receives videos or photos taken by users via email or messaging apps, and simultaneously collects the user's emotional information. For example, a user may take a photo of a hit-and-run accident scene and send the video and their emotional information via email.

[1067] Step 2:

[1068] The server temporarily stores the received data, including metadata such as sender information and the date and time of receipt.

[1069] Step 3:

[1070] The server preprocesses the stored data, a step that involves removing noise and adjusting resolution so that each frame can be analyzed individually.

[1071] Step 4:

[1072] The server extracts vehicle feature information from the preprocessed data in real time, again using object detection algorithms and OCR technology. The emotion engine then analyzes the user's emotional information. For example, the server analyzes the license plate of the vehicle in the photo and simultaneously analyzes the user's emotional information.

[1073] Step 5:

[1074] The server compares the extracted feature information with a crime database to detect any abnormalities. It prioritizes the analysis results based on the user's emotional information and determines how to respond based on the level of urgency. For example, if the license plate of a photographed vehicle matches a list of hit-and-run vehicles, and the user feels extreme shock or fear, it will immediately send a report to the police.

[1075] Step 6:

[1076] If an abnormality is detected, the server immediately notifies the police or fire department. The notification content includes analyzed feature information and the user's emotional information. The urgency and level of detail of the notification are adjusted based on the emotional information. For example, if more detailed information or a faster response is required based on the user's emotional information, this information is communicated to the police. The system of the present invention achieves early detection and rapid response of car crimes using a dual approach of continuous monitoring and emergency confirmation. Furthermore, by combining an emotion engine, the urgency and level of detail of the analysis results are optimized, aiming to enable faster and more appropriate responses from relevant authorities.

[1077] Example 2

[1078] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1079] Conventional vehicle crime monitoring systems are capable of collecting and analyzing image data in real time, but lack the ability to prioritize emergency responses based on user emotional information. This makes it difficult for relevant authorities to respond appropriately and quickly when a crime is detected. Furthermore, when analyzing data based on emergency calls from users, it is difficult to determine the reliability and urgency of the data, which can cause delays in situations where a rapid response is required. A new system is needed to solve these problems.

[1080] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting image data in real time from sensors installed on traffic lights, public buildings, and vehicles, means for preprocessing the image data and extracting vehicle feature information, means for comparing the feature information with a pre-stored database, means for collecting user emotion information and setting the urgency level of the analysis results, and means for notifying relevant authorities when an abnormality is detected. This enables relevant authorities to respond appropriately and quickly when a crime is detected.

[1081] A "traffic light" is a device installed to control traffic on a road and displays signals to vehicles and pedestrians.

[1082] A "public building" is a building used for public purposes and functions as an administrative agency or public facility.

[1083] A "sensor" is a device that detects physical changes and outputs them as electrical signals.

[1084] "Image data" is a digital representation of optical information acquired by a camera or sensor.

[1085] "Preprocessing" refers to processing carried out before data analysis, such as removing noise and shaping data.

[1086] "Vehicle characteristic information" is information necessary for identifying and classifying vehicles, and includes license plates, vehicle models, colors, etc.

[1087] A "database" is a systematically organized collection of data that allows for efficient data retrieval and updating.

[1088] "User emotion information" is digital data that expresses the emotion felt by the user in a particular situation.

[1089] The "urgency of the analysis results" is an index that indicates the priority and urgency of the response based on the analyzed data.

[1090] "Related agencies" refers to administrative agencies and public facilities that respond to crimes and emergencies, including police and fire departments.

[1091] This invention is a surveillance system for detecting automobile crimes and notifying relevant authorities, with the aim of optimizing the level of vigilance and the urgency of response by using user emotional information in particular. This system detects crimes by collecting and analyzing data from cameras and sensors installed in traffic lights, public buildings, and vehicles. Furthermore, it analyzes emotional information based on emergency calls from users and notifies relevant authorities based on that information.

[1092] Hardware and Software

[1093] Hardware

[1094] Cameras and sensors: installed on traffic lights, public buildings, and vehicles. They collect image data in real time.

[1095] Server: A high-performance computing device for data collection, pre-processing, analysis, and notification.

[1096] Cloud storage: Used for temporary data storage and backup.

[1097] software

[1098] Image analysis algorithms: Object detection algorithms (YOLO, SSD, etc.) and OCR technology are used.

[1099] Databases: Crime database and vehicle inspection database.

[1100] Emotion engine: Used to analyze the user's emotional information and set the level of urgency.

[1101] Notification system: Communication software that notifies relevant authorities when an abnormality is detected.

[1102] Data flow

[1103] Data collection

[1104] The server collects image data in real time from cameras and sensors installed in traffic lights, public buildings, and vehicles. For example, a camera installed at an intersection captures images of a specific vehicle and sends the data to the server.

[1105] Pretreatment

[1106] The server preprocesses the received image data. In this step, noise is removed and the image is divided into frames to prepare it for analysis. For example, unnecessary parts are removed from the captured image to extract only the vehicle.

[1107] Analysis and Feature Extraction

[1108] The server extracts vehicle feature information from the preprocessed image data, using object detection algorithms such as YOLO, SSD, and OCR technology, analyzing features such as license plates, car models, and colors.

[1109] Database Matching

[1110] The server compares the extracted feature information with a criminal database and a vehicle inspection database, allowing it to detect anomalies, such as whether the license plate is included in a list of stolen vehicles.

[1111] Sentiment analysis and urgency setting

[1112] The server collects and analyzes emotional information along with video and image data based on emergency calls from users. The emotion engine sets the urgency level of the analysis results based on the user's emotional information. For example, if the user feels extreme shock or fear, the urgency level is set high.

[1113] notification

[1114] If an abnormality is detected, the server notifies the relevant authorities. The notification content includes vehicle characteristics, location information, and user emotion information. For example, if a stolen vehicle is detected, the police will be notified in real time along with its location information.

[1115] Specific examples

[1116] For example, if a camera installed at an intersection detects what appears to be a stolen vehicle, the image data is sent to a server. The server preprocesses the image, extracts features, and compares them with a database. If it matches a stolen vehicle, the server immediately notifies the police. Also, if a user witnesses a hit-and-run incident, the video is sent to the server along with emotional information. The server uses an emotion engine to set the urgency of the analysis results to a high level and promptly notify the police.

[1117] Prompt Sentence Examples

[1118] Here are some examples of prompts to input to the generative AI model:

[1119] "Please explain the process of a system that detects hit-and-run incidents and notifies the police based on user-recorded vehicle video and its emotional information."

[1120] This invention realizes early detection and rapid response to car crimes using a dual approach of continuous monitoring and emergency confirmation. In addition, the introduction of an emotion engine allows for faster and more appropriate notification to relevant authorities, contributing to improved safety for society as a whole.

[1121] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1122] Step 1: Data collection

[1123] Input: Real-time image data captured by cameras and sensors.

[1124] How it works: The server collects image data in real time from cameras and sensors installed on traffic lights, public buildings, and vehicles. This data is sent to the server via the internet or dedicated lines.

[1125] Output: Raw image data stored in the server storage.

[1126] Step 2: Image preprocessing

[1127] Input: Raw image data stored in storage.

[1128] How it works: The server applies a noise reduction filter to the received image data, splits the image into frames, cuts out the background and unnecessary parts, and formats the image in a way that is suitable for analysis.

[1129] Output: Preprocessed and clean image data.

[1130] Step 3: Feature extraction

[1131] Input: Preprocessed and clean image data.

[1132] How it works: The server uses an object detection algorithm (e.g., YOLO, SSD) to identify vehicles and OCR technology to read license plates, specifically identifying the make, color, and number plate characters in the image.

[1133] Output: Vehicle characteristics information (model, color, license plate, etc.).

[1134] Step 4: Database Matching

[1135] Input: Extracted vehicle feature information.

[1136] Operation: The server compares the extracted vehicle characteristics with the crime database and vehicle inspection database to determine whether the vehicle is a stolen or illegal vehicle.

[1137] Output: Matching result (whether anomalies are found or not).

[1138] Step 5: Anomaly detection

[1139] Input: Match results obtained from database matching.

[1140] How it works: The server determines whether something is an anomaly based on the match. For example, if the license plate matches a list of stolen vehicles, it will flag it as an anomaly.

[1141] Output: Anomaly detection flag and its detailed information.

[1142] Step 6: Notification and Information

[1143] Input: Anomaly detection flag and its details.

[1144] Operation: If an abnormality is detected, the server notifies the relevant authorities (police or fire department). The notification includes vehicle characteristics and location information. For example, the location information of a stolen vehicle is sent to the police in real time.

[1145] Output: Notification information sent to the appropriate authorities.

[1146] ---

[1147] The specific operation of each step is clarified and the processing flow from input to output is shown, making it easier to understand the operation of the entire system.

[1148] (Application example 2)

[1149] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1150] While early detection and rapid response to vehicle crimes are required, current systems are unable to consider the user's emotional state or level of urgency. This can result in a lack of accuracy in crime detection and the effectiveness of emergency notifications. Furthermore, there is no way to effectively utilize information provided by users via direct photography, making it difficult to accurately detect crimes in real time and respond quickly.

[1151] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting image data in real time from cameras installed on traffic lights, public buildings, and vehicles, means for analyzing the image data and extracting characteristic information about the vehicle, means for comparing the characteristic information with a pre-stored crime database and vehicle inspection database, means for notifying the police or fire department when an abnormality is detected, means for a user to capture video of the surroundings using a mobile communication device and collect emotional information, and means for adjusting the urgency of the analysis results based on the emotional information of the user. This makes it possible to adjust the urgency taking into account the emotional state of the user, thereby achieving a faster and more appropriate response.

[1152] A "traffic light" is a device installed to control traffic flow and issue instructions to vehicles and pedestrians.

[1153] "Public building" means a building open to the public, including government agencies and facilities providing public services.

[1154] "Vehicle" means a mechanical device, usually an engine-powered means of transportation, designed to transport people or goods.

[1155] A "camera" is an optical device for capturing video or images, and includes a lens and an image sensor.

[1156] "Image data" is data that digitally represents visual information acquired by a device such as a camera.

[1157] "User" refers to an individual or entity that operates a system or device and provides information.

[1158] "Mobile communication device" means a portable device that can send and receive data using wireless communication, including smartphones and tablets.

[1159] "Emotional information" is data that represents an individual's emotional state and is information collected based on voice analysis and facial expression recognition.

[1160] A "crime database" is a database for recording and managing information on crimes that have occurred in the past.

[1161] The "vehicle inspection database" is a database that stores and manages vehicle inspection history and registration information.

[1162] "Artificial intelligence" refers to computer systems that mimic or replicate human intelligence, including software that performs data analysis and pattern recognition.

[1163] "Analysis Results" refers to the conclusions or information obtained after processing and analyzing collected data.

[1164] "Police" means a government agency responsible for maintaining public safety and enforcing the law.

[1165] "Fire department" refers to a public institution that carries out fire prevention and extinguishing and emergency rescue activities.

[1166] "Notification" is the act or process of making specific information known to another person or system.

[1167] "Real-time" refers to a situation where data and information are processed and analyzed immediately, and results are provided without delay.

[1168] The present invention relates to a system for detecting motor vehicle crime and notifying appropriate authorities, optimizing vigilance and response based on the emotional state of the user. Specific embodiments for implementing the invention are described in detail herein.

[1169] System Configuration

[1170] The system of the present invention consists of the following main components:

[1171] 1. Data Collection Module

[1172] Cameras: Capture image data in real time from cameras installed on traffic lights, public buildings, and vehicles.

[1173] Mobile communication device: The user uses a mobile communication device (smartphone) to capture images of the surroundings and collect emotional information.

[1174] GPS: Location information is obtained using the GPS module built into the mobile communication device.

[1175] 2. Data Processing Module

[1176] Image preprocessing: Using a library such as OpenCV, image data acquired from the camera is denoised and segmented into individual frames.

[1177] Feature extraction: Object detection is performed using TensorFlow to extract vehicle feature information, and Tesseract is used for OCR processing to read license plate information.

[1178] Emotion analysis: Using artificial intelligence such as Watson Tone Analyzer, emotional information is analyzed from the voice data provided by the user.

[1179] 3. Database Matching Module

[1180] Database matching: The extracted vehicle feature information is matched with pre-stored crime databases and vehicle inspection databases.

[1181] 4. Notification Module

[1182] Notification system: Uses Firebase Cloud Messaging to notify police and fire departments when an abnormality is detected.

[1183] Example

[1184] For example, when a user takes a photo of a suspicious vehicle with a mobile communication device, the video and audio data are sent to a server. The server first extracts the vehicle's characteristics from the video and determines whether it is a target vehicle. At the same time, it analyzes the user's emotional information from the audio data and adjusts the alert level as necessary. If an abnormality is detected, the server immediately notifies the police or fire department and provides the vehicle's characteristics, location, and the user's emotional information.

[1185] Examples of prompts include:

[1186] If you notice a suspicious vehicle nearby, capture video and audio of the vehicle using your smartphone camera and microphone. The application will detect signs of crime from the video and audio and quickly notify the appropriate authorities based on the user's emotional state.

[1187] In this way, the present invention provides a system that effectively utilizes the video and emotional information provided by the user and optimizes the level of urgency and detail, thereby enabling a quick and appropriate response.

[1188] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1189] Step 1: Data collection

[1190] The user takes a video of the surroundings using the camera on their mobile communication device (smartphone). The camera captures video data in real time, and simultaneously collects audio data using the microphone. Location information is also obtained using the GPS module. The input data consists of video data, audio data, and location information. This data is sent to the server and used for subsequent analysis.

[1191] Step 2: Image preprocessing

[1192] The server performs preprocessing on the received video data. In preprocessing, OpenCV is used to remove noise and segment the image frame by frame. The input is the video data acquired in step 1, and the output is image data with noise removed and suitable for analysis. This process ensures that the subsequent feature extraction process can be performed accurately and efficiently.

[1193] Step 3: Feature extraction

[1194] The server extracts vehicle feature information from the preprocessed image data. It uses TensorFlow to perform object detection and identify the vehicle's position and shape. It also uses Tesseract to perform OCR processing of license plates. The input is the preprocessed image data, and the output is vehicle feature information and license plate information. This allows for vehicle identification.

[1195] Step 4: Sentiment Analysis

[1196] The server analyzes the voice data collected by the user when taking a photo. It uses artificial intelligence such as Watson Tone Analyzer to obtain the user's emotional information. The input is the voice data obtained in step 1, and the output is the analyzed emotional information of the user. This allows the emotion the user was feeling at the time to be detected.

[1197] Step 5: Database Matching

[1198] The server compares the extracted vehicle characteristic information and license plate information with the crime database and vehicle inspection database. This comparison determines whether the vehicle is a stolen or illegal vehicle. The input is the vehicle characteristic information and license plate information obtained in step 3, and the output is the comparison result. This allows anomalies to be detected.

[1199] Step 6: Adjusting urgency

[1200] The server integrates the matching results with the user's emotional information and adjusts the urgency of the anomaly. If the urgency is high based on the emotional information, it increases the priority of notification to the police and fire department. The input is the emotional information acquired in step 4 and the matching results obtained in step 5, and the output is the adjusted urgency information. This optimizes the urgency.

[1201] Step 7: Notification

[1202] The server notifies the police and fire department of the adjusted urgency information and details of the abnormality. It uses a notification system such as Firebase Cloud Messaging to transmit the information to the relevant agencies in real time. The input is the urgency information and details of the abnormality obtained in step 6, and the output is a notification message to the police and fire department. This enables a rapid response.

[1203] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1204] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1205] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1206] [Fourth embodiment]

[1207] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1208] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1209] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1210] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1211] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1212] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1213] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1214] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1215] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1216] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1217] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1218] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1219] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1220] The present invention relates to a system that detects automobile crimes by collecting image data using cameras installed on traffic lights, public buildings, and vehicles, and analyzing the collected image data in real time, and then notifies the relevant authorities. Below, we will create a program for this system and explain its processing with specific examples.

[1221] Implementation of continuous monitoring

[1222] 1. Data Collection

[1223] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[1224] For example, a camera attached to a traffic light at an intersection captures video of passing vehicles.

[1225] 2. Image Preprocessing

[1226] The server performs pre-processing on the received image data, including noise removal and frame-by-frame image segmentation.

[1227] This process removes unnecessary information from the image data, making it suitable for analysis.

[1228] 3. Feature Extraction

[1229] The server extracts vehicle feature information from the preprocessed image data, specifically identifying the vehicle model, license plate number, and driving situation using object detection algorithms (e.g., YOLO, SSD) and OCR technology.

[1230] For example, the license plate number of a particular vehicle can be read from an image captured by a camera and the information stored in a database.

[1231] 4. Database Verification

[1232] The server compares the extracted vehicle license plates with a criminal database and a vehicle inspection information database, and this comparison detects any anomalies.

[1233] For example, it checks whether the license plate being read is on a list of stolen vehicles.

[1234] 5. Anomaly detection and notification

[1235] If the server detects an abnormality based on the comparison results, it notifies the police or fire department, including the vehicle's characteristics and location information.

[1236] For example, if a stolen vehicle is detected, the information and current location will be sent to the police in real time.

[1237] This series of processes enables a rapid response when a car crime is detected during road surveillance.

[1238] Emergency confirmation implementation

[1239] 1. Data Reception

[1240] The server receives videos and photos taken by users via email or messaging apps.

[1241] For example, a user may take a video of a hit-and-run incident and send the video to a server via email.

[1242] 2. Data storage

[1243] The server temporarily stores the received data, including metadata such as sender information and the date and time of receipt.

[1244] 3. Image Preprocessing

[1245] The server preprocesses the stored data, making each frame individually analyzable, by removing noise and adjusting the resolution.

[1246] 4. Feature Extraction

[1247] The server extracts vehicle feature information from the pre-processed data in real time, again using object detection algorithms and OCR technology.

[1248] 5. Urgent Analysis and Notification

[1249] The server compares the extracted characteristic information with a crime database, and if it detects an abnormality, it notifies the police or fire department of the results.

[1250] For example, if the license plate of a photographed vehicle is found to be on a list of hit-and-run vehicles, the information will be immediately notified to the police.

[1251] By using the above means, the present invention is a system that realizes early detection and rapid response to automobile crimes through both constant monitoring and emergency confirmation.

[1252] The processing flow will be explained below.

[1253] Continuous monitoring process steps

[1254] Step 1:

[1255] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[1256] Step 2:

[1257] The server performs preprocessing on the received image data. In this step, it removes noise from the image and divides the image into frames. This process removes unnecessary information from the image data and makes it suitable for analysis.

[1258] Step 3:

[1259] The server extracts vehicle feature information from the preprocessed image data, using object detection algorithms (e.g., YOLO, SSD) and OCR technology to identify the vehicle type, license plate number, and driving situation. For example, it reads the license plate number of a specific vehicle from the image captured by the camera.

[1260] Step 4:

[1261] The server compares the extracted vehicle license plate information with a criminal database and a vehicle inspection information database, and this comparison detects any anomalies.

[1262] Step 5:

[1263] The server determines an anomaly based on the matching result, for example, if a license plate matching a stolen vehicle list is detected.

[1264] Step 6:

[1265] If an abnormality is detected, the server immediately notifies the police or fire department. The notification includes vehicle characteristics (such as license plate, model, and location information), allowing the relevant authorities to respond quickly.

[1266] Processing steps for urgent confirmation

[1267] Step 1:

[1268] The server receives videos or photos taken by users via email or messaging apps. For example, a user may take a video of a hit-and-run accident and send the video via email.

[1269] Step 2:

[1270] The server temporarily stores the received data, which also includes metadata such as sender information and the date and time of receipt.

[1271] Step 3:

[1272] The server preprocesses the stored data, which involves removing noise and adjusting the resolution so that each frame can be analyzed individually.

[1273] Step 4:

[1274] The server extracts vehicle feature information from the pre-processed data in real time, again using object detection algorithms and OCR technology.

[1275] Step 5:

[1276] The server compares the extracted feature information with a crime database to detect anomalies, such as when the license plate of the photographed vehicle is included in a list of hit-and-run vehicles.

[1277] Step 6:

[1278] If the server detects an abnormality, it immediately notifies the police or fire department. The notification includes analyzed characteristic information and sender information, allowing the relevant authorities to respond quickly.

[1279] Example 1

[1280] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1281] In modern society, improving the safety and efficiency of transportation infrastructure is an important issue. Rapid detection and response to vehicle crimes is particularly important, but conventional surveillance systems have struggled to accurately detect abnormalities in real time and notify relevant authorities. Furthermore, there is a lack of means for users to efficiently analyze data captured in emergency situations and respond quickly. To address these issues, a system that combines advanced image processing and database matching technologies is needed.

[1282] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1283] In this invention, the server includes means for collecting visible data in real time from optical devices installed on traffic lights, public buildings, and vehicles, means for preprocessing the visible data to make it analyzable, means for extracting characteristic information about the vehicle from the visible data, means for comparing the characteristic information with a pre-stored database, and means for notifying relevant authorities when an abnormality is detected, thereby enabling early detection and rapid response to automobile crimes.

[1284] The server also includes a means for receiving visible data captured by a user, a means for preprocessing the received visible data to make it analyzable, a means for extracting characteristic information of the vehicle from the visible data, a means for comparing the characteristic information with a pre-stored database, a means for notifying relevant organizations when an abnormality is detected, and a means for managing the analysis results sent to the relevant organizations by the notifying means, thereby enabling emergency information from users to be processed quickly and accurately.

[1285] Furthermore, by connecting the optical equipment to a device equipped with a machine learning algorithm and having the means to automatically analyze the visible data, high-precision analysis can be performed in real time, improving the accuracy of detecting vehicle crimes and enabling rapid response.

[1286] "Optical equipment" refers to cameras and sensors used to acquire image and video data.

[1287] "Visible data" refers to image data or video data acquired by optical instruments, and includes visual information to be analyzed.

[1288] "Preprocessing" refers to a series of processing operations performed to prepare visible data for analysis, including noise removal, image segmentation, and resolution adjustment.

[1289] "Feature information" refers to significant information extracted from visible data, specifically vehicle model, license plate number, driving conditions, etc.

[1290] A "database" is a computer system that systematically stores information for later retrieval or collation, and includes crime databases and vehicle inspection databases.

[1291] A "machine learning algorithm" is an algorithm that automatically learns patterns and rules from data and uses that knowledge to analyze new data.

[1292] "Relevant agencies" are agencies to which notification will be sent if an abnormality such as a car crime is detected, specifically the police and fire department.

[1293] "Means of notification" refers to the communication method used to notify relevant authorities of an abnormality, and includes SMS, email, voice calls, etc.

[1294] "Receiving means" refers to the method or function by which the server receives the visible data sent by the user.

[1295] "Management means" refers to the method or function for recording and storing the analysis results sent to the relevant authorities by the notification means, and for use in future reference and analysis.

[1296] This invention is a system that collects visible data in real time from optical devices installed on traffic lights, public buildings, and vehicles, preprocesses this data to extract vehicle characteristic information, compares it with a crime database, and notifies relevant authorities if an abnormality is detected. The following hardware and software are used to implement this invention.

[1297] Hardware and software used

[1298] 1. Optical equipment

[1299] Cameras and sensors are used as optical instruments. For example, surveillance cameras are installed to capture data in real time 24 hours a day, 365 days a year.

[1300] 2. Server

[1301] The server preprocesses the received visible data, extracts feature information, compares it with a database, and notifies users when an anomaly is detected. The following software is used on the server:

[1302] "OpenCV": Used for preprocessing the visual data, which includes denoising and frame-by-frame segmentation.

[1303] "YOLO" or "SSD": Used as an object detection algorithm to extract vehicle feature information from visual data.

[1304] "Tesseract OCR": Uses text recognition technology to analyze the characters on license plates.

[1305] "Elasticsearch" or "MySQL": Used to match the extracted feature information with the database.

[1306] "Twilio API": Used to notify relevant authorities (e.g., police and fire department) when an anomaly is detected.

[1307] 3. Communication Network

[1308] A network for transmitting data from cameras and sensors to a server. Data is transferred via the Internet or a dedicated data transfer network.

[1309] System operation example

[1310] For example, consider a surveillance camera installed at a traffic light at an intersection. This camera captures video data of passing vehicles in real time. This data is then sent to a server via a communication network.

[1311] 1. Data Collection

[1312] The server collects visible data received from optical devices, in this case video data transmitted from surveillance cameras.

[1313] 2. Image Preprocessing

[1314] The server preprocesses the collected visual data, for example, using OpenCV to perform image processing such as noise removal and frame segmentation, thereby creating data suitable for analysis.

[1315] 3. Feature Extraction

[1316] The server uses object detection algorithms such as YOLO and SSD to extract vehicle feature information from the preprocessed data, then uses Tesseract OCR to read the license plate characters and store them in a database.

[1317] 4. Database Verification

[1318] The server compares the extracted feature information with databases stored in Elasticsearch or MySQL, thereby detecting anomalies by comparing them with criminal databases and vehicle inspection information databases.

[1319] 5. Anomaly detection and notification

[1320] If the server detects an abnormality, it will use the Twilio API to notify the police, fire department, and other relevant authorities. This notification will include information about the vehicle's characteristics and current location, enabling a prompt response.

[1321] Prompt Sentence Examples

[1322] Prompt: Please explain the processing procedure of this system in natural language. Please specify what hardware or software is used to process or calculate data. Please also specify the specific name of the hardware or software used.

[1323] 1. The server receives real-time visual data from optical devices installed on traffic lights, public buildings, and vehicles, for example, dedicated surveillance cameras.

[1324] 2. The server preprocesses the received visual data. In this step, it uses OpenCV to remove noise and segment the image for each frame.

[1325] 3. The server extracts vehicle feature information from the preprocessed visual data using object detection algorithms such as YOLO and SSD, as well as Tesseract OCR.

[1326] 4. The server compares the extracted vehicle feature information with the database, using Elasticsearch and MySQL for this comparison.

[1327] 5. If the server detects an anomaly based on the matching results, it notifies the relevant authorities using the "Twilio API."

[1328] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1329] Program processing steps

[1330] Step 1:

[1331] The server collects visible data in real time from optical devices installed in traffic lights, public buildings, and vehicles. The input data from these optical devices includes images and videos. The input data is stored as is and then moved on to the next processing step. Specifically, surveillance cameras capture image data in real time 24 hours a day, 365 days a year, and send it to the server via the Internet. The received data is then stored in the server's storage.

[1332] Step 2:

[1333] The server preprocesses the collected visual data to make it ready for analysis. This process uses image processing techniques to remove noise and segment each frame. The input is the image data collected in step 1, and the output is the preprocessed image data. Specific examples include using OpenCV to remove noise from the image data and adjust the image resolution as needed. This ensures that the data used for analysis is of high quality.

[1334] Step 3:

[1335] The server extracts vehicle feature information from the preprocessed visual data. This process uses an object detection algorithm to identify features such as the vehicle model and license plate. The input is the image data preprocessed in step 2, and the output is vehicle feature information. Specifically, vehicles are detected using "YOLO" or "SSD," and text information from license plates is extracted using "Tesseract OCR." The extracted information is temporarily stored in the server's memory.

[1336] Step 4:

[1337] The server compares the extracted vehicle characteristics information with a database. This comparison checks for matching data in criminal databases or vehicle inspection databases. The input is the vehicle characteristics information extracted in step 3, and the output is the results of an anomaly inspection. For example, a query is run against a database stored in Elasticsearch or MySQL to match license plates and vehicle characteristics.

[1338] Step 5:

[1339] If the server detects an abnormality based on the matching results, it will take measures to notify the relevant authorities. The notification content includes vehicle characteristics and location information. The input is the abnormality inspection results obtained in step 4, and the output is the notified information. Specifically, it uses the "Twilio API" to send real-time notifications to the police and fire department via email, SMS, voice calls, etc. It may also use the "Google Maps API" to obtain location information, allowing the relevant authorities to respond quickly.

[1340] Step 6:

[1341] A user takes a photo of an accident scene and sends the image data to the server. This data is sent to the server via email or a messaging app. The input is the image data sent by the user, and the output is the data received by the server. For example, this includes the user taking a photo of the accident scene and sending the image to the server via email.

[1342] Step 7:

[1343] The server preprocesses the image data received from the user to make it ready for analysis. The input is the image data received in step 6, and the output is the preprocessed image data. As a specific example, "OpenCV" is used as part of the preprocessing to remove noise and divide the image frames.

[1344] Step 8:

[1345] The server extracts vehicle feature information from the preprocessed data. The input is the image data preprocessed in step 7, and the output is the vehicle feature information. Specific operations include detecting vehicles using the YOLO model and reading license plates using Tesseract OCR.

[1346] Step 9:

[1347] The server compares the extracted feature information with the database and notifies the relevant authorities if an abnormality is detected. The input is the vehicle feature information extracted in step 8, and the output is the notification result. As a specific example, a query is executed against the database, and if an abnormality is detected, the relevant authorities are notified using the "Twilio API." This process enables a fast and accurate response even in emergencies.

[1348] (Application example 1)

[1349] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1350] The problem that this invention aims to solve is to detect automobile crimes in parking lots and on roads in real time and respond promptly and appropriately. Conventional technologies have had the problem that it takes time from the time a crime occurs until it is reported, making it difficult to respond quickly. In addition, there was a lack of means to efficiently identify vehicles illegally entering parking lots or stolen vehicles, which placed a heavy burden on managers and police.

[1351] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1352] In this invention, the server includes means for collecting image data in real time from imaging devices installed in traffic signals, public facilities, and vehicles, means for analyzing the image data and extracting characteristic information about the vehicle, means for comparing the characteristic information with a pre-stored crime database and vehicle inspection database, means for notifying an administrator and the police when an abnormality is detected, and means for installing a predetermined application on a smart device and displaying images and notifications in real time when an abnormality is detected, thereby enabling fast and efficient detection and management of automobile crimes in parking lots and on roads.

[1353] A "signal device" is a device that emits a signal, and includes traffic lights and the like for controlling traffic flow.

[1354] "Public facilities" are facilities that are available to the general public, such as city halls, schools, and train stations.

[1355] An "imaging device" is a device for capturing images or videos, and includes cameras and video cameras.

[1356] "Real-time" refers to processing or responding to an event as it occurs.

[1357] "Image data" refers to digital data of images or videos captured by an imaging device.

[1358] "Characteristic information" is information that indicates specific attributes or characteristics extracted from analyzed data, such as license plates and vehicle models.

[1359] A "crime database" is a database that stores information related to past crimes.

[1360] A "vehicle inspection database" is a database that stores vehicle registration information and inspection history.

[1361] "Matching" refers to comparing one piece of data with another to see if they match.

[1362] "Abnormal" refers to an unusual or unexpected condition, such as the discovery of a stolen vehicle.

[1363] "Notification" means the communication of specific information to another person.

[1364] "Administrator" refers to a person responsible for operating and monitoring a system or facility.

[1365] "Police" means an agency that enforces the law to maintain public order and safety.

[1366] "Smart devices" refer to electronic devices with advanced functions, including smartphones and head-mounted displays.

[1367] An "application" is a software program that provides a particular function or service.

[1368] The system for implementing this invention collects image data in real time from imaging devices installed in traffic signals, public facilities, and vehicles, analyzes the image data to extract characteristic information about the vehicle, compares it with a crime database and a vehicle inspection database, and notifies administrators and the police if an abnormality is detected. It also includes displaying this information in real time on a smart device with a specified application installed.

[1369] Hardware

[1370] The system includes the following hardware:

[1371] Imaging devices (cameras): Installed on traffic signals, public facilities, and vehicles.

[1372] Server (data processing server): Receives image data in real time and performs analysis.

[1373] Smart Devices: Includes smartphones and head-mounted displays (HMDs), which receive notifications and display footage in real time.

[1374] software

[1375] The software used is as follows:

[1376] YOLOv5: Used as an object detection algorithm.

[1377] OpenCV: A library for image processing.

[1378] Requests: A library for making HTTP requests.

[1379] Python: The primary development language in which the entire program is written.

[1380] Data processing and calculation

[1381] After receiving image data from the imaging device in real time, the server performs the following processes:

[1382] 1. Preprocessing of image data: noise removal and resolution adjustment.

[1383] 2. Feature extraction: YOLOv5 is used to extract vehicle features, specifically identifying license plates and vehicle types.

[1384] 3. Database matching: The extracted feature information is matched with the crime database and vehicle inspection database to detect suspicious vehicles.

[1385] 4. Notification: If an abnormality is detected, a notification will be sent to the smart device and detailed information will be forwarded to the administrator and police.

[1386] Example

[1387] As an example, we provide a parking lot security monitor. When the parking lot manager launches the "Parking Lot Security Monitor" app on their smartphone, real-time video footage of the parking lot is displayed. If a suspicious vehicle is detected, details are displayed in the smartphone's notification center. In addition, by using a head-mounted display, the situation inside the parking lot can be directly viewed.

[1388] Prompt Sentence Examples

[1389] "Create a Python program that uses YOLOv5 to extract vehicle characteristics from parking lot surveillance cameras in real time, and notify the manager and police if there are any abnormalities based on the matching results."

[1390] The system enables fast and efficient detection and management of car crime in parking lots and on roads.

[1391] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1392] Step 1:

[1393] The server collects image data in real time from image capture devices installed on traffic signals, public facilities, and vehicles. As input, it receives image data transmitted from each image capture device. As output, it stores the raw image data in memory. This data is used for subsequent processing. Specifically, the server retrieves data from the URL of the camera stream.

[1394] Step 2:

[1395] The server preprocesses the received image data. It uses raw image data as input. It gets preprocessed image data as output. This preprocessing includes noise removal and resolution adjustment. Specifically, it uses the OpenCV library to remove noise and adjust the resolution of the image.

[1396] Step 3:

[1397] The server extracts vehicle feature information from the preprocessed image data. The preprocessed image data is used as input. Feature information such as license plates and vehicle types is obtained as output. Specifically, object detection is performed using YOLOv5, and license plate recognition is performed.

[1398] Step 4:

[1399] The server compares the extracted feature information with the pre-stored crime database and vehicle inspection database. The feature information and database information are used as input. The comparison result is obtained as output. Specifically, the server sends an HTTP request to query the database.

[1400] Step 5:

[1401] If the server detects an anomaly based on the matching results, it notifies the administrator and the police. It uses the matching results as input, and generates and sends a notification message as output. Specifically, it implements a function to send an alert via the notification system.

[1402] Step 6:

[1403] When an anomaly is detected, the server installs a specific application on the smart device and displays images and notifications in real time. The server uses the anomaly detection information as input. The server sends a notification to the smart device as output. Specifically, the server displays an alert in the notification center of the smartphone or head-mounted display.

[1404] Step 7:

[1405] The server manages the collected and processed data. It uses feature information, matching results, and notification history as input. It obtains analysis results as output, which are stored in a database. Specific operations include recording all data to a central server, making it available for later analysis and review.

[1406] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1407] This invention relates to a system that detects automobile crimes and notifies relevant authorities by combining an emotion engine with a system that optimizes alertness and response based on the user's emotion information. Below, we will create a program for this system and explain its processing with specific examples.

[1408] Implementation of continuous monitoring

[1409] 1. Data Collection

[1410] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[1411] For example, a camera attached to a traffic light at an intersection captures video of passing vehicles.

[1412] 2. Image Preprocessing

[1413] The server performs pre-processing on the received image data, including noise removal and frame-by-frame image segmentation.

[1414] This removes unnecessary information from the image data, making it suitable for analysis.

[1415] 3. Feature Extraction

[1416] The server extracts vehicle feature information from the pre-processed image data, using object detection algorithms (e.g., YOLO, SSD) and OCR technology to identify the vehicle model, license plate number, and driving situation.

[1417] For example, the license plate number of a particular vehicle can be read from an image captured by a camera.

[1418] 4. Database Verification

[1419] The server compares the extracted vehicle license plate information with the crime database and vehicle inspection information database, and any anomalies are detected through the comparison process.

[1420] For example, it checks whether the license plate being read is on a list of stolen vehicles.

[1421] 5. Anomaly Detection

[1422] The server determines an anomaly based on the matching result, for example, if a license plate matching a stolen vehicle list is detected.

[1423] 6. Notice and Information

[1424] If an abnormality is detected, the server notifies the police or fire department, and the notification includes information about the vehicle's characteristics and location.

[1425] For example, if a stolen vehicle is detected, the information and current location will be sent to the police in real time.

[1426] Emergency confirmation implementation

[1427] 1. Data Reception

[1428] The server receives videos and photos taken by users via email or messaging apps, and simultaneously collects their emotional information.

[1429] For example, a user can film the scene of a hit-and-run accident and send the video along with emotional information from that time via email.

[1430] 2. Data storage

[1431] The server temporarily stores the received data, including metadata such as sender information and the date and time of receipt.

[1432] 3. Image Preprocessing

[1433] The server preprocesses the stored data, which involves removing noise and adjusting the resolution, making each frame individually analyzable.

[1434] 4. Feature Extraction

[1435] The server extracts vehicle feature information from the preprocessed data in real time using object detection algorithms and OCR technology, and the emotion engine analyzes the user's emotion information.

[1436] For example, the license plate of a photographed vehicle is analyzed, and at the same time, the user's emotional information is analyzed.

[1437] 5. Emergency analysis

[1438] The server compares the extracted feature information with a criminal database to detect abnormalities, prioritizes the analysis results based on the user's emotional information, and determines how to respond based on the level of urgency.

[1439] For example, if the license plate of a photographed vehicle matches a list of hit-and-run vehicles, and the user feels extreme shock or fear, an immediate alert will be sent to the police.

[1440] 6. Notification of Results and Response

[1441] If the server detects an abnormality, it immediately notifies the police or fire department. The notification includes analyzed feature information and the user's emotional information. The urgency and detail of the notification are adjusted based on the emotional information.

[1442] For example, if more detailed information or a faster response is required based on the user's emotional information, this will be communicated to the police.

[1443] The system of this invention achieves early detection and rapid response to car crimes by using a dual approach of continuous monitoring and emergency confirmation. Furthermore, by combining it with an emotion engine, the urgency and detail of the analysis results are optimized, aiming to enable faster and more appropriate responses from relevant authorities.

[1444] The processing flow will be explained below.

[1445] Continuous monitoring process steps

[1446] Step 1:

[1447] The server receives real-time image data from cameras installed on traffic lights, public buildings, and vehicles, allowing for constant monitoring of vehicles on the roads.

[1448] Step 2:

[1449] The server performs preprocessing on the received image data. In this step, it removes noise from the image and divides the image into frames. This process removes unnecessary information from the image data and makes it suitable for analysis.

[1450] Step 3:

[1451] The server extracts vehicle feature information from the preprocessed image data, using object detection algorithms (e.g., YOLO, SSD) and OCR technology to identify the vehicle type, license plate number, and driving situation. For example, it reads the license plate number of a specific vehicle from the image captured by the camera.

[1452] Step 4:

[1453] The server compares the extracted vehicle license plate information with a criminal database and a vehicle inspection information database, and any anomalies are detected through this comparison.

[1454] Step 5:

[1455] The server determines an anomaly based on the matching result, for example, if a license plate matching a stolen vehicle list is detected.

[1456] Step 6:

[1457] If an abnormality is detected, the server immediately notifies the police or fire department. The notification includes vehicle characteristics (such as license plate, model, and location information), allowing the relevant authorities to respond quickly.

[1458] Processing steps for urgent confirmation

[1459] Step 1:

[1460] The server receives videos or photos taken by users via email or messaging apps, and simultaneously collects the user's emotional information. For example, a user may take a photo of a hit-and-run accident scene and send the video and their emotional information via email.

[1461] Step 2:

[1462] The server temporarily stores the received data, including metadata such as sender information and the date and time of receipt.

[1463] Step 3:

[1464] The server preprocesses the stored data, a step that involves removing noise and adjusting resolution so that each frame can be analyzed individually.

[1465] Step 4:

[1466] The server extracts vehicle feature information from the preprocessed data in real time, again using object detection algorithms and OCR technology. The emotion engine then analyzes the user's emotional information. For example, the server analyzes the license plate of the vehicle in the photo and simultaneously analyzes the user's emotional information.

[1467] Step 5:

[1468] The server compares the extracted feature information with a crime database to detect any abnormalities. It prioritizes the analysis results based on the user's emotional information and determines how to respond based on the level of urgency. For example, if the license plate of a photographed vehicle matches a list of hit-and-run vehicles, and the user feels extreme shock or fear, it will immediately send a report to the police.

[1469] Step 6:

[1470] If an abnormality is detected, the server immediately notifies the police or fire department. The notification content includes analyzed feature information and the user's emotional information. The urgency and level of detail of the notification are adjusted based on the emotional information. For example, if more detailed information or a faster response is required based on the user's emotional information, this information is communicated to the police. The system of the present invention achieves early detection and rapid response of car crimes using a dual approach of continuous monitoring and emergency confirmation. Furthermore, by combining an emotion engine, the urgency and level of detail of the analysis results are optimized, aiming to enable faster and more appropriate responses from relevant authorities.

[1471] Example 2

[1472] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1473] Conventional vehicle crime monitoring systems are capable of collecting and analyzing image data in real time, but lack the ability to prioritize emergency responses based on user emotional information. This makes it difficult for relevant authorities to respond appropriately and quickly when a crime is detected. Furthermore, when analyzing data based on emergency calls from users, it is difficult to determine the reliability and urgency of the data, which can cause delays in situations where a rapid response is required. A new system is needed to solve these problems.

[1474] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting image data in real time from sensors installed on traffic lights, public buildings, and vehicles, means for preprocessing the image data and extracting vehicle feature information, means for comparing the feature information with a pre-stored database, means for collecting user emotion information and setting the urgency level of the analysis results, and means for notifying relevant authorities when an abnormality is detected. This enables relevant authorities to respond appropriately and quickly when a crime is detected.

[1475] A "traffic light" is a device installed to control traffic on a road and displays signals to vehicles and pedestrians.

[1476] A "public building" is a building used for public purposes and functions as an administrative agency or public facility.

[1477] A "sensor" is a device that detects physical changes and outputs them as electrical signals.

[1478] "Image data" is a digital representation of optical information acquired by a camera or sensor.

[1479] "Preprocessing" refers to processing carried out before data analysis, such as removing noise and shaping data.

[1480] "Vehicle characteristic information" is information necessary for identifying and classifying vehicles, and includes license plates, vehicle models, colors, etc.

[1481] A "database" is a systematically organized collection of data that allows for efficient data retrieval and updating.

[1482] "User emotion information" is digital data that expresses the emotion felt by the user in a particular situation.

[1483] The "urgency of the analysis results" is an index that indicates the priority and urgency of the response based on the analyzed data.

[1484] "Related agencies" refers to administrative agencies and public facilities that respond to crimes and emergencies, including police and fire departments.

[1485] This invention is a surveillance system for detecting automobile crimes and notifying relevant authorities, with the aim of optimizing the level of vigilance and the urgency of response by using user emotional information in particular. This system detects crimes by collecting and analyzing data from cameras and sensors installed in traffic lights, public buildings, and vehicles. Furthermore, it analyzes emotional information based on emergency calls from users and notifies relevant authorities based on that information.

[1486] Hardware and Software

[1487] Hardware

[1488] Cameras and sensors: installed on traffic lights, public buildings, and vehicles. They collect image data in real time.

[1489] Server: A high-performance computing device for data collection, pre-processing, analysis, and notification.

[1490] Cloud storage: Used for temporary data storage and backup.

[1491] software

[1492] Image analysis algorithms: Object detection algorithms (YOLO, SSD, etc.) and OCR technology are used.

[1493] Databases: Crime database and vehicle inspection database.

[1494] Emotion engine: Used to analyze the user's emotional information and set the level of urgency.

[1495] Notification system: Communication software that notifies relevant authorities when an abnormality is detected.

[1496] Data flow

[1497] Data collection

[1498] The server collects image data in real time from cameras and sensors installed in traffic lights, public buildings, and vehicles. For example, a camera installed at an intersection captures images of a specific vehicle and sends the data to the server.

[1499] Pretreatment

[1500] The server preprocesses the received image data. In this step, noise is removed and the image is divided into frames to prepare it for analysis. For example, unnecessary parts are removed from the captured image to extract only the vehicle.

[1501] Analysis and Feature Extraction

[1502] The server extracts vehicle feature information from the preprocessed image data, using object detection algorithms such as YOLO, SSD, and OCR technology, analyzing features such as license plates, car models, and colors.

[1503] Database Matching

[1504] The server compares the extracted feature information with a criminal database and a vehicle inspection database, allowing it to detect anomalies, such as whether the license plate is included in a list of stolen vehicles.

[1505] Sentiment analysis and urgency setting

[1506] The server collects and analyzes emotional information along with video and image data based on emergency calls from users. The emotion engine sets the urgency level of the analysis results based on the user's emotional information. For example, if the user feels extreme shock or fear, the urgency level is set high.

[1507] notification

[1508] If an abnormality is detected, the server notifies the relevant authorities. The notification content includes vehicle characteristics, location information, and user emotion information. For example, if a stolen vehicle is detected, the police will be notified in real time along with its location information.

[1509] Specific examples

[1510] For example, if a camera installed at an intersection detects what appears to be a stolen vehicle, the image data is sent to a server. The server preprocesses the image, extracts features, and compares them with a database. If it matches a stolen vehicle, the server immediately notifies the police. Also, if a user witnesses a hit-and-run incident, the video is sent to the server along with emotional information. The server uses an emotion engine to set the urgency of the analysis results to a high level and promptly notify the police.

[1511] Prompt Sentence Examples

[1512] Here are some examples of prompts to input to the generative AI model:

[1513] "Please explain the process of a system that detects hit-and-run incidents and notifies the police based on user-recorded vehicle video and its emotional information."

[1514] This invention realizes early detection and rapid response to car crimes using a dual approach of continuous monitoring and emergency confirmation. In addition, the introduction of an emotion engine allows for faster and more appropriate notification to relevant authorities, contributing to improved safety for society as a whole.

[1515] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1516] Step 1: Data collection

[1517] Input: Real-time image data captured by cameras and sensors.

[1518] How it works: The server collects image data in real time from cameras and sensors installed on traffic lights, public buildings, and vehicles. This data is sent to the server via the internet or dedicated lines.

[1519] Output: Raw image data stored in the server storage.

[1520] Step 2: Image preprocessing

[1521] Input: Raw image data stored in storage.

[1522] How it works: The server applies a noise reduction filter to the received image data, splits the image into frames, cuts out the background and unnecessary parts, and formats the image in a way that is suitable for analysis.

[1523] Output: Preprocessed and clean image data.

[1524] Step 3: Feature extraction

[1525] Input: Preprocessed and clean image data.

[1526] How it works: The server uses an object detection algorithm (e.g., YOLO, SSD) to identify vehicles and OCR technology to read license plates, specifically identifying the make, color, and number plate characters in the image.

[1527] Output: Vehicle characteristics information (model, color, license plate, etc.).

[1528] Step 4: Database Matching

[1529] Input: Extracted vehicle feature information.

[1530] Operation: The server compares the extracted vehicle characteristics with the crime database and vehicle inspection database to determine whether the vehicle is a stolen or illegal vehicle.

[1531] Output: Matching result (whether anomalies are found or not).

[1532] Step 5: Anomaly detection

[1533] Input: Match results obtained from database matching.

[1534] How it works: The server determines whether something is an anomaly based on the match. For example, if the license plate matches a list of stolen vehicles, it will flag it as an anomaly.

[1535] Output: Anomaly detection flag and its detailed information.

[1536] Step 6: Notification and Information

[1537] Input: Anomaly detection flag and its details.

[1538] Operation: If an abnormality is detected, the server notifies the relevant authorities (police or fire department). The notification includes vehicle characteristics and location information. For example, the location information of a stolen vehicle is sent to the police in real time.

[1539] Output: Notification information sent to the appropriate authorities.

[1540] ---

[1541] The specific operation of each step is clarified and the processing flow from input to output is shown, making it easier to understand the operation of the entire system.

[1542] (Application example 2)

[1543] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1544] While early detection and rapid response to vehicle crimes are required, current systems are unable to consider the user's emotional state or level of urgency. This can result in a lack of accuracy in crime detection and the effectiveness of emergency notifications. Furthermore, there is no way to effectively utilize information provided by users via direct photography, making it difficult to accurately detect crimes in real time and respond quickly.

[1545] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting image data in real time from cameras installed on traffic lights, public buildings, and vehicles, means for analyzing the image data and extracting characteristic information about the vehicle, means for comparing the characteristic information with a pre-stored crime database and vehicle inspection database, means for notifying the police or fire department when an abnormality is detected, means for a user to capture video of the surroundings using a mobile communication device and collect emotional information, and means for adjusting the urgency of the analysis results based on the emotional information of the user. This makes it possible to adjust the urgency taking into account the emotional state of the user, thereby achieving a faster and more appropriate response.

[1546] A "traffic light" is a device installed to control traffic flow and issue instructions to vehicles and pedestrians.

[1547] "Public building" means a building open to the public, including government agencies and facilities providing public services.

[1548] "Vehicle" means a mechanical device, usually an engine-powered means of transportation, designed to transport people or goods.

[1549] A "camera" is an optical device for capturing video or images, and includes a lens and an image sensor.

[1550] "Image data" is data that digitally represents visual information acquired by a device such as a camera.

[1551] "User" refers to an individual or entity that operates a system or device and provides information.

[1552] "Mobile communication device" means a portable device that can send and receive data using wireless communication, including smartphones and tablets.

[1553] "Emotional information" is data that represents an individual's emotional state and is information collected based on voice analysis and facial expression recognition.

[1554] A "crime database" is a database for recording and managing information on crimes that have occurred in the past.

[1555] The "vehicle inspection database" is a database that stores and manages vehicle inspection history and registration information.

[1556] "Artificial intelligence" refers to computer systems that mimic or replicate human intelligence, including software that performs data analysis and pattern recognition.

[1557] "Analysis Results" refers to the conclusions or information obtained after processing and analyzing collected data.

[1558] "Police" means a government agency responsible for maintaining public safety and enforcing the law.

[1559] "Fire department" refers to a public institution that carries out fire prevention and extinguishing and emergency rescue activities.

[1560] "Notification" is the act or process of making specific information known to another person or system.

[1561] "Real-time" refers to a situation where data and information are processed and analyzed immediately, and results are provided without delay.

[1562] The present invention relates to a system for detecting motor vehicle crime and notifying appropriate authorities, optimizing vigilance and response based on the emotional state of the user. Specific embodiments for implementing the invention are described in detail herein.

[1563] System Configuration

[1564] The system of the present invention consists of the following main components:

[1565] 1. Data Collection Module

[1566] Cameras: Capture image data in real time from cameras installed on traffic lights, public buildings, and vehicles.

[1567] Mobile communication device: The user uses a mobile communication device (smartphone) to capture images of the surroundings and collect emotional information.

[1568] GPS: Location information is obtained using the GPS module built into the mobile communication device.

[1569] 2. Data Processing Module

[1570] Image preprocessing: Using a library such as OpenCV, image data acquired from the camera is denoised and segmented into individual frames.

[1571] Feature extraction: Object detection is performed using TensorFlow to extract vehicle feature information, and Tesseract is used for OCR processing to read license plate information.

[1572] Emotion analysis: Using artificial intelligence such as Watson Tone Analyzer, emotional information is analyzed from the voice data provided by the user.

[1573] 3. Database Matching Module

[1574] Database matching: The extracted vehicle feature information is matched with pre-stored crime databases and vehicle inspection databases.

[1575] 4. Notification Module

[1576] Notification system: Uses Firebase Cloud Messaging to notify police and fire departments when an abnormality is detected.

[1577] Example

[1578] For example, when a user takes a photo of a suspicious vehicle with a mobile communication device, the video and audio data are sent to a server. The server first extracts the vehicle's characteristics from the video and determines whether it is a target vehicle. At the same time, it analyzes the user's emotional information from the audio data and adjusts the alert level as necessary. If an abnormality is detected, the server immediately notifies the police or fire department and provides the vehicle's characteristics, location, and the user's emotional information.

[1579] Examples of prompts include:

[1580] If you notice a suspicious vehicle nearby, capture video and audio of the vehicle using your smartphone camera and microphone. The application will detect signs of crime from the video and audio and quickly notify the appropriate authorities based on the user's emotional state.

[1581] In this way, the present invention provides a system that effectively utilizes the video and emotional information provided by the user and optimizes the level of urgency and detail, thereby enabling a quick and appropriate response.

[1582] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1583] Step 1: Data collection

[1584] The user takes a video of the surroundings using the camera on their mobile communication device (smartphone). The camera captures video data in real time, and simultaneously collects audio data using the microphone. Location information is also obtained using the GPS module. The input data consists of video data, audio data, and location information. This data is sent to the server and used for subsequent analysis.

[1585] Step 2: Image preprocessing

[1586] The server performs preprocessing on the received video data. In preprocessing, OpenCV is used to remove noise and segment the image frame by frame. The input is the video data acquired in step 1, and the output is image data with noise removed and suitable for analysis. This process ensures that the subsequent feature extraction process can be performed accurately and efficiently.

[1587] Step 3: Feature extraction

[1588] The server extracts vehicle feature information from the preprocessed image data. It uses TensorFlow to perform object detection and identify the vehicle's position and shape. It also uses Tesseract to perform OCR processing of license plates. The input is the preprocessed image data, and the output is vehicle feature information and license plate information. This allows for vehicle identification.

[1589] Step 4: Sentiment Analysis

[1590] The server analyzes the voice data collected by the user when taking a photo. It uses artificial intelligence such as Watson Tone Analyzer to obtain the user's emotional information. The input is the voice data obtained in step 1, and the output is the analyzed emotional information of the user. This allows the emotion the user was feeling at the time to be detected.

[1591] Step 5: Database Matching

[1592] The server compares the extracted vehicle characteristic information and license plate information with the crime database and vehicle inspection database. This comparison determines whether the vehicle is a stolen or illegal vehicle. The input is the vehicle characteristic information and license plate information obtained in step 3, and the output is the comparison result. This allows anomalies to be detected.

[1593] Step 6: Adjusting urgency

[1594] The server integrates the matching results with the user's emotional information and adjusts the urgency of the anomaly. If the urgency is high based on the emotional information, it increases the priority of notification to the police and fire department. The input is the emotional information acquired in step 4 and the matching results obtained in step 5, and the output is the adjusted urgency information. This optimizes the urgency.

[1595] Step 7: Notification

[1596] The server notifies the police and fire department of the adjusted urgency information and details of the abnormality. It uses a notification system such as Firebase Cloud Messaging to transmit the information to the relevant agencies in real time. The input is the urgency information and details of the abnormality obtained in step 6, and the output is a notification message to the police and fire department. This enables a rapid response.

[1597] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1598] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1599] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1600] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1601] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1602] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1603] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1604] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1605] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1606] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1607] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1608] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1609] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1610] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1611] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1612] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1613] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1614] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1615] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1616] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1617] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1618] The following is further disclosed regarding the above embodiment.

[1619] (Claim 1)

[1620] means for collecting image data in real time from cameras installed on traffic lights, public buildings, and vehicles;

[1621] means for analyzing the image data and extracting characteristic information of the vehicle;

[1622] means for comparing the characteristic information with a pre-stored crime database and a pre-stored vehicle inspection database;

[1623] A means of notifying the police or fire department if an abnormality is detected;

[1624] A system including:

[1625] (Claim 2)

[1626] means for receiving image data captured by a user;

[1627] means for analyzing the received image data and extracting characteristic information of the vehicle;

[1628] means for comparing the characteristic information with a pre-stored crime database and a pre-stored vehicle inspection database;

[1629] A means of notifying the police or fire department if an abnormality is detected;

[1630] a means for managing the analysis results transmitted to the police or fire department by the notification means;

[1631] A system including:

[1632] (Claim 3)

[1633] 2. The system of claim 1, wherein the camera is connected to an artificial intelligence device, the artificial intelligence device having means for automatically analyzing the image data.

[1634] "Example 1"

[1635] (Claim 1)

[1636] a means of collecting real-time visibility data from optical devices installed on traffic lights, public buildings, and vehicles;

[1637] means for preprocessing the visual data to make it analyzable;

[1638] means for extracting vehicle feature information from the visual data;

[1639] means for comparing the characteristic information with a pre-stored database;

[1640] A means of notifying relevant authorities when an abnormality is detected;

[1641] A system including:

[1642] (Claim 2)

[1643] means for receiving visual data captured by a user;

[1644] means for preprocessing the received visual data to make it analyzable;

[1645] means for extracting vehicle feature information from the visual data;

[1646] means for comparing the characteristic information with a pre-stored database;

[1647] A means of notifying relevant authorities when an abnormality is detected;

[1648] a means for managing the analysis results sent to the relevant organization by the notification means;

[1649] A system including:

[1650] (Claim 3)

[1651] 2. The system of claim 1, wherein the optical instrument is connected to a device equipped with a machine learning algorithm, the device comprising means for automatically analyzing the visual data.

[1652] "Application Example 1"

[1653] (Claim 1)

[1654] A means for collecting image data in real time from imaging devices installed in traffic signals, public facilities, and vehicles;

[1655] means for analyzing the image data and extracting characteristic information of the vehicle;

[1656] means for comparing the characteristic information with a pre-stored criminal database and a pre-stored vehicle inspection database;

[1657] A means of notifying administrators and the police if an abnormality is detected;

[1658] A method for installing a specific application on a smart device to display images and notifications in real time when an abnormality is detected, and

[1659] A system including:

[1660] (Claim 2)

[1661] A means for receiving image data captured by a user;

[1662] means for analyzing the received image data and extracting characteristic information of the vehicle;

[1663] means for comparing the characteristic information with a pre-stored criminal database and a pre-stored vehicle inspection database;

[1664] A means of notifying administrators and the police if an abnormality is detected;

[1665] a means for managing the analysis results transmitted to the administrator and the police by the notification means;

[1666] 10. The system of claim 1, comprising:

[1667] (Claim 3)

[1668] 10. The system of claim 1, wherein the imaging device is connected to a device having artificial intelligence, the device having means for automatically analyzing the image data.

[1669] "Example 2: Combining Emotion Engines"

[1670] (Claim 1)

[1671] a means for collecting image data in real time from sensors installed on traffic lights, public buildings, and vehicles;

[1672] means for preprocessing the image data and extracting vehicle feature information;

[1673] means for comparing the characteristic information with a pre-stored database;

[1674] A means for collecting user emotion information and setting the urgency of the analysis results;

[1675] A means of notifying relevant authorities when an abnormality is detected;

[1676] A system including:

[1677] (Claim 2)

[1678] means for receiving image data captured by a user;

[1679] means for pre-processing the received image data and extracting vehicle characteristic information;

[1680] means for comparing the characteristic information with a pre-stored database;

[1681] A means for setting a priority of the analysis results based on the emotion information of the user;

[1682] A means of notifying relevant authorities when an abnormality is detected;

[1683] a means for managing the analysis results sent to the relevant organization by the notification means;

[1684] A system including:

[1685] (Claim 3)

[1686] 2. The system of claim 1, wherein the camera is connected to an artificial intelligence device, the artificial intelligence device having means for automatically analyzing the image data.

[1687] "Application example 2 when combining emotion engines"

[1688] (Claim 1)

[1689] means for collecting image data in real time from cameras installed on traffic lights, public buildings, and vehicles;

[1690] means for analyzing the image data and extracting characteristic information of the vehicle;

[1691] means for comparing the characteristic information with a pre-stored crime database and a pre-stored vehicle inspection database;

[1692] A means of notifying the police or fire department if an abnormality is detected;

[1693] A means for a user to capture an image of the surroundings using a mobile communication device and collect emotion information;

[1694] a means for adjusting the urgency of the analysis result based on emotional information of the user;

[1695] A system including:

[1696] (Claim 2)

[1697] means for receiving image data captured by a user;

[1698] means for analyzing the received image data and extracting characteristic information of the vehicle;

[1699] means for comparing the characteristic information with a pre-stored crime database and a pre-stored vehicle inspection database;

[1700] A means of notifying the police or fire department if an abnormality is detected;

[1701] a means for managing the analysis results transmitted to the police or fire department by the notification means;

[1702] means for analyzing emotion information collected by a user using a mobile communication device;

[1703] 10. The system of claim 1, comprising:

[1704] (Claim 3)

[1705] 2. The system of claim 1, wherein the camera is connected to an artificial intelligence device, the artificial intelligence device having means for automatically analyzing the image data. [Explanation of symbols]

[1706] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for collecting image data in real time from cameras installed on traffic lights, public buildings, and vehicles; means for analyzing the image data and extracting characteristic information of the vehicle; means for comparing the characteristic information with a pre-stored crime database and a pre-stored vehicle inspection database; A means of notifying the police or fire department if an abnormality is detected; A system including:

2. means for receiving image data captured by a user; means for analyzing the received image data and extracting characteristic information of the vehicle; means for comparing the characteristic information with a pre-stored crime database and a pre-stored vehicle inspection database; A means of notifying the police or fire department if an abnormality is detected; a means for managing the analysis results transmitted to the police or fire department by the notification means; A system including:

3. 2. The system of claim 1, wherein the camera is connected to an artificial intelligence device, the artificial intelligence device having means for automatically analyzing the image data.

Images