system

Abstract

We provide the system. [Solution] It is an information processing device for collecting real-time data. An analysis means for analyzing data collected by the aforementioned information processing device to detect signs of a crime occurring, A notification means that generates a warning based on the signs detected by the analysis means and presents countermeasures, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In current investigation activities, it is difficult to prevent the occurrence of crimes in advance, and there is a problem that attention has to be focused on the response after the occurrence. In particular, in order to detect early signs of crimes and take appropriate measures, it is required to extract meaningful information from a huge amount of real-time data and quickly present countermeasures. However, with conventional technologies, the accuracy of data collection and analysis is insufficient, and it has been difficult to contribute to rapid and effective crime prevention.

Means for Solving the Problems

[0005] This invention provides an information processing device that collects real-time data and analyzes that data with high accuracy. Specifically, it collects diverse data within a region using security monitoring devices, information and communication networks, and environmental sensors. Then, by using an analysis means that detects signs of crime occurrence based on the collected data, it realizes a system that assesses risk in real time and quickly generates and notifies specific warnings and countermeasures. This system can improve the accuracy of risk assessment by comparing past crime data with the current analysis results.

[0006] "Real-time data" refers to digital data that is constantly being collected, processed, and updated.

[0007] An "information processing device" is a system that includes hardware and software for collecting, analyzing, and outputting data.

[0008] "Analysis means" refers to algorithms or tools used to analyze collected data and detect specific patterns or anomalies.

[0009] "Notification means" refers to a device or software that has the function of transmitting warnings generated based on analysis results to the user or an external device.

[0010] A "security monitoring device" is a device that monitors the situation within a specific area using video or audio and detects suspicious activity.

[0011] An "information and communication network" is an infrastructure consisting of communication protocols and equipment for sending and receiving electronic data.

[0012] An "environmental sensor" is a device that detects physical changes in the surroundings, such as noise or light levels.

[0013] "Risk assessment" is the process of quantitatively or qualitatively evaluating potential risks based on data analysis results. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] The embodiments for carrying out the present invention will now be described. This system consists of multiple modules, each performing a specific function. At the core of the system are four modules: a data collection module, a data analysis module, a risk assessment module, and a notification module.

[0036] The data acquisition module acquires data in real time using security monitoring devices, information and communication networks, and environmental sensors. Examples include video data from security cameras, social media posts, and environmental sensor data such as noise and light levels.

[0037] The data analysis module applies natural language processing and computer vision technologies to the data provided by the collection module. This allows for the extraction of specific keywords from social media and the detection of abnormal behavior from camera footage. This enables the immediate detection of potential anomalies and signs of crime.

[0038] The risk assessment module compares past crime data with current conditions based on the analyzed data. This allows for the assessment of the risk of crime occurring in the target area, and the results can be quantified or ranked.

[0039] Finally, the notification module generates an alert based on the risk level calculated by the risk assessment module. This alert is sent to investigators' mobile devices or computers and provides information including specific countermeasures. For example, this may include instructions to increase patrols in specific areas or to change the direction of surveillance cameras.

[0040] For example, if an increase in keywords such as "gathering" or "riot" is detected on social media in a certain area, and at the same time, noise levels in that area rise, this system will assess the risk as high and immediately recommend increased patrols to the police. This makes it possible to take preventative measures before a crime occurs.

[0041] These modules work together to enable real-time data analysis and risk assessment, allowing for rapid and effective crime prevention. Implementing this system will improve public safety and reduce the burden on law enforcement agencies.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] Server: The data collection module starts up and begins ingesting data from security monitoring devices, social media APIs, and environmental sensors. Video data is divided into a series of image frames, and text data from social media is acquired as a stream.

[0045] Step 2:

[0046] Server: In the pre-processing stage, the collected data is organized. Irrelevant frames are removed from video data, and filtering is performed on text data to remove noise. Sensor data is formatted to a standardized format for easier analysis.

[0047] Step 3:

[0048] Server: The data analysis module extracts specific keywords from text data using natural language processing techniques. In addition, it utilizes computer vision algorithms to detect abnormal movements and behaviors from video data. Based on sensor data, it identifies patterns that are different from the norm.

[0049] Step 4:

[0050] Server: The risk assessment module compares the data with a database of past crimes and evaluates the risk level based on the analysis results. Depending on the frequency and similarity of the abnormal patterns, the risk is classified into ranks such as "low," "medium," and "high."

[0051] Step 5:

[0052] Server: A notification module is activated to generate alerts, creating notifications that include specific countermeasures based on the assessed risk level. This may include increased patrols or monitoring of priority areas.

[0053] Step 6:

[0054] Terminal: Generated warnings and countermeasures are sent in real time to the investigator's terminal and displayed on the screen. The investigator can understand the situation and take immediate action through the terminal.

[0055] Step 7:

[0056] User: Enters an actual activity report in response to a warning from their terminal and sends it to the server. This is saved as feedback data and used for future analysis.

[0057] These processing steps enable the system to efficiently and quickly detect signs of crime and propose effective preventative measures.

[0058] (Example 1)

[0059] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0060] Conventional security systems use only limited information to detect criminal activity, resulting in a lack of accuracy in warnings and the ability to respond quickly. Furthermore, it is difficult to effectively link past crime data with current situations for evaluation, leading to insufficient prediction and prevention.

[0061] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0062] In this invention, the server includes means for using a device to acquire information in real time, analysis means for applying natural language processing technology and image recognition technology using the information, evaluation means for determining and quantifying risk, and notification means for generating warnings and suggesting countermeasures. This enables rapid information collection and detailed analysis, and by providing effective risk assessment and appropriate countermeasures, it becomes possible to enhance safety.

[0063] "Real-time" is a term that refers to a state in which information is acquired and processed the moment it occurs.

[0064] "Devices for acquiring information" refers to hardware or software for automatically collecting data via sensor devices or communication networks.

[0065] "Natural language processing technology" is a term that refers to the technology used by computers to understand, interpret, and generate human language.

[0066] "Image recognition technology" refers to the technology that allows computers to automatically identify and detect objects and features in digital images and videos.

[0067] "Analysis means" refers to a mechanism for processing and analyzing information using collected data to detect specific patterns or anomalies.

[0068] "Evaluation methods" refer to techniques or devices used to calculate, quantify, or rank risks and situations based on analyzed data.

[0069] "Notification means" refers to a method or device for communicating warnings or instructions generated based on the results of analysis and evaluation to the user.

[0070] This invention is a system that enables the acquisition, analysis, and evaluation of information in real time. The system mainly consists of a server, terminals, and users.

[0071] The server acquires information in real time using sensor devices and communication networks. This system includes surveillance cameras, environmental sensors, and information and communication networks, and includes noise levels, image data, and text data collected from social media.

[0072] The terminal receives information transmitted by the server and analyzes the data using natural language processing and image recognition technologies. The natural language processing technology uses algorithms to extract specific keywords from text data, and the image recognition technology uses algorithms to detect anomalies from video data.

[0073] Users can take specific countermeasures based on the evaluation results notified by the server. The evaluation results are displayed on the terminal, from which users can take immediate action. For example, they can issue instructions such as increasing patrols or changing the placement of surveillance cameras.

[0074] For example, if noise levels increase in a certain area and the keyword "demonstration" frequently appears on social media, the system will assess the risk as high and notify users. By taking swift action based on this information, users can prevent potential problems.

[0075] An example of a prompt for a generative AI model is, "Detect increases in 'demonstrations' and 'noise' from recent text data in a specific area, perform a risk assessment, and notify the user." Based on this prompt, the system collects and analyzes information and provides the user with the necessary instructions.

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] The server acquires information in real time through sensor devices and communication networks. Inputs include surveillance camera footage, social media posts, and environmental sensor readings. This information is stored in a database and prepared for the next analysis step.

[0079] Step 2:

[0080] The terminal analyzes data acquired from the server. Inputs include stored video, text, and environmental data. The terminal applies natural language processing techniques to text data to extract specific keywords. Image recognition techniques are used on image data to detect abnormal behavior and patterns. The output includes a list of extracted keywords and characteristic information of detected anomalies.

[0081] Step 3:

[0082] The server performs a risk assessment based on the analysis results sent from the terminal. It uses keyword lists, anomaly characteristic information, and past crime history data as input. The server uses this information to run a risk assessment model and quantify the risk level for each region. The output is data regarding the risk level for each region.

[0083] Step 4:

[0084] The server activates a notification module based on the evaluation results. The input is risk level data. Based on this, it generates necessary warnings and countermeasures and notifies the user. The output of the notification is a warning message and specific countermeasures sent to the terminal.

[0085] Step 5:

[0086] Users receive notifications displayed on their devices and take specific actions. Inputs are the notified warning messages and corresponding actions. Users use this information to implement necessary defensive measures, such as increasing patrols or adjusting surveillance equipment. Outputs are the actual actions taken in the field, leading to the avoidance of potential risks.

[0087] (Application Example 1)

[0088] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0089] In modern society, detecting early signs of crime and taking swift and effective countermeasures is crucial for ensuring public safety. However, conventional systems lack sufficient real-time data collection and analysis capabilities, resulting in a lack of methods for providing rapid warnings.

[0090] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0091] In this invention, the server is an information processing device for collecting real-time data, and includes an analysis means for analyzing the collected data, detecting signs of crime occurrence and providing safety notifications; a notification means for generating warnings based on the detected signs and presenting specific countermeasures; and a communication means for quickly transmitting warnings to users using a mobile information terminal. This enables citizens to quickly detect signs of crime and take appropriate action.

[0092] An "information processing device" is a device that has the function of collecting data and analyzing it.

[0093] "Analysis means" refers to a function that detects specific events or signs based on collected data and derives related information.

[0094] A "notification method" is a system that communicates information such as warnings and countermeasures to the user based on the analysis results.

[0095] "Communication means" refers to technologies for quickly transmitting information to users via mobile information terminals.

[0096] A "monitoring device" is a device used to record changes in a specific region or environment and collect data.

[0097] "Information and communication means" refers to methods for transmitting data and information using the internet or wireless communication.

[0098] An "external environment sensor" is a sensor that detects the state of the external environment, such as noise and light, and generates data.

[0099] "Data processing means" refers to technologies for analyzing collected information and evaluating risk by comparing past patterns with the current situation.

[0100] In an embodiment of this invention, the server functions as an information processing device, collecting data from the environment in real time. Specifically, it acquires video and audio data using monitoring devices and external environmental sensors, and transmits this information to the server via information communication means. Based on the collected data, the server uses analysis means to detect signs of crime. This analysis may utilize computer vision libraries (e.g., OpenCV) or natural language processing tools (e.g., NLTK). The data to be analyzed may include, for example, keywords extracted from social media posts or abnormal behavior captured in camera footage.

[0101] Next, the server uses data processing tools based on the analyzed information to assess the risk by comparing it with past anomaly occurrence data. This data processing employs statistical methods and machine learning models (e.g., AI models using TENSORFLOW®). Based on the assessment results, if the risk level exceeds a certain threshold, the server generates a warning via notification tools and generates information suggesting specific countermeasures.

[0102] This information is transmitted to mobile devices via communication means and notified to users in real time. Users can use their mobile devices to read the warnings and suggested countermeasures they receive. For example, if keywords such as "gathering" and "noise" suddenly increase on social media in a certain area, and at the same time the noise sensor readings rise, the system will assess this as a moderate risk and notify the user with a message such as, "An event is taking place in your neighborhood. Please take precautions for your safety."

[0103] Examples of prompt messages include the following:

[0104] "Based on social media activity and environmental sensor data in a specific area, assess the safety risks for the next 48 hours. Related keywords include 'gathering,' 'party,' and 'riot,' and environmental sensors indicate increased noise levels."

[0105] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0106] Step 1:

[0107] The server receives real-time data from monitoring devices and external environmental sensors. Inputs include video files, audio data, and social media posts, which are processed via information and communication means. The server formats each data item, preparing it for subsequent analysis.

[0108] Step 2:

[0109] The server uses analysis tools to identify abnormal behavior and important keywords from the received data. It performs motion analysis on video data using computer vision and detects abnormal noises from audio data. It also extracts keywords such as "gathering" and "riot" from social media data using natural language processing techniques. As a result of analyzing the input data, it outputs feature data indicating potential anomalies.

[0110] Step 3:

[0111] The server further processes the analyzed feature data using data processing tools and evaluates the risk by comparing it with past anomaly occurrence data. Based on the feature data obtained as input, it generates an AI model and determines the risk level by quantifying or ranking the results. As a result of the risk evaluation, it outputs one of the following risk levels: low, medium, or high.

[0112] Step 4:

[0113] The server uses notification methods to generate warnings and countermeasures based on the assessed risk level. Based on the input risk information, it formulates specific warning messages and proposed countermeasures (e.g., increased patrols, warnings) and generates them as output.

[0114] Step 5:

[0115] The server sends the generated warnings and countermeasures to the terminal via a communication method. It sends notifications to the user's mobile device, allowing the user to receive warnings in real time and check the countermeasures. The output notifications are displayed on the terminal as alerts prompting the user to take safety measures.

[0116] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0117] This invention combines a crime prevention system that collects and analyzes real-time data with an emotion engine that recognizes user emotions. This system includes a data collection module, a data analysis module, a risk assessment module, a notification module, and an emotion engine module.

[0118] The data collection module acquires a wide range of information in real time using security monitoring devices, information and communication networks, and environmental sensors. This includes video and audio data, social media posts, and data about the surrounding environment.

[0119] The data analysis module extracts specific keywords and features from the collected data. At this stage, the emotion engine performs the function of analyzing the user's emotions from the text and audio data.

[0120] The emotion engine module uses natural language processing and speech analysis technologies to identify emotional states from user utterances and posts. For example, if emotions such as anger or fear are strongly expressed, that data can be given greater weight.

[0121] The risk assessment module uses information obtained from data analysis to compare it with past crime data and evaluate the likelihood of crime occurring. Sentimental data from the sentiment engine also contributes to risk assessment, and posts and locations from users showing particularly high emotional responses are given priority for warnings.

[0122] The notification module generates alerts based on the results of risk assessments and sends them in real time to investigators' terminals. Based on the emotion engine's assessment, it includes instructions to prompt immediate action in high-priority situations. For example, it may issue patrol orders or requests for on-site investigations in specific areas.

[0123] For example, if there is an area where keywords such as "riot" or "danger" are increasing on social media, and the emotion engine detects strong feelings of "anger" or "fear" from posts in that area, the region will be assessed as high risk. Based on this information, a warning for early response is sent to investigators, and on-site investigations and increased security are recommended.

[0124] This system utilizes an emotion engine to take into account even subtle risks that could not be captured by conventional information analysis alone, thereby supporting more effective crime prevention activities.

[0125] The following describes the processing flow.

[0126] Step 1:

[0127] Server: The data collection module collects video, audio, and text data in real time through security monitoring devices, information and communication networks, and environmental sensors. At this stage, information from each data source is aggregated in one place.

[0128] Step 2:

[0129] Server: Preprocesses the collected data. Specifically, it filters video frames, removes noisy audio, and removes spam from text data. This creates a clean dataset suitable for analysis.

[0130] Step 3:

[0131] Server: The data analysis module starts up and uses natural language processing techniques to extract violent or dangerous keywords from text data. Simultaneously, computer vision is used to analyze suspicious behavior from video data.

[0132] Step 4:

[0133] Server: The emotion engine module analyzes text and audio data obtained from users to identify emotions. Particular attention should be paid to data that emphasizes emotions such as anger or fear. This information helps to consider the emotional aspects in crime prevention measures.

[0134] Step 5:

[0135] Server: The risk assessment module integrates analyzed keywords, behavioral patterns, and sentiment data and evaluates their relevance to past crime data. For example, based on high emotional responses and an increase in crime-related keywords in a specific area, the risk is set to "high."

[0136] Step 6:

[0137] Server: The notification module generates alerts based on the risk assessment results and prepares appropriate countermeasures. This comprehensive information is sent in a format optimized for investigators' terminals.

[0138] Step 7:

[0139] Terminal: Based on the received warnings, investigators carry out on-site operations. Immediate responses are required depending on the situation, and orders are given to increase patrols in specific areas or concentrate video surveillance.

[0140] Step 8:

[0141] User: Investigators input their field findings into the system as feedback, which helps improve the accuracy of future analyses. This allows the system to continuously learn and enable more accurate and effective preventative measures.

[0142] (Example 2)

[0143] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0144] In modern society, accurate and rapid risk assessment is essential to prevent crime. However, conventional systems lack sufficient real-time capabilities and consideration of emotions, potentially overlooking potential threats. In particular, the inability to adequately reflect nuanced data, including user emotions, in risk assessments is a significant challenge.

[0145] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0146] In this invention, the server includes means for collecting a wide range of data using sensors and networks, emotion recognition means for extracting keywords and features from the collected data and analyzing emotions, and evaluation means for comparing the emotion data with past data patterns and evaluating the risk. This enables highly accurate assessment of the likelihood of crime occurring and allows for rapid warnings and the presentation of countermeasures.

[0147] A "sensor" is a device used to acquire data from the physical environment, collecting information such as light, sound, and temperature in real time.

[0148] A "network" is an information and communication infrastructure used for sending and receiving data, and refers to communication methods including the internet and local networks.

[0149] "Emotion recognition means" refers to technologies for identifying emotions from a user's text or voice data, using natural language processing and speech analysis technologies to analyze emotions such as "anger" and "fear."

[0150] "Evaluation methods" refer to processes and techniques for determining crime risk based on collected and analyzed data, including analysis that involves cross-referencing with past crime data.

[0151] "Notification means" refers to a mechanism that provides users with warnings and instructions generated based on evaluation results, and includes a function that transmits information in real time through the terminal.

[0152] A "generative AI model" is a machine learning model used to identify emotional states from user statements and posts, and it performs highly accurate analysis from diverse data.

[0153] This invention is a system for assessing crime risk in real time and responding quickly. This system includes the following components:

[0154] First, the server collects data through sensors and the network. Video data is acquired via cameras, and audio data via microphones. APIs are also used to collect posts from social media platforms. This collected data is managed by a data collection module running on the server.

[0155] Next, the collected data is passed to a data analysis module on the server. This module uses natural language processing and speech analysis techniques to extract specific keywords and features from the text and audio. In this process, the collected data is converted into text data, and keyword extraction and feature analysis are performed.

[0156] The emotion recognition module, an emotion engine, utilizes a generative AI model installed on the server to identify the user's emotions from extracted text and audio. For example, if emotions such as "anger" or "fear" are strongly expressed, these are given importance and used in risk assessment. This allows for more accurate crime prediction.

[0157] The server then uses an evaluation system to compare the collected data with past crime data patterns and assesses the risk of crime based on the collected data and sentiment data. Based on the evaluation results, the notification system sends alerts to terminals in real time. This evaluation reflects the output of the sentiment engine and includes instructions that immediately suggest countermeasures for high-risk areas and situations.

[0158] For example, if there is a surge in social media posts in a particular area that frequently contain keywords such as "riot" or "danger," the emotion engine will strongly detect feelings of "anger" and "fear" from the posts. The area will be assessed as high-risk, and investigators will be advised to conduct on-site investigations and increase security.

[0159] Examples of prompts for the generating AI model include, "Analyze the sentiment of social media posts in a specific region and identify areas where feelings of anger or fear are strongly expressed." In this way, servers, terminals, and users cooperate to realize an advanced system for strengthening crime prevention.

[0160] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0161] Step 1:

[0162] The server collects data in real time using sensors and networks. This input data includes video data from surveillance cameras, audio data from microphones, and social media posts obtained through SNS APIs. The server manages this data in collection modules and prepares it for the next analysis step.

[0163] Step 2:

[0164] The server processes input data using a data analysis module. Video data is converted into text data using image recognition technology, and audio data is converted into text data using speech recognition technology. Next, specific keywords such as "danger" and "urgent" are extracted from the text data using natural language processing technology. This output data consists of the analyzed keyword information and related feature data.

[0165] Step 3:

[0166] The server identifies the emotions of the text data analyzed using an emotion engine module. Using a generative AI model, it analyzes the emotions within the input text and strongly identifies emotions such as "anger" and "fear." This output is an emotion score for each text, which is then used in the subsequent risk assessment.

[0167] Step 4:

[0168] The server uses a risk assessment module to match sentiment scores and extracted keywords with historical crime data. Through the assessment mechanism, it calculates the likelihood of a crime occurring based on the input data and ranks the risk level. The output is a dataset with the risk levels assessed. This risk data is used to rank the level of danger and proceed to the next notification step.

[0169] Step 5:

[0170] The terminal generates warnings based on risk assessment results sent from the server. The notification module takes risk data as input and generates necessary warning messages and instructions in real time. This output includes specific action instructions for patrolling specific areas or conducting on-site checks.

[0171] Step 6:

[0172] The user receives notifications from their device and takes prompt action. Based on the warning, the user initiates on-site patrols and verification work and takes appropriate measures according to the situation. This output represents the user's on-site action plan and its execution.

[0173] (Application Example 2)

[0174] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0175] Conventional crime prevention systems were limited to detecting early signs of crime through real-time data analysis, and were unable to adequately consider the emotional state of users, thus limiting their ability to detect and prevent crime early. Furthermore, a challenge in assessing crime risk was the inability to adequately evaluate potential dangers stemming from emotions.

[0176] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0177] In this invention, the server includes information processing means for collecting real-time data, analysis means for analyzing the collected data and detecting signs of crime, and emotion analysis means for analyzing the user's emotions. This makes it possible to perform risk assessment that takes emotion data into account in addition to signs of crime.

[0178] "Real-time data" refers to data that is collected and analyzed instantly without any time delay.

[0179] An "information processing device" is an electronic device used to collect, process, and analyze data.

[0180] "Analysis means" refers to a method or apparatus for processing collected data to detect specific information or anomalies.

[0181] "Emotional analysis means" refers to a technology or device that analyzes and identifies a user's emotional state from data.

[0182] "Notification means" refers to a method or device for generating warnings or information based on analysis results and informing the user or relevant parties.

[0183] A "surveillance system" is a device or program used to continuously observe a specific area or object.

[0184] A "communication network" is an infrastructure for exchanging data between remote locations.

[0185] A "sensor" is a device used to detect and measure specific environmental conditions or variables.

[0186] "Risk assessment" is the process of evaluating the likelihood and risk of a specific event occurring based on collected data.

[0187] The system for realizing this invention is based on a complex sensor network and an advanced data analysis system. The server collects real-time data from various sensors and monitoring devices and performs information processing to analyze that data. This includes an emotion analysis function that identifies the user's emotional state. Emotion analysis utilizes natural language processing and speech analysis technologies to analyze user utterances and posts using a deep learning model. This makes it possible to respond quickly even when the user's emotions are heightened.

[0188] The server uses machine learning frameworks such as TensorFlow to extract specific features from audio and video, and analyzes emotions using Hugging Face's natural language processing model. This analysis result is integrated with past crime data to perform risk assessment and make real-time situational judgments.

[0189] For example, in a security system for a shopping mall, surveillance cameras and audio recording devices monitor activity within the facility and instantly transmit data to a server. The server analyzes this data and, if it determines that "anger" is increasing in a particular location, displays a warning on the security guard's terminal. At this time, the terminal receives both visual and auditory alerts to prompt a quick response.

[0190] An example of a prompt message might be: "Design a system that would inform security personnel how to respond if they detect heightened anger in a specific location." This would enable a quick and appropriate response to potential risks.

[0191] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0192] Step 1:

[0193] The server collects real-time data from various sensors and monitoring devices. It receives sensor information, such as camera and audio data, as input and stores it as digital data. This process includes transmitting the data to the server using network communication.

[0194] Step 2:

[0195] The server performs audio and video analysis using TensorFlow to analyze the collected data. It receives the data saved in step 1 as input and applies a feature extraction algorithm to identify important features. As output, it generates a feature vector representing the user's emotional state. Specific operations include extracting acoustic characteristics from audio data and analyzing facial expressions from video data.

[0196] Step 3:

[0197] The server uses Hugging Face's natural language processing model to perform sentiment analysis. It receives the feature vectors from Step 2 as input and feeds them into the model. As output, it identifies sentiment categories (e.g., anger, joy, sadness, etc.). This allows the server to assess the user's emotional state. Specific operations include lexical analysis of text data and calculation of sentiment intensity.

[0198] Step 4:

[0199] The server integrates historical crime data with the aforementioned sentiment data to perform a risk assessment. It uses the results of sentiment analysis and historical databases as input, performs data matching, and calculates a risk score. The output generates the risk assessment results. This process includes searching for similarities in crime patterns and quantifying real-time risk.

[0200] Step 5:

[0201] The server sends an alert to the terminal based on the assessed risk. It uses the risk assessment results from step 4 as input to generate the necessary alert message. As output, it generates alert information combining text messages, alert sounds, and visual notifications. This provides immediate response instructions to security personnel on the terminal. Specific actions include data transmission via communication protocols and triggering the terminal's alarm system.

[0202] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0203] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0204] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0205] [Second Embodiment]

[0206] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0207] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0208] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0209] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0210] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0211] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0212] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0213] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0214] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0215] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0216] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0217] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0218] The embodiments for carrying out the present invention will now be described. This system consists of multiple modules, each performing a specific function. At the core of the system are four modules: a data collection module, a data analysis module, a risk assessment module, and a notification module.

[0219] The data acquisition module acquires data in real time using security monitoring devices, information and communication networks, and environmental sensors. Examples include video data from security cameras, social media posts, and environmental sensor data such as noise and light levels.

[0220] The data analysis module applies natural language processing and computer vision technologies to the data provided by the collection module. This allows for the extraction of specific keywords from social media and the detection of abnormal behavior from camera footage. This enables the immediate detection of potential anomalies and signs of crime.

[0221] The risk assessment module compares past crime data with current conditions based on the analyzed data. This allows for the assessment of the risk of crime occurring in the target area, and the results can be quantified or ranked.

[0222] Finally, the notification module generates an alert based on the risk level calculated by the risk assessment module. This alert is sent to investigators' mobile devices or computers and provides information including specific countermeasures. For example, this may include instructions to increase patrols in specific areas or to change the direction of surveillance cameras.

[0223] For example, if an increase in keywords such as "gathering" or "riot" is detected on social media in a certain area, and at the same time, noise levels in that area rise, this system will assess the risk as high and immediately recommend increased patrols to the police. This makes it possible to take preventative measures before a crime occurs.

[0224] These modules work together to enable real-time data analysis and risk assessment, allowing for rapid and effective crime prevention. Implementing this system will improve public safety and reduce the burden on law enforcement agencies.

[0225] The following describes the processing flow.

[0226] Step 1:

[0227] Server: The data collection module starts up and begins ingesting data from security monitoring devices, social media APIs, and environmental sensors. Video data is divided into a series of image frames, and text data from social media is acquired as a stream.

[0228] Step 2:

[0229] Server: In the pre-processing stage, the collected data is organized. Irrelevant frames are removed from video data, and filtering is performed on text data to remove noise. Sensor data is formatted to a standardized format for easier analysis.

[0230] Step 3:

[0231] Server: The data analysis module extracts specific keywords from text data using natural language processing techniques. In addition, it utilizes computer vision algorithms to detect abnormal movements and behaviors from video data. Based on sensor data, it identifies patterns that are different from the norm.

[0232] Step 4:

[0233] Server: The risk assessment module compares the data with a database of past crimes and evaluates the risk level based on the analysis results. Depending on the frequency and similarity of the abnormal patterns, the risk is classified into ranks such as "low," "medium," and "high."

[0234] Step 5:

[0235] Server: A notification module is activated to generate alerts, creating notifications that include specific countermeasures based on the assessed risk level. This may include increased patrols or monitoring of priority areas.

[0236] Step 6:

[0237] Terminal: Generated warnings and countermeasures are sent in real time to the investigator's terminal and displayed on the screen. The investigator can understand the situation and take immediate action through the terminal.

[0238] Step 7:

[0239] User: Enters an actual activity report in response to a warning from their terminal and sends it to the server. This is saved as feedback data and used for future analysis.

[0240] These processing steps enable the system to efficiently and quickly detect signs of crime and propose effective preventative measures.

[0241] (Example 1)

[0242] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0243] Conventional security systems use only limited information to detect criminal activity, resulting in a lack of accuracy in warnings and the ability to respond quickly. Furthermore, it is difficult to effectively link past crime data with current situations for evaluation, leading to insufficient prediction and prevention.

[0244] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0245] In this invention, the server includes means for using a device to acquire information in real time, analysis means for applying natural language processing technology and image recognition technology using the information, evaluation means for determining and quantifying risk, and notification means for generating warnings and suggesting countermeasures. This enables rapid information collection and detailed analysis, and by providing effective risk assessment and appropriate countermeasures, it becomes possible to enhance safety.

[0246] "Real-time" is a term that refers to a state in which information is acquired and processed the moment it occurs.

[0247] "Devices for acquiring information" refers to hardware or software for automatically collecting data via sensor devices or communication networks.

[0248] "Natural language processing technology" is a term that refers to the technology used by computers to understand, interpret, and generate human language.

[0249] "Image recognition technology" refers to the technology that allows computers to automatically identify and detect objects and features in digital images and videos.

[0250] "Analysis means" refers to a mechanism for processing and analyzing information using collected data to detect specific patterns or anomalies.

[0251] "Evaluation methods" refer to techniques or devices used to calculate, quantify, or rank risks and situations based on analyzed data.

[0252] "Notification means" refers to a method or device for communicating warnings or instructions generated based on the results of analysis and evaluation to the user.

[0253] This invention is a system that enables the acquisition, analysis, and evaluation of information in real time. The system mainly consists of a server, terminals, and users.

[0254] The server acquires information in real time using sensor devices and communication networks. This system includes surveillance cameras, environmental sensors, and information and communication networks, and includes noise levels, image data, and text data collected from social media.

[0255] The terminal receives information transmitted by the server and analyzes the data using natural language processing and image recognition technologies. The natural language processing technology uses algorithms to extract specific keywords from text data, and the image recognition technology uses algorithms to detect anomalies from video data.

[0256] Users can take specific countermeasures based on the evaluation results notified by the server. The evaluation results are displayed on the terminal, from which users can take immediate action. For example, they can issue instructions such as increasing patrols or changing the placement of surveillance cameras.

[0257] For example, if noise levels increase in a certain area and the keyword "demonstration" frequently appears on social media, the system will assess the risk as high and notify users. By taking swift action based on this information, users can prevent potential problems.

[0258] An example of a prompt for a generative AI model is, "Detect increases in 'demonstrations' and 'noise' from recent text data in a specific area, perform a risk assessment, and notify the user." Based on this prompt, the system collects and analyzes information and provides the user with the necessary instructions.

[0259] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0260] Step 1:

[0261] The server acquires information in real time through sensor devices and communication networks. Inputs include surveillance camera footage, social media posts, and environmental sensor readings. This information is stored in a database and prepared for the next analysis step.

[0262] Step 2:

[0263] The terminal analyzes data acquired from the server. Inputs include stored video, text, and environmental data. The terminal applies natural language processing techniques to text data to extract specific keywords. Image recognition techniques are used on image data to detect abnormal behavior and patterns. The output includes a list of extracted keywords and characteristic information of detected anomalies.

[0264] Step 3:

[0265] The server performs a risk assessment based on the analysis results sent from the terminal. It uses keyword lists, anomaly characteristic information, and past crime history data as input. The server uses this information to run a risk assessment model and quantify the risk level for each region. The output is data regarding the risk level for each region.

[0266] Step 4:

[0267] The server activates a notification module based on the evaluation results. The input is risk level data. Based on this, it generates necessary warnings and countermeasures and notifies the user. The output of the notification is a warning message and specific countermeasures sent to the terminal.

[0268] Step 5:

[0269] Users receive notifications displayed on their devices and take specific actions. Inputs are the notified warning messages and corresponding actions. Users use this information to implement necessary defensive measures, such as increasing patrols or adjusting surveillance equipment. Outputs are the actual actions taken in the field, leading to the avoidance of potential risks.

[0270] (Application Example 1)

[0271] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0272] In modern society, detecting early signs of crime and taking swift and effective countermeasures is crucial for ensuring public safety. However, conventional systems lack sufficient real-time data collection and analysis capabilities, resulting in a lack of methods for providing rapid warnings.

[0273] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0274] In this invention, the server is an information processing device for collecting real-time data, and includes an analysis means for analyzing the collected data, detecting signs of crime occurrence and providing safety notifications; a notification means for generating warnings based on the detected signs and presenting specific countermeasures; and a communication means for quickly transmitting warnings to users using a mobile information terminal. This enables citizens to quickly detect signs of crime and take appropriate action.

[0275] An "information processing device" is a device that has the function of collecting data and analyzing it.

[0276] "Analysis means" refers to a function that detects specific events or signs based on collected data and derives related information.

[0277] A "notification method" is a system that communicates information such as warnings and countermeasures to the user based on the analysis results.

[0278] "Communication means" refers to technologies for quickly transmitting information to users via mobile information terminals.

[0279] A "monitoring device" is a device used to record changes in a specific region or environment and collect data.

[0280] "Information and communication means" refers to methods for transmitting data and information using the internet or wireless communication.

[0281] An "external environment sensor" is a sensor that detects the state of the external environment such as noise and light and generates data.

[0282] "Data processing means" is a technology for evaluating risks by analyzing the collected information and comparing the past patterns with the current situation.

[0283] As a form of implementing this invention, the server functions as an information processing device and collects data from the environment in real time. Specifically, using monitoring devices and external environment sensors, video data and acoustic data are acquired and these pieces of information are transmitted to the server through information communication means. The server detects signs of crime occurrence using analysis means based on the collected data. For this analysis, libraries of computer vision (e.g., OpenCV) and natural language processing tools (e.g., NLTK) may be used. The data to be analyzed includes, for example, keywords extracted from SNS posts and abnormal behaviors shown in camera videos.

[0284] Next, the server utilizes data processing means based on the analyzed information and evaluates the risk by comparing it with past abnormal occurrence data. For this data processing, statistical methods and machine learning models (e.g., AI models using TensorFlow) are used. Based on the evaluation result, when the risk level exceeds a certain threshold, the server generates a warning through notification means and generates information presenting specific countermeasures.

[0285] This information is transmitted to a mobile information terminal through communication means and is notified to the user in real time. The user can read the received warning and the proposed countermeasures using the mobile information terminal. As a specific example, when keywords such as "assembly" and "noise" suddenly increase on SNS in a certain area and at the same time the value of the noise sensor is rising, the system evaluates it as a medium-level risk and notifies the user "An event is occurring in the neighborhood. Please take care of your safety."

[0286] Examples of prompt sentences include the following.

[0287] "Please evaluate the safety risk within the next 48 hours based on SNS activities and environmental sensor data in a certain area. The related keywords are 'assembly', 'party', 'riot', and the environmental sensor indicates an increase in noise level."

[0288] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0289] Step 1:

[0290] The server receives real-time data from the monitoring device and external environmental sensors. As inputs, video files, acoustic data, SNS posting data, etc. are provided, and this is processed via information communication means. The server formats each data and shapes it into a form suitable for subsequent analysis processing.

[0291] Step 2:

[0292] The server uses analysis means to identify abnormal behaviors and important keywords from the received data. From the video data, motion analysis is performed using computer vision, and abnormal noise is detected from the acoustic data. Also, keywords such as 'assembly' and 'riot' are extracted from the SNS data using natural language processing technology. As a result of analyzing the input data, characteristic data indicating the possibility of abnormality is output.

[0293] Step 3:

[0294] The server further processes the analyzed characteristic data using data processing means and evaluates the risk by comparing it with past abnormal occurrence data. Based on the characteristic data obtained as an input, an AI model is generated, and the result is quantified or ranked to determine the risk level. As a result of the risk assessment, a risk level of either low, medium, or high is output.

[0295] Step 4:

[0296] The server uses notification methods to generate warnings and countermeasures based on the assessed risk level. Based on the input risk information, it formulates specific warning messages and proposed countermeasures (e.g., increased patrols, warnings) and generates them as output.

[0297] Step 5:

[0298] The server sends the generated warnings and countermeasures to the terminal via a communication method. It sends notifications to the user's mobile device, allowing the user to receive warnings in real time and check the countermeasures. The output notifications are displayed on the terminal as alerts prompting the user to take safety measures.

[0299] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0300] This invention combines a crime prevention system that collects and analyzes real-time data with an emotion engine that recognizes user emotions. This system includes a data collection module, a data analysis module, a risk assessment module, a notification module, and an emotion engine module.

[0301] The data collection module acquires a wide range of information in real time using security monitoring devices, information and communication networks, and environmental sensors. This includes video and audio data, social media posts, and data about the surrounding environment.

[0302] The data analysis module extracts specific keywords and features from the collected data. At this stage, the emotion engine performs the function of analyzing the user's emotions from the text and audio data.

[0303] In the emotion engine module, natural language processing technology and voice analysis technology are used to identify the emotional state from the user's speech and posts. For example, when emotions such as anger and fear strongly appear, the data can be emphasized.

[0304] Based on the information obtained from data analysis, the risk assessment module compares with past crime data and evaluates the possibility of crime occurrence. The emotion data from the emotion engine also contributes to the risk judgment, and particularly the posts and regions of users showing a high emotional reaction are preferentially targeted for warning.

[0305] The notification module generates warnings based on the results of risk assessment at any time and transmits them to the terminals of investigators in real time. According to the evaluation of the emotion engine, instructions to prompt prompt actions in highly urgent situations are included. For example, patrol instructions and requests for on-site confirmation in a specific area are issued.

[0306] As a specific example, if there is an area on the SNS where keywords such as "riot" and "danger" are increasing, and the emotion engine detects strong emotions of "anger" and "fear" from the posts in this area, this area is evaluated as a high risk. Based on this information, a warning for early response is sent to the investigators, and on-site investigation and security reinforcement are recommended.

[0307] This system takes into account the subtle risks that could not be fully captured by conventional information analysis by using an emotion engine, and supports more effective crime prevention activities.

[0308] The following describes the processing flow.

[0309] Step 1:

[0310] Server: The data collection module collects video, audio, and text data in real time through security monitoring devices, information communication networks, and environmental sensors. At this stage, the information from each data source is aggregated in one place.

[0311] Step 2:

[0312] Server: Preprocesses the collected data. Specifically, it filters video frames, removes noisy audio, and removes spam from text data. This creates a clean dataset suitable for analysis.

[0313] Step 3:

[0314] Server: The data analysis module starts up and uses natural language processing techniques to extract violent or dangerous keywords from text data. Simultaneously, computer vision is used to analyze suspicious behavior from video data.

[0315] Step 4:

[0316] Server: The emotion engine module analyzes text and audio data obtained from users to identify emotions. Particular attention should be paid to data that emphasizes emotions such as anger or fear. This information helps to consider the emotional aspects in crime prevention measures.

[0317] Step 5:

[0318] Server: The risk assessment module integrates analyzed keywords, behavioral patterns, and sentiment data and evaluates their relevance to past crime data. For example, based on high emotional responses and an increase in crime-related keywords in a specific area, the risk is set to "high."

[0319] Step 6:

[0320] Server: The notification module generates alerts based on the risk assessment results and prepares appropriate countermeasures. This comprehensive information is sent in a format optimized for investigators' terminals.

[0321] Step 7:

[0322] Terminal: Based on the received warnings, investigators carry out on-site operations. Immediate responses are required depending on the situation, and orders are given to increase patrols in specific areas or concentrate video surveillance.

[0323] Step 8:

[0324] User: Investigators input their field findings into the system as feedback, which helps improve the accuracy of future analyses. This allows the system to continuously learn and enable more accurate and effective preventative measures.

[0325] (Example 2)

[0326] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0327] In modern society, accurate and rapid risk assessment is essential to prevent crime. However, conventional systems lack sufficient real-time capabilities and consideration of emotions, potentially overlooking potential threats. In particular, the inability to adequately reflect nuanced data, including user emotions, in risk assessments is a significant challenge.

[0328] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0329] In this invention, the server includes means for collecting a wide range of data using sensors and networks, emotion recognition means for extracting keywords and features from the collected data and analyzing emotions, and evaluation means for comparing the emotion data with past data patterns and evaluating the risk. This enables highly accurate assessment of the likelihood of crime occurring and allows for rapid warnings and the presentation of countermeasures.

[0330] A "sensor" is a device used to acquire data from the physical environment, collecting information such as light, sound, and temperature in real time.

[0331] A "network" is an information and communication infrastructure used for sending and receiving data, and refers to communication methods including the internet and local networks.

[0332] "Emotion recognition means" refers to technologies for identifying emotions from a user's text or voice data, using natural language processing and speech analysis technologies to analyze emotions such as "anger" and "fear."

[0333] "Evaluation methods" refer to processes and techniques for determining crime risk based on collected and analyzed data, including analysis that involves cross-referencing with past crime data.

[0334] "Notification means" refers to a mechanism that provides users with warnings and instructions generated based on evaluation results, and includes a function that transmits information in real time through the terminal.

[0335] A "generative AI model" is a machine learning model used to identify emotional states from user statements and posts, and it performs highly accurate analysis from diverse data.

[0336] This invention is a system for assessing crime risk in real time and responding quickly. This system includes the following components:

[0337] First, the server collects data through sensors and the network. Video data is acquired via cameras, and audio data via microphones. APIs are also used to collect posts from social media platforms. This collected data is managed by a data collection module running on the server.

[0338] Next, the collected data is passed to a data analysis module on the server. This module uses natural language processing and speech analysis techniques to extract specific keywords and features from the text and audio. In this process, the collected data is converted into text data, and keyword extraction and feature analysis are performed.

[0339] The emotion recognition module, an emotion engine, utilizes a generative AI model installed on the server to identify the user's emotions from extracted text and audio. For example, if emotions such as "anger" or "fear" are strongly expressed, these are given importance and used in risk assessment. This allows for more accurate crime prediction.

[0340] The server then uses an evaluation system to compare the collected data with past crime data patterns and assesses the risk of crime based on the collected data and sentiment data. Based on the evaluation results, the notification system sends alerts to terminals in real time. This evaluation reflects the output of the sentiment engine and includes instructions that immediately suggest countermeasures for high-risk areas and situations.

[0341] For example, if there is a surge in social media posts in a particular area that frequently contain keywords such as "riot" or "danger," the emotion engine will strongly detect feelings of "anger" and "fear" from the posts. The area will be assessed as high-risk, and investigators will be advised to conduct on-site investigations and increase security.

[0342] Examples of prompts for the generating AI model include, "Analyze the sentiment of social media posts in a specific region and identify areas where feelings of anger or fear are strongly expressed." In this way, servers, terminals, and users cooperate to realize an advanced system for strengthening crime prevention.

[0343] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0344] Step 1:

[0345] The server collects data in real time using sensors and networks. This input data includes video data from surveillance cameras, audio data from microphones, and social media posts obtained through SNS APIs. The server manages this data in collection modules and prepares it for the next analysis step.

[0346] Step 2:

[0347] The server processes input data using a data analysis module. Video data is converted into text data using image recognition technology, and audio data is converted into text data using speech recognition technology. Next, specific keywords such as "danger" and "urgent" are extracted from the text data using natural language processing technology. This output data consists of the analyzed keyword information and related feature data.

[0348] Step 3:

[0349] The server identifies the emotions of the text data analyzed using an emotion engine module. Using a generative AI model, it analyzes the emotions within the input text and strongly identifies emotions such as "anger" and "fear." This output is an emotion score for each text, which is then used in the subsequent risk assessment.

[0350] Step 4:

[0351] The server uses a risk assessment module to match sentiment scores and extracted keywords with historical crime data. Through the assessment mechanism, it calculates the likelihood of a crime occurring based on the input data and ranks the risk level. The output is a dataset with the risk levels assessed. This risk data is used to rank the level of danger and proceed to the next notification step.

[0352] Step 5:

[0353] The terminal generates warnings based on risk assessment results sent from the server. The notification module takes risk data as input and generates necessary warning messages and instructions in real time. This output includes specific action instructions for patrolling specific areas or conducting on-site checks.

[0354] Step 6:

[0355] The user receives notifications from their device and takes prompt action. Based on the warning, the user initiates on-site patrols and verification work and takes appropriate measures according to the situation. This output represents the user's on-site action plan and its execution.

[0356] (Application Example 2)

[0357] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0358] Conventional crime prevention systems were limited to detecting early signs of crime through real-time data analysis, and were unable to adequately consider the emotional state of users, thus limiting their ability to detect and prevent crime early. Furthermore, a challenge in assessing crime risk was the inability to adequately evaluate potential dangers stemming from emotions.

[0359] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0360] In this invention, the server includes information processing means for collecting real-time data, analysis means for analyzing the collected data and detecting signs of crime, and emotion analysis means for analyzing the user's emotions. This makes it possible to perform risk assessment that takes emotion data into account in addition to signs of crime.

[0361] "Real-time data" refers to data that is collected and analyzed instantly without any time delay.

[0362] An "information processing device" is an electronic device used to collect, process, and analyze data.

[0363] "Analysis means" refers to a method or apparatus for processing collected data to detect specific information or anomalies.

[0364] "Emotional analysis means" refers to a technology or device that analyzes and identifies a user's emotional state from data.

[0365] "Notification means" refers to a method or device for generating warnings or information based on analysis results and informing the user or relevant parties.

[0366] A "surveillance system" is a device or program used to continuously observe a specific area or object.

[0367] A "communication network" is an infrastructure for exchanging data between remote locations.

[0368] A "sensor" is a device used to detect and measure specific environmental conditions or variables.

[0369] "Risk assessment" is the process of evaluating the likelihood and risk of a specific event occurring based on collected data.

[0370] The system for realizing this invention is based on a complex sensor network and an advanced data analysis system. The server collects real-time data from various sensors and monitoring devices and performs information processing to analyze that data. This includes an emotion analysis function that identifies the user's emotional state. Emotion analysis utilizes natural language processing and speech analysis technologies to analyze user utterances and posts using a deep learning model. This makes it possible to respond quickly even when the user's emotions are heightened.

[0371] The server uses machine learning frameworks such as TensorFlow to extract specific features from audio and video, and analyzes emotions using Hugging Face's natural language processing model. This analysis result is integrated with past crime data to perform risk assessment and make real-time situational judgments.

[0372] For example, in a security system for a shopping mall, surveillance cameras and audio recording devices monitor activity within the facility and instantly transmit data to a server. The server analyzes this data and, if it determines that "anger" is increasing in a particular location, displays a warning on the security guard's terminal. At this time, the terminal receives both visual and auditory alerts to prompt a quick response.

[0373] An example of a prompt message might be: "Design a system that would inform security personnel how to respond if they detect heightened anger in a specific location." This would enable a quick and appropriate response to potential risks.

[0374] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0375] Step 1:

[0376] The server collects real-time data from various sensors and monitoring devices. It receives sensor information, such as camera and audio data, as input and stores it as digital data. This process includes transmitting the data to the server using network communication.

[0377] Step 2:

[0378] The server performs audio and video analysis using TensorFlow to analyze the collected data. It receives the data saved in step 1 as input and applies a feature extraction algorithm to identify important features. As output, it generates a feature vector representing the user's emotional state. Specific operations include extracting acoustic characteristics from audio data and analyzing facial expressions from video data.

[0379] Step 3:

[0380] The server uses Hugging Face's natural language processing model to perform sentiment analysis. It receives the feature vectors from Step 2 as input and feeds them into the model. As output, it identifies sentiment categories (e.g., anger, joy, sadness, etc.). This allows the server to assess the user's emotional state. Specific operations include lexical analysis of text data and calculation of sentiment intensity.

[0381] Step 4:

[0382] The server integrates historical crime data with the aforementioned sentiment data to perform a risk assessment. It uses the results of sentiment analysis and historical databases as input, performs data matching, and calculates a risk score. The output generates the risk assessment results. This process includes searching for similarities in crime patterns and quantifying real-time risk.

[0383] Step 5:

[0384] The server sends an alert to the terminal based on the assessed risk. It uses the risk assessment results from step 4 as input to generate the necessary alert message. As output, it generates alert information combining text messages, alert sounds, and visual notifications. This provides immediate response instructions to security personnel on the terminal. Specific actions include data transmission via communication protocols and triggering the terminal's alarm system.

[0385] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0386] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0387] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0388] [Third Embodiment]

[0389] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0390] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0391] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0392] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0393] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0394] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0395] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0396] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0397] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0398] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0399] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0400] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0401] The embodiments for carrying out the present invention will now be described. This system consists of multiple modules, each performing a specific function. At the core of the system are four modules: a data collection module, a data analysis module, a risk assessment module, and a notification module.

[0402] The data acquisition module acquires data in real time using security monitoring devices, information and communication networks, and environmental sensors. Examples include video data from security cameras, social media posts, and environmental sensor data such as noise and light levels.

[0403] The data analysis module applies natural language processing and computer vision technologies to the data provided by the collection module. This allows for the extraction of specific keywords from social media and the detection of abnormal behavior from camera footage. This enables the immediate detection of potential anomalies and signs of crime.

[0404] The risk assessment module compares past crime data with current conditions based on the analyzed data. This allows for the assessment of the risk of crime occurring in the target area, and the results can be quantified or ranked.

[0405] Finally, the notification module generates an alert based on the risk level calculated by the risk assessment module. This alert is sent to investigators' mobile devices or computers and provides information including specific countermeasures. For example, this may include instructions to increase patrols in specific areas or to change the direction of surveillance cameras.

[0406] For example, if an increase in keywords such as "gathering" or "riot" is detected on social media in a certain area, and at the same time, noise levels in that area rise, this system will assess the risk as high and immediately recommend increased patrols to the police. This makes it possible to take preventative measures before a crime occurs.

[0407] These modules work together to enable real-time data analysis and risk assessment, allowing for rapid and effective crime prevention. Implementing this system will improve public safety and reduce the burden on law enforcement agencies.

[0408] The following describes the processing flow.

[0409] Step 1:

[0410] Server: The data collection module starts up and begins ingesting data from security monitoring devices, social media APIs, and environmental sensors. Video data is divided into a series of image frames, and text data from social media is acquired as a stream.

[0411] Step 2:

[0412] Server: In the pre-processing stage, the collected data is organized. Irrelevant frames are removed from video data, and filtering is performed on text data to remove noise. Sensor data is formatted to a standardized format for easier analysis.

[0413] Step 3:

[0414] Server: The data analysis module extracts specific keywords from text data using natural language processing techniques. In addition, it utilizes computer vision algorithms to detect abnormal movements and behaviors from video data. Based on sensor data, it identifies patterns that are different from the norm.

[0415] Step 4:

[0416] Server: The risk assessment module compares the data with a database of past crimes and evaluates the risk level based on the analysis results. Depending on the frequency and similarity of the abnormal patterns, the risk is classified into ranks such as "low," "medium," and "high."

[0417] Step 5:

[0418] Server: A notification module is activated to generate alerts, creating notifications that include specific countermeasures based on the assessed risk level. This may include increased patrols or monitoring of priority areas.

[0419] Step 6:

[0420] Terminal: Generated warnings and countermeasures are sent in real time to the investigator's terminal and displayed on the screen. The investigator can understand the situation and take immediate action through the terminal.

[0421] Step 7:

[0422] User: Enters an actual activity report in response to a warning from their terminal and sends it to the server. This is saved as feedback data and used for future analysis.

[0423] These processing steps enable the system to efficiently and quickly detect signs of crime and propose effective preventative measures.

[0424] (Example 1)

[0425] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0426] Conventional security systems use only limited information to detect criminal activity, resulting in a lack of accuracy in warnings and the ability to respond quickly. Furthermore, it is difficult to effectively link past crime data with current situations for evaluation, leading to insufficient prediction and prevention.

[0427] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0428] In this invention, the server includes means for using a device to acquire information in real time, analysis means for applying natural language processing technology and image recognition technology using the information, evaluation means for determining and quantifying risk, and notification means for generating warnings and suggesting countermeasures. This enables rapid information collection and detailed analysis, and by providing effective risk assessment and appropriate countermeasures, it becomes possible to enhance safety.

[0429] "Real-time" is a term that refers to a state in which information is acquired and processed the moment it occurs.

[0430] "Devices for acquiring information" refers to hardware or software for automatically collecting data via sensor devices or communication networks.

[0431] "Natural language processing technology" is a term that refers to the technology used by computers to understand, interpret, and generate human language.

[0432] "Image recognition technology" refers to the technology that allows computers to automatically identify and detect objects and features in digital images and videos.

[0433] "Analysis means" refers to a mechanism for processing and analyzing information using collected data to detect specific patterns or anomalies.

[0434] "Evaluation methods" refer to techniques or devices used to calculate, quantify, or rank risks and situations based on analyzed data.

[0435] "Notification means" refers to a method or device for communicating warnings or instructions generated based on the results of analysis and evaluation to the user.

[0436] This invention is a system that enables the acquisition, analysis, and evaluation of information in real time. The system mainly consists of a server, terminals, and users.

[0437] The server acquires information in real time using sensor devices and communication networks. This system includes surveillance cameras, environmental sensors, and information and communication networks, and includes noise levels, image data, and text data collected from social media.

[0438] The terminal receives information transmitted by the server and analyzes the data using natural language processing and image recognition technologies. The natural language processing technology uses algorithms to extract specific keywords from text data, and the image recognition technology uses algorithms to detect anomalies from video data.

[0439] Users can take specific countermeasures based on the evaluation results notified by the server. The evaluation results are displayed on the terminal, from which users can take immediate action. For example, they can issue instructions such as increasing patrols or changing the placement of surveillance cameras.

[0440] For example, if noise levels increase in a certain area and the keyword "demonstration" frequently appears on social media, the system will assess the risk as high and notify users. By taking swift action based on this information, users can prevent potential problems.

[0441] An example of a prompt for a generative AI model is, "Detect increases in 'demonstrations' and 'noise' from recent text data in a specific area, perform a risk assessment, and notify the user." Based on this prompt, the system collects and analyzes information and provides the user with the necessary instructions.

[0442] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0443] Step 1:

[0444] The server acquires information in real time through sensor devices and communication networks. Inputs include surveillance camera footage, social media posts, and environmental sensor readings. This information is stored in a database and prepared for the next analysis step.

[0445] Step 2:

[0446] The terminal analyzes data acquired from the server. Inputs include stored video, text, and environmental data. The terminal applies natural language processing techniques to text data to extract specific keywords. Image recognition techniques are used on image data to detect abnormal behavior and patterns. The output includes a list of extracted keywords and characteristic information of detected anomalies.

[0447] Step 3:

[0448] The server performs a risk assessment based on the analysis results sent from the terminal. It uses keyword lists, anomaly characteristic information, and past crime history data as input. The server uses this information to run a risk assessment model and quantify the risk level for each region. The output is data regarding the risk level for each region.

[0449] Step 4:

[0450] The server activates a notification module based on the evaluation results. The input is risk level data. Based on this, it generates necessary warnings and countermeasures and notifies the user. The output of the notification is a warning message and specific countermeasures sent to the terminal.

[0451] Step 5:

[0452] Users receive notifications displayed on their devices and take specific actions. Inputs are the notified warning messages and corresponding actions. Users use this information to implement necessary defensive measures, such as increasing patrols or adjusting surveillance equipment. Outputs are the actual actions taken in the field, leading to the avoidance of potential risks.

[0453] (Application Example 1)

[0454] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0455] In modern society, detecting early signs of crime and taking swift and effective countermeasures is crucial for ensuring public safety. However, conventional systems lack sufficient real-time data collection and analysis capabilities, resulting in a lack of methods for providing rapid warnings.

[0456] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0457] In this invention, the server is an information processing device for collecting real-time data, and includes an analysis means for analyzing the collected data, detecting signs of crime occurrence and providing safety notifications; a notification means for generating warnings based on the detected signs and presenting specific countermeasures; and a communication means for quickly transmitting warnings to users using a mobile information terminal. This enables citizens to quickly detect signs of crime and take appropriate action.

[0458] An "information processing device" is a device that has the function of collecting data and analyzing it.

[0459] "Analysis means" refers to a function that detects specific events or signs based on collected data and derives related information.

[0460] A "notification method" is a system that communicates information such as warnings and countermeasures to the user based on the analysis results.

[0461] "Communication means" refers to technologies for quickly transmitting information to users via mobile information terminals.

[0462] A "monitoring device" is a device used to record changes in a specific region or environment and collect data.

[0463] "Information and communication means" refers to methods for transmitting data and information using the internet or wireless communication.

[0464] An "external environment sensor" is a sensor that detects the state of the external environment, such as noise and light, and generates data.

[0465] "Data processing means" refers to technologies for analyzing collected information and evaluating risk by comparing past patterns with the current situation.

[0466] In an embodiment of this invention, the server functions as an information processing device, collecting data from the environment in real time. Specifically, it acquires video and audio data using monitoring devices and external environmental sensors, and transmits this information to the server via information communication means. Based on the collected data, the server uses analysis means to detect signs of crime. This analysis may utilize computer vision libraries (e.g., OpenCV) or natural language processing tools (e.g., NLTK). The data to be analyzed may include, for example, keywords extracted from social media posts or abnormal behavior captured in camera footage.

[0467] Next, the server uses data processing tools based on the analyzed information to assess the risk by comparing it with past anomaly occurrence data. This data processing employs statistical methods and machine learning models (e.g., AI models using TensorFlow). Based on the assessment results, if the risk level exceeds a certain threshold, the server generates a warning via a notification system and generates information suggesting specific countermeasures.

[0468] This information is transmitted to mobile devices via communication means and notified to users in real time. Users can use their mobile devices to read the warnings and suggested countermeasures they receive. For example, if keywords such as "gathering" and "noise" suddenly increase on social media in a certain area, and at the same time the noise sensor readings rise, the system will assess this as a moderate risk and notify the user with a message such as, "An event is taking place in your neighborhood. Please take precautions for your safety."

[0469] Examples of prompt messages include the following:

[0470] "Based on social media activity and environmental sensor data in a specific area, assess the safety risks for the next 48 hours. Related keywords include 'gathering,' 'party,' and 'riot,' and environmental sensors indicate increased noise levels."

[0471] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0472] Step 1:

[0473] The server receives real-time data from monitoring devices and external environmental sensors. Inputs include video files, audio data, and social media posts, which are processed via information and communication means. The server formats each data item, preparing it for subsequent analysis.

[0474] Step 2:

[0475] The server uses analysis tools to identify abnormal behavior and important keywords from the received data. It performs motion analysis on video data using computer vision and detects abnormal noises from audio data. It also extracts keywords such as "gathering" and "riot" from social media data using natural language processing techniques. As a result of analyzing the input data, it outputs feature data indicating potential anomalies.

[0476] Step 3:

[0477] The server further processes the analyzed feature data using data processing tools and evaluates the risk by comparing it with past anomaly occurrence data. Based on the feature data obtained as input, it generates an AI model and determines the risk level by quantifying or ranking the results. As a result of the risk evaluation, it outputs one of the following risk levels: low, medium, or high.

[0478] Step 4:

[0479] The server uses notification methods to generate warnings and countermeasures based on the assessed risk level. Based on the input risk information, it formulates specific warning messages and proposed countermeasures (e.g., increased patrols, warnings) and generates them as output.

[0480] Step 5:

[0481] The server sends the generated warnings and countermeasures to the terminal via a communication method. It sends notifications to the user's mobile device, allowing the user to receive warnings in real time and check the countermeasures. The output notifications are displayed on the terminal as alerts prompting the user to take safety measures.

[0482] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0483] This invention combines a crime prevention system that collects and analyzes real-time data with an emotion engine that recognizes user emotions. This system includes a data collection module, a data analysis module, a risk assessment module, a notification module, and an emotion engine module.

[0484] The data collection module acquires a wide range of information in real time using security monitoring devices, information and communication networks, and environmental sensors. This includes video and audio data, social media posts, and data about the surrounding environment.

[0485] The data analysis module extracts specific keywords and features from the collected data. At this stage, the emotion engine performs the function of analyzing the user's emotions from the text and audio data.

[0486] The emotion engine module uses natural language processing and speech analysis technologies to identify emotional states from user utterances and posts. For example, if emotions such as anger or fear are strongly expressed, that data can be given greater weight.

[0487] The risk assessment module uses information obtained from data analysis to compare it with past crime data and evaluate the likelihood of crime occurring. Sentimental data from the sentiment engine also contributes to risk assessment, and posts and locations from users showing particularly high emotional responses are given priority for warnings.

[0488] The notification module generates alerts based on the results of risk assessments and sends them in real time to investigators' terminals. Based on the emotion engine's assessment, it includes instructions to prompt immediate action in high-priority situations. For example, it may issue patrol orders or requests for on-site investigations in specific areas.

[0489] For example, if there is an area where keywords such as "riot" or "danger" are increasing on social media, and the emotion engine detects strong feelings of "anger" or "fear" from posts in that area, the region will be assessed as high risk. Based on this information, a warning for early response is sent to investigators, and on-site investigations and increased security are recommended.

[0490] This system utilizes an emotion engine to take into account even subtle risks that could not be captured by conventional information analysis alone, thereby supporting more effective crime prevention activities.

[0491] The following describes the processing flow.

[0492] Step 1:

[0493] Server: The data collection module collects video, audio, and text data in real time through security monitoring devices, information and communication networks, and environmental sensors. At this stage, information from each data source is aggregated in one place.

[0494] Step 2:

[0495] Server: Preprocesses the collected data. Specifically, it filters video frames, removes noisy audio, and removes spam from text data. This creates a clean dataset suitable for analysis.

[0496] Step 3:

[0497] Server: The data analysis module starts up and uses natural language processing techniques to extract violent or dangerous keywords from text data. Simultaneously, computer vision is used to analyze suspicious behavior from video data.

[0498] Step 4:

[0499] Server: The emotion engine module analyzes text and audio data obtained from users to identify emotions. Particular attention should be paid to data that emphasizes emotions such as anger or fear. This information helps to consider the emotional aspects in crime prevention measures.

[0500] Step 5:

[0501] Server: The risk assessment module integrates analyzed keywords, behavioral patterns, and sentiment data and evaluates their relevance to past crime data. For example, based on high emotional responses and an increase in crime-related keywords in a specific area, the risk is set to "high."

[0502] Step 6:

[0503] Server: The notification module generates alerts based on the risk assessment results and prepares appropriate countermeasures. This comprehensive information is sent in a format optimized for investigators' terminals.

[0504] Step 7:

[0505] Terminal: Based on the received warnings, investigators carry out on-site operations. Immediate responses are required depending on the situation, and orders are given to increase patrols in specific areas or concentrate video surveillance.

[0506] Step 8:

[0507] User: Investigators input their field findings into the system as feedback, which helps improve the accuracy of future analyses. This allows the system to continuously learn and enable more accurate and effective preventative measures.

[0508] (Example 2)

[0509] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0510] In modern society, accurate and rapid risk assessment is essential to prevent crime. However, conventional systems lack sufficient real-time capabilities and consideration of emotions, potentially overlooking potential threats. In particular, the inability to adequately reflect nuanced data, including user emotions, in risk assessments is a significant challenge.

[0511] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0512] In this invention, the server includes means for collecting a wide range of data using sensors and networks, emotion recognition means for extracting keywords and features from the collected data and analyzing emotions, and evaluation means for comparing the emotion data with past data patterns and evaluating the risk. This enables highly accurate assessment of the likelihood of crime occurring and allows for rapid warnings and the presentation of countermeasures.

[0513] A "sensor" is a device used to acquire data from the physical environment, collecting information such as light, sound, and temperature in real time.

[0514] A "network" is an information and communication infrastructure used for sending and receiving data, and refers to communication methods including the internet and local networks.

[0515] "Emotion recognition means" refers to technologies for identifying emotions from a user's text or voice data, using natural language processing and speech analysis technologies to analyze emotions such as "anger" and "fear."

[0516] "Evaluation methods" refer to processes and techniques for determining crime risk based on collected and analyzed data, including analysis that involves cross-referencing with past crime data.

[0517] "Notification means" refers to a mechanism that provides users with warnings and instructions generated based on evaluation results, and includes a function that transmits information in real time through the terminal.

[0518] A "generative AI model" is a machine learning model used to identify emotional states from user statements and posts, and it performs highly accurate analysis from diverse data.

[0519] This invention is a system for assessing crime risk in real time and responding quickly. This system includes the following components:

[0520] First, the server collects data through sensors and the network. Video data is acquired via cameras, and audio data via microphones. APIs are also used to collect posts from social media platforms. This collected data is managed by a data collection module running on the server.

[0521] Next, the collected data is passed to a data analysis module on the server. This module uses natural language processing and speech analysis techniques to extract specific keywords and features from the text and audio. In this process, the collected data is converted into text data, and keyword extraction and feature analysis are performed.

[0522] The emotion recognition module, an emotion engine, utilizes a generative AI model installed on the server to identify the user's emotions from extracted text and audio. For example, if emotions such as "anger" or "fear" are strongly expressed, these are given importance and used in risk assessment. This allows for more accurate crime prediction.

[0523] The server then uses an evaluation system to compare the collected data with past crime data patterns and assesses the risk of crime based on the collected data and sentiment data. Based on the evaluation results, the notification system sends alerts to terminals in real time. This evaluation reflects the output of the sentiment engine and includes instructions that immediately suggest countermeasures for high-risk areas and situations.

[0524] For example, if there is a surge in social media posts in a particular area that frequently contain keywords such as "riot" or "danger," the emotion engine will strongly detect feelings of "anger" and "fear" from the posts. The area will be assessed as high-risk, and investigators will be advised to conduct on-site investigations and increase security.

[0525] Examples of prompts for the generating AI model include, "Analyze the sentiment of social media posts in a specific region and identify areas where feelings of anger or fear are strongly expressed." In this way, servers, terminals, and users cooperate to realize an advanced system for strengthening crime prevention.

[0526] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0527] Step 1:

[0528] The server collects data in real time using sensors and networks. This input data includes video data from surveillance cameras, audio data from microphones, and social media posts obtained through SNS APIs. The server manages this data in collection modules and prepares it for the next analysis step.

[0529] Step 2:

[0530] The server processes input data using a data analysis module. Video data is converted into text data using image recognition technology, and audio data is converted into text data using speech recognition technology. Next, specific keywords such as "danger" and "urgent" are extracted from the text data using natural language processing technology. This output data consists of the analyzed keyword information and related feature data.

[0531] Step 3:

[0532] The server identifies the emotions of the text data analyzed using an emotion engine module. Using a generative AI model, it analyzes the emotions within the input text and strongly identifies emotions such as "anger" and "fear." This output is an emotion score for each text, which is then used in the subsequent risk assessment.

[0533] Step 4:

[0534] The server uses a risk assessment module to match sentiment scores and extracted keywords with historical crime data. Through the assessment mechanism, it calculates the likelihood of a crime occurring based on the input data and ranks the risk level. The output is a dataset with the risk levels assessed. This risk data is used to rank the level of danger and proceed to the next notification step.

[0535] Step 5:

[0536] The terminal generates warnings based on risk assessment results sent from the server. The notification module takes risk data as input and generates necessary warning messages and instructions in real time. This output includes specific action instructions for patrolling specific areas or conducting on-site checks.

[0537] Step 6:

[0538] The user receives notifications from their device and takes prompt action. Based on the warning, the user initiates on-site patrols and verification work and takes appropriate measures according to the situation. This output represents the user's on-site action plan and its execution.

[0539] (Application Example 2)

[0540] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0541] Conventional crime prevention systems were limited to detecting early signs of crime through real-time data analysis, and were unable to adequately consider the emotional state of users, thus limiting their ability to detect and prevent crime early. Furthermore, a challenge in assessing crime risk was the inability to adequately evaluate potential dangers stemming from emotions.

[0542] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0543] In this invention, the server includes information processing means for collecting real-time data, analysis means for analyzing the collected data and detecting signs of crime, and emotion analysis means for analyzing the user's emotions. This makes it possible to perform risk assessment that takes emotion data into account in addition to signs of crime.

[0544] "Real-time data" refers to data that is collected and analyzed instantly without any time delay.

[0545] An "information processing device" is an electronic device used to collect, process, and analyze data.

[0546] "Analysis means" refers to a method or apparatus for processing collected data to detect specific information or anomalies.

[0547] "Emotional analysis means" refers to a technology or device that analyzes and identifies a user's emotional state from data.

[0548] "Notification means" refers to a method or device for generating warnings or information based on analysis results and informing the user or relevant parties.

[0549] A "surveillance system" is a device or program used to continuously observe a specific area or object.

[0550] A "communication network" is an infrastructure for exchanging data between remote locations.

[0551] A "sensor" is a device used to detect and measure specific environmental conditions or variables.

[0552] "Risk assessment" is the process of evaluating the likelihood and risk of a specific event occurring based on collected data.

[0553] The system for realizing this invention is based on a complex sensor network and an advanced data analysis system. The server collects real-time data from various sensors and monitoring devices and performs information processing to analyze that data. This includes an emotion analysis function that identifies the user's emotional state. Emotion analysis utilizes natural language processing and speech analysis technologies to analyze user utterances and posts using a deep learning model. This makes it possible to respond quickly even when the user's emotions are heightened.

[0554] The server uses machine learning frameworks such as TensorFlow to extract specific features from audio and video, and analyzes emotions using Hugging Face's natural language processing model. This analysis result is integrated with past crime data to perform risk assessment and make real-time situational judgments.

[0555] For example, in a security system for a shopping mall, surveillance cameras and audio recording devices monitor activity within the facility and instantly transmit data to a server. The server analyzes this data and, if it determines that "anger" is increasing in a particular location, displays a warning on the security guard's terminal. At this time, the terminal receives both visual and auditory alerts to prompt a quick response.

[0556] An example of a prompt message might be: "Design a system that would inform security personnel how to respond if they detect heightened anger in a specific location." This would enable a quick and appropriate response to potential risks.

[0557] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0558] Step 1:

[0559] The server collects real-time data from various sensors and monitoring devices. It receives sensor information, such as camera and audio data, as input and stores it as digital data. This process includes transmitting the data to the server using network communication.

[0560] Step 2:

[0561] The server performs audio and video analysis using TensorFlow to analyze the collected data. It receives the data saved in step 1 as input and applies a feature extraction algorithm to identify important features. As output, it generates a feature vector representing the user's emotional state. Specific operations include extracting acoustic characteristics from audio data and analyzing facial expressions from video data.

[0562] Step 3:

[0563] The server uses Hugging Face's natural language processing model to perform sentiment analysis. It receives the feature vectors from Step 2 as input and feeds them into the model. As output, it identifies sentiment categories (e.g., anger, joy, sadness, etc.). This allows the server to assess the user's emotional state. Specific operations include lexical analysis of text data and calculation of sentiment intensity.

[0564] Step 4:

[0565] The server integrates historical crime data with the aforementioned sentiment data to perform a risk assessment. It uses the results of sentiment analysis and historical databases as input, performs data matching, and calculates a risk score. The output generates the risk assessment results. This process includes searching for similarities in crime patterns and quantifying real-time risk.

[0566] Step 5:

[0567] The server sends an alert to the terminal based on the assessed risk. It uses the risk assessment results from step 4 as input to generate the necessary alert message. As output, it generates alert information combining text messages, alert sounds, and visual notifications. This provides immediate response instructions to security personnel on the terminal. Specific actions include data transmission via communication protocols and triggering the terminal's alarm system.

[0568] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0569] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0570] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0571] [Fourth Embodiment]

[0572] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0573] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0574] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0575] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0576] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0577] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0578] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0579] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0580] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0581] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0582] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0583] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0584] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0585] The embodiments for carrying out the present invention will now be described. This system consists of multiple modules, each performing a specific function. At the core of the system are four modules: a data collection module, a data analysis module, a risk assessment module, and a notification module.

[0586] The data acquisition module acquires data in real time using security monitoring devices, information and communication networks, and environmental sensors. Examples include video data from security cameras, social media posts, and environmental sensor data such as noise and light levels.

[0587] The data analysis module applies natural language processing and computer vision technologies to the data provided by the collection module. This allows for the extraction of specific keywords from social media and the detection of abnormal behavior from camera footage. This enables the immediate detection of potential anomalies and signs of crime.

[0588] The risk assessment module compares past crime data with current conditions based on the analyzed data. This allows for the assessment of the risk of crime occurring in the target area, and the results can be quantified or ranked.

[0589] Finally, the notification module generates an alert based on the risk level calculated by the risk assessment module. This alert is sent to investigators' mobile devices or computers and provides information including specific countermeasures. For example, this may include instructions to increase patrols in specific areas or to change the direction of surveillance cameras.

[0590] For example, if an increase in keywords such as "gathering" or "riot" is detected on social media in a certain area, and at the same time, noise levels in that area rise, this system will assess the risk as high and immediately recommend increased patrols to the police. This makes it possible to take preventative measures before a crime occurs.

[0591] These modules work together to enable real-time data analysis and risk assessment, allowing for rapid and effective crime prevention. Implementing this system will improve public safety and reduce the burden on law enforcement agencies.

[0592] The following describes the processing flow.

[0593] Step 1:

[0594] Server: The data collection module starts up and begins ingesting data from security monitoring devices, social media APIs, and environmental sensors. Video data is divided into a series of image frames, and text data from social media is acquired as a stream.

[0595] Step 2:

[0596] Server: In the pre-processing stage, the collected data is organized. Irrelevant frames are removed from video data, and filtering is performed on text data to remove noise. Sensor data is formatted to a standardized format for easier analysis.

[0597] Step 3:

[0598] Server: The data analysis module extracts specific keywords from text data using natural language processing techniques. In addition, it utilizes computer vision algorithms to detect abnormal movements and behaviors from video data. Based on sensor data, it identifies patterns that are different from the norm.

[0599] Step 4:

[0600] Server: The risk assessment module compares the data with a database of past crimes and evaluates the risk level based on the analysis results. Depending on the frequency and similarity of the abnormal patterns, the risk is classified into ranks such as "low," "medium," and "high."

[0601] Step 5:

[0602] Server: A notification module is activated to generate alerts, creating notifications that include specific countermeasures based on the assessed risk level. This may include increased patrols or monitoring of priority areas.

[0603] Step 6:

[0604] Terminal: Generated warnings and countermeasures are sent in real time to the investigator's terminal and displayed on the screen. The investigator can understand the situation and take immediate action through the terminal.

[0605] Step 7:

[0606] User: Enters an actual activity report in response to a warning from their terminal and sends it to the server. This is saved as feedback data and used for future analysis.

[0607] These processing steps enable the system to efficiently and quickly detect signs of crime and propose effective preventative measures.

[0608] (Example 1)

[0609] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0610] Conventional security systems use only limited information to detect criminal activity, resulting in a lack of accuracy in warnings and the ability to respond quickly. Furthermore, it is difficult to effectively link past crime data with current situations for evaluation, leading to insufficient prediction and prevention.

[0611] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0612] In this invention, the server includes means for using a device to acquire information in real time, analysis means for applying natural language processing technology and image recognition technology using the information, evaluation means for determining and quantifying risk, and notification means for generating warnings and suggesting countermeasures. This enables rapid information collection and detailed analysis, and by providing effective risk assessment and appropriate countermeasures, it becomes possible to enhance safety.

[0613] "Real-time" is a term that refers to a state in which information is acquired and processed the moment it occurs.

[0614] "Devices for acquiring information" refers to hardware or software for automatically collecting data via sensor devices or communication networks.

[0615] "Natural language processing technology" is a term that refers to the technology used by computers to understand, interpret, and generate human language.

[0616] "Image recognition technology" refers to the technology that allows computers to automatically identify and detect objects and features in digital images and videos.

[0617] "Analysis means" refers to a mechanism for processing and analyzing information using collected data to detect specific patterns or anomalies.

[0618] "Evaluation methods" refer to techniques or devices used to calculate, quantify, or rank risks and situations based on analyzed data.

[0619] "Notification means" refers to a method or device for communicating warnings or instructions generated based on the results of analysis and evaluation to the user.

[0620] This invention is a system that enables the acquisition, analysis, and evaluation of information in real time. The system mainly consists of a server, terminals, and users.

[0621] The server acquires information in real time using sensor devices and communication networks. This system includes surveillance cameras, environmental sensors, and information and communication networks, and includes noise levels, image data, and text data collected from social media.

[0622] The terminal receives information transmitted by the server and analyzes the data using natural language processing and image recognition technologies. The natural language processing technology uses algorithms to extract specific keywords from text data, and the image recognition technology uses algorithms to detect anomalies from video data.

[0623] Users can take specific countermeasures based on the evaluation results notified by the server. The evaluation results are displayed on the terminal, from which users can take immediate action. For example, they can issue instructions such as increasing patrols or changing the placement of surveillance cameras.

[0624] For example, if noise levels increase in a certain area and the keyword "demonstration" frequently appears on social media, the system will assess the risk as high and notify users. By taking swift action based on this information, users can prevent potential problems.

[0625] An example of a prompt for a generative AI model is, "Detect increases in 'demonstrations' and 'noise' from recent text data in a specific area, perform a risk assessment, and notify the user." Based on this prompt, the system collects and analyzes information and provides the user with the necessary instructions.

[0626] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0627] Step 1:

[0628] The server acquires information in real time through sensor devices and communication networks. Inputs include surveillance camera footage, social media posts, and environmental sensor readings. This information is stored in a database and prepared for the next analysis step.

[0629] Step 2:

[0630] The terminal analyzes data acquired from the server. Inputs include stored video, text, and environmental data. The terminal applies natural language processing techniques to text data to extract specific keywords. Image recognition techniques are used on image data to detect abnormal behavior and patterns. The output includes a list of extracted keywords and characteristic information of detected anomalies.

[0631] Step 3:

[0632] The server performs a risk assessment based on the analysis results sent from the terminal. It uses keyword lists, anomaly characteristic information, and past crime history data as input. The server uses this information to run a risk assessment model and quantify the risk level for each region. The output is data regarding the risk level for each region.

[0633] Step 4:

[0634] The server activates a notification module based on the evaluation results. The input is risk level data. Based on this, it generates necessary warnings and countermeasures and notifies the user. The output of the notification is a warning message and specific countermeasures sent to the terminal.

[0635] Step 5:

[0636] Users receive notifications displayed on their devices and take specific actions. Inputs are the notified warning messages and corresponding actions. Users use this information to implement necessary defensive measures, such as increasing patrols or adjusting surveillance equipment. Outputs are the actual actions taken in the field, leading to the avoidance of potential risks.

[0637] (Application Example 1)

[0638] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0639] In modern society, detecting early signs of crime and taking swift and effective countermeasures is crucial for ensuring public safety. However, conventional systems lack sufficient real-time data collection and analysis capabilities, resulting in a lack of methods for providing rapid warnings.

[0640] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0641] In this invention, the server is an information processing device for collecting real-time data, and includes an analysis means for analyzing the collected data, detecting signs of crime occurrence and providing safety notifications; a notification means for generating warnings based on the detected signs and presenting specific countermeasures; and a communication means for quickly transmitting warnings to users using a mobile information terminal. This enables citizens to quickly detect signs of crime and take appropriate action.

[0642] An "information processing device" is a device that has the function of collecting data and analyzing it.

[0643] "Analysis means" refers to a function that detects specific events or signs based on collected data and derives related information.

[0644] A "notification method" is a system that communicates information such as warnings and countermeasures to the user based on the analysis results.

[0645] "Communication means" refers to technologies for quickly transmitting information to users via mobile information terminals.

[0646] A "monitoring device" is a device used to record changes in a specific region or environment and collect data.

[0647] "Information and communication means" refers to methods for transmitting data and information using the internet or wireless communication.

[0648] An "external environment sensor" is a sensor that detects the state of the external environment, such as noise and light, and generates data.

[0649] "Data processing means" refers to technologies for analyzing collected information and evaluating risk by comparing past patterns with the current situation.

[0650] In an embodiment of this invention, the server functions as an information processing device, collecting data from the environment in real time. Specifically, it acquires video and audio data using monitoring devices and external environmental sensors, and transmits this information to the server via information communication means. Based on the collected data, the server uses analysis means to detect signs of crime. This analysis may utilize computer vision libraries (e.g., OpenCV) or natural language processing tools (e.g., NLTK). The data to be analyzed may include, for example, keywords extracted from social media posts or abnormal behavior captured in camera footage.

[0651] Next, the server uses data processing tools based on the analyzed information to assess the risk by comparing it with past anomaly occurrence data. This data processing employs statistical methods and machine learning models (e.g., AI models using TensorFlow). Based on the assessment results, if the risk level exceeds a certain threshold, the server generates a warning via a notification system and generates information suggesting specific countermeasures.

[0652] This information is transmitted to mobile devices via communication means and notified to users in real time. Users can use their mobile devices to read the warnings and suggested countermeasures they receive. For example, if keywords such as "gathering" and "noise" suddenly increase on social media in a certain area, and at the same time the noise sensor readings rise, the system will assess this as a moderate risk and notify the user with a message such as, "An event is taking place in your neighborhood. Please take precautions for your safety."

[0653] Examples of prompt messages include the following:

[0654] "Based on social media activity and environmental sensor data in a specific area, assess the safety risks for the next 48 hours. Related keywords include 'gathering,' 'party,' and 'riot,' and environmental sensors indicate increased noise levels."

[0655] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0656] Step 1:

[0657] The server receives real-time data from monitoring devices and external environmental sensors. Inputs include video files, audio data, and social media posts, which are processed via information and communication means. The server formats each data item, preparing it for subsequent analysis.

[0658] Step 2:

[0659] The server uses analysis tools to identify abnormal behavior and important keywords from the received data. It performs motion analysis on video data using computer vision and detects abnormal noises from audio data. It also extracts keywords such as "gathering" and "riot" from social media data using natural language processing techniques. As a result of analyzing the input data, it outputs feature data indicating potential anomalies.

[0660] Step 3:

[0661] The server further processes the analyzed feature data using data processing tools and evaluates the risk by comparing it with past anomaly occurrence data. Based on the feature data obtained as input, it generates an AI model and determines the risk level by quantifying or ranking the results. As a result of the risk evaluation, it outputs one of the following risk levels: low, medium, or high.

[0662] Step 4:

[0663] The server uses notification methods to generate warnings and countermeasures based on the assessed risk level. Based on the input risk information, it formulates specific warning messages and proposed countermeasures (e.g., increased patrols, warnings) and generates them as output.

[0664] Step 5:

[0665] The server sends the generated warnings and countermeasures to the terminal via a communication method. It sends notifications to the user's mobile device, allowing the user to receive warnings in real time and check the countermeasures. The output notifications are displayed on the terminal as alerts prompting the user to take safety measures.

[0666] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0667] This invention combines a crime prevention system that collects and analyzes real-time data with an emotion engine that recognizes user emotions. This system includes a data collection module, a data analysis module, a risk assessment module, a notification module, and an emotion engine module.

[0668] The data collection module acquires a wide range of information in real time using security monitoring devices, information and communication networks, and environmental sensors. This includes video and audio data, social media posts, and data about the surrounding environment.

[0669] The data analysis module extracts specific keywords and features from the collected data. At this stage, the emotion engine performs the function of analyzing the user's emotions from the text and audio data.

[0670] The emotion engine module uses natural language processing and speech analysis technologies to identify emotional states from user utterances and posts. For example, if emotions such as anger or fear are strongly expressed, that data can be given greater weight.

[0671] The risk assessment module uses information obtained from data analysis to compare it with past crime data and evaluate the likelihood of crime occurring. Sentimental data from the sentiment engine also contributes to risk assessment, and posts and locations from users showing particularly high emotional responses are given priority for warnings.

[0672] The notification module generates alerts based on the results of risk assessments and sends them in real time to investigators' terminals. Based on the emotion engine's assessment, it includes instructions to prompt immediate action in high-priority situations. For example, it may issue patrol orders or requests for on-site investigations in specific areas.

[0673] For example, if there is an area where keywords such as "riot" or "danger" are increasing on social media, and the emotion engine detects strong feelings of "anger" or "fear" from posts in that area, the region will be assessed as high risk. Based on this information, a warning for early response is sent to investigators, and on-site investigations and increased security are recommended.

[0674] This system utilizes an emotion engine to take into account even subtle risks that could not be captured by conventional information analysis alone, thereby supporting more effective crime prevention activities.

[0675] The following describes the processing flow.

[0676] Step 1:

[0677] Server: The data collection module collects video, audio, and text data in real time through security monitoring devices, information and communication networks, and environmental sensors. At this stage, information from each data source is aggregated in one place.

[0678] Step 2:

[0679] Server: Preprocesses the collected data. Specifically, it filters video frames, removes noisy audio, and removes spam from text data. This creates a clean dataset suitable for analysis.

[0680] Step 3:

[0681] Server: The data analysis module starts up and uses natural language processing techniques to extract violent or dangerous keywords from text data. Simultaneously, computer vision is used to analyze suspicious behavior from video data.

[0682] Step 4:

[0683] Server: The emotion engine module analyzes text and audio data obtained from users to identify emotions. Particular attention should be paid to data that emphasizes emotions such as anger or fear. This information helps to consider the emotional aspects in crime prevention measures.

[0684] Step 5:

[0685] Server: The risk assessment module integrates analyzed keywords, behavioral patterns, and sentiment data and evaluates their relevance to past crime data. For example, based on high emotional responses and an increase in crime-related keywords in a specific area, the risk is set to "high."

[0686] Step 6:

[0687] Server: The notification module generates alerts based on the risk assessment results and prepares appropriate countermeasures. This comprehensive information is sent in a format optimized for investigators' terminals.

[0688] Step 7:

[0689] Terminal: Based on the received warnings, investigators carry out on-site operations. Immediate responses are required depending on the situation, and orders are given to increase patrols in specific areas or concentrate video surveillance.

[0690] Step 8:

[0691] User: Investigators input their field findings into the system as feedback, which helps improve the accuracy of future analyses. This allows the system to continuously learn and enable more accurate and effective preventative measures.

[0692] (Example 2)

[0693] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0694] In modern society, accurate and rapid risk assessment is essential to prevent crime. However, conventional systems lack sufficient real-time capabilities and consideration of emotions, potentially overlooking potential threats. In particular, the inability to adequately reflect nuanced data, including user emotions, in risk assessments is a significant challenge.

[0695] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0696] In this invention, the server includes means for collecting a wide range of data using sensors and networks, emotion recognition means for extracting keywords and features from the collected data and analyzing emotions, and evaluation means for comparing the emotion data with past data patterns and evaluating the risk. This enables highly accurate assessment of the likelihood of crime occurring and allows for rapid warnings and the presentation of countermeasures.

[0697] A "sensor" is a device used to acquire data from the physical environment, collecting information such as light, sound, and temperature in real time.

[0698] A "network" is an information and communication infrastructure used for sending and receiving data, and refers to communication methods including the internet and local networks.

[0699] "Emotion recognition means" refers to technologies for identifying emotions from a user's text or voice data, using natural language processing and speech analysis technologies to analyze emotions such as "anger" and "fear."

[0700] "Evaluation methods" refer to processes and techniques for determining crime risk based on collected and analyzed data, including analysis that involves cross-referencing with past crime data.

[0701] "Notification means" refers to a mechanism that provides users with warnings and instructions generated based on evaluation results, and includes a function that transmits information in real time through the terminal.

[0702] A "generative AI model" is a machine learning model used to identify emotional states from user statements and posts, and it performs highly accurate analysis from diverse data.

[0703] This invention is a system for assessing crime risk in real time and responding quickly. This system includes the following components:

[0704] First, the server collects data through sensors and the network. Video data is acquired via cameras, and audio data via microphones. APIs are also used to collect posts from social media platforms. This collected data is managed by a data collection module running on the server.

[0705] Next, the collected data is passed to a data analysis module on the server. This module uses natural language processing and speech analysis techniques to extract specific keywords and features from the text and audio. In this process, the collected data is converted into text data, and keyword extraction and feature analysis are performed.

[0706] The emotion recognition module, an emotion engine, utilizes a generative AI model installed on the server to identify the user's emotions from extracted text and audio. For example, if emotions such as "anger" or "fear" are strongly expressed, these are given importance and used in risk assessment. This allows for more accurate crime prediction.

[0707] The server then uses an evaluation system to compare the collected data with past crime data patterns and assesses the risk of crime based on the collected data and sentiment data. Based on the evaluation results, the notification system sends alerts to terminals in real time. This evaluation reflects the output of the sentiment engine and includes instructions that immediately suggest countermeasures for high-risk areas and situations.

[0708] For example, if there is a surge in social media posts in a particular area that frequently contain keywords such as "riot" or "danger," the emotion engine will strongly detect feelings of "anger" and "fear" from the posts. The area will be assessed as high-risk, and investigators will be advised to conduct on-site investigations and increase security.

[0709] Examples of prompts for the generating AI model include, "Analyze the sentiment of social media posts in a specific region and identify areas where feelings of anger or fear are strongly expressed." In this way, servers, terminals, and users cooperate to realize an advanced system for strengthening crime prevention.

[0710] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0711] Step 1:

[0712] The server collects data in real time using sensors and networks. This input data includes video data from surveillance cameras, audio data from microphones, and social media posts obtained through SNS APIs. The server manages this data in collection modules and prepares it for the next analysis step.

[0713] Step 2:

[0714] The server processes input data using a data analysis module. Video data is converted into text data using image recognition technology, and audio data is converted into text data using speech recognition technology. Next, specific keywords such as "danger" and "urgent" are extracted from the text data using natural language processing technology. This output data consists of the analyzed keyword information and related feature data.

[0715] Step 3:

[0716] The server identifies the emotions of the text data analyzed using an emotion engine module. Using a generative AI model, it analyzes the emotions within the input text and strongly identifies emotions such as "anger" and "fear." This output is an emotion score for each text, which is then used in the subsequent risk assessment.

[0717] Step 4:

[0718] The server uses a risk assessment module to match sentiment scores and extracted keywords with historical crime data. Through the assessment mechanism, it calculates the likelihood of a crime occurring based on the input data and ranks the risk level. The output is a dataset with the risk levels assessed. This risk data is used to rank the level of danger and proceed to the next notification step.

[0719] Step 5:

[0720] The terminal generates warnings based on risk assessment results sent from the server. The notification module takes risk data as input and generates necessary warning messages and instructions in real time. This output includes specific action instructions for patrolling specific areas or conducting on-site checks.

[0721] Step 6:

[0722] The user receives notifications from their device and takes prompt action. Based on the warning, the user initiates on-site patrols and verification work and takes appropriate measures according to the situation. This output represents the user's on-site action plan and its execution.

[0723] (Application Example 2)

[0724] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0725] Conventional crime prevention systems were limited to detecting early signs of crime through real-time data analysis, and were unable to adequately consider the emotional state of users, thus limiting their ability to detect and prevent crime early. Furthermore, a challenge in assessing crime risk was the inability to adequately evaluate potential dangers stemming from emotions.

[0726] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0727] In this invention, the server includes information processing means for collecting real-time data, analysis means for analyzing the collected data and detecting signs of crime, and emotion analysis means for analyzing the user's emotions. This makes it possible to perform risk assessment that takes emotion data into account in addition to signs of crime.

[0728] "Real-time data" refers to data that is collected and analyzed instantly without any time delay.

[0729] An "information processing device" is an electronic device used to collect, process, and analyze data.

[0730] "Analysis means" refers to a method or apparatus for processing collected data to detect specific information or anomalies.

[0731] "Emotional analysis means" refers to a technology or device that analyzes and identifies a user's emotional state from data.

[0732] "Notification means" refers to a method or device for generating warnings or information based on analysis results and informing the user or relevant parties.

[0733] A "surveillance system" is a device or program used to continuously observe a specific area or object.

[0734] A "communication network" is an infrastructure for exchanging data between remote locations.

[0735] A "sensor" is a device used to detect and measure specific environmental conditions or variables.

[0736] "Risk assessment" is the process of evaluating the likelihood and risk of a specific event occurring based on collected data.

[0737] The system for realizing this invention is based on a complex sensor network and an advanced data analysis system. The server collects real-time data from various sensors and monitoring devices and performs information processing to analyze that data. This includes an emotion analysis function that identifies the user's emotional state. Emotion analysis utilizes natural language processing and speech analysis technologies to analyze user utterances and posts using a deep learning model. This makes it possible to respond quickly even when the user's emotions are heightened.

[0738] The server uses machine learning frameworks such as TensorFlow to extract specific features from audio and video, and analyzes emotions using Hugging Face's natural language processing model. This analysis result is integrated with past crime data to perform risk assessment and make real-time situational judgments.

[0739] For example, in a security system for a shopping mall, surveillance cameras and audio recording devices monitor activity within the facility and instantly transmit data to a server. The server analyzes this data and, if it determines that "anger" is increasing in a particular location, displays a warning on the security guard's terminal. At this time, the terminal receives both visual and auditory alerts to prompt a quick response.

[0740] An example of a prompt message might be: "Design a system that would inform security personnel how to respond if they detect heightened anger in a specific location." This would enable a quick and appropriate response to potential risks.

[0741] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0742] Step 1:

[0743] The server collects real-time data from various sensors and monitoring devices. It receives sensor information, such as camera and audio data, as input and stores it as digital data. This process includes transmitting the data to the server using network communication.

[0744] Step 2:

[0745] The server performs audio and video analysis using TensorFlow to analyze the collected data. It receives the data saved in step 1 as input and applies a feature extraction algorithm to identify important features. As output, it generates a feature vector representing the user's emotional state. Specific operations include extracting acoustic characteristics from audio data and analyzing facial expressions from video data.

[0746] Step 3:

[0747] The server uses Hugging Face's natural language processing model to perform sentiment analysis. It receives the feature vectors from Step 2 as input and feeds them into the model. As output, it identifies sentiment categories (e.g., anger, joy, sadness, etc.). This allows the server to assess the user's emotional state. Specific operations include lexical analysis of text data and calculation of sentiment intensity.

[0748] Step 4:

[0749] The server integrates historical crime data with the aforementioned sentiment data to perform a risk assessment. It uses the results of sentiment analysis and historical databases as input, performs data matching, and calculates a risk score. The output generates the risk assessment results. This process includes searching for similarities in crime patterns and quantifying real-time risk.

[0750] Step 5:

[0751] The server sends an alert to the terminal based on the assessed risk. It uses the risk assessment results from step 4 as input to generate the necessary alert message. As output, it generates alert information combining text messages, alert sounds, and visual notifications. This provides immediate response instructions to security personnel on the terminal. Specific actions include data transmission via communication protocols and triggering the terminal's alarm system.

[0752] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0753] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0754] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0755] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0756] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0757] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0758] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0759] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0760] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0761] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0762] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0763] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0764] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0765] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0766] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0767] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0768] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0769] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0770] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0771] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0772] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0773] The following is further disclosed regarding the embodiments described above.

[0774] (Claim 1)

[0775] It is an information processing device for collecting real-time data.

[0776] An analysis means for analyzing data collected by the aforementioned information processing device to detect signs of a crime occurring,

[0777] A notification means that generates a warning based on the signs detected by the analysis means and presents countermeasures,

[0778] A system that includes this.

[0779] (Claim 2)

[0780] The system according to claim 1, wherein a security monitoring device, an information and communication network, and environmental sensors are used as means for collecting data within the region.

[0781] (Claim 3)

[0782] The system according to claim 1, further comprising data processing means for comparing patterns in past crime data with analysis results of real-time data and performing risk assessment.

[0783] "Example 1"

[0784] (Claim 1)

[0785] It is a device for acquiring information in real time.

[0786] An analysis means that applies natural language processing technology and image recognition technology using information acquired by the aforementioned device,

[0787] An evaluation means that determines and quantifies the risk based on the anomaly detected by the analysis means,

[0788] A notification means that generates a warning based on the risk determined by the evaluation means and presents specific countermeasures,

[0789] A system that includes this.

[0790] (Claim 2)

[0791] The system according to claim 1, which acquires information using a sensor device, a communication network, and a monitoring sensor.

[0792] (Claim 3)

[0793] The system according to claim 1, further comprising processing means for comparing past criminal records with the results of real-time information analysis and for evaluating risk.

[0794] "Application Example 1"

[0795] (Claim 1)

[0796] It is an information processing device for collecting real-time data.

[0797] An analysis means that analyzes data collected by the aforementioned information processing device to detect signs of crime and provides safety-related notifications,

[0798] A notification means that generates a warning based on the signs detected by the analysis means and presents specific countermeasures,

[0799] A communication method that uses a mobile information terminal to quickly send a warning to the user,

[0800] A system that includes this.

[0801] (Claim 2)

[0802] The system according to claim 1, wherein a monitoring device, information and communication means, and an external environmental sensor are used as means for collecting data within the region.

[0803] (Claim 3)

[0804] The system according to claim 1, further comprising data processing means for comparing patterns of past anomaly occurrence data with analysis results of real-time data and performing risk assessment.

[0805] "Example 2 of combining an emotion engine"

[0806] (Claim 1)

[0807] A means of collecting a wide range of data using sensors and networks,

[0808] A means of emotion recognition that extracts keywords and features from collected data and analyzes emotions,

[0809] An evaluation method that assesses risk by considering emotional data and comparing it with past data patterns,

[0810] A notification means that generates warnings and presents instructions based on evaluation results,

[0811] A system that includes this.

[0812] (Claim 2)

[0813] The system according to claim 1, characterized in that it uses natural language processing technology and speech analysis technology in data analysis.

[0814] (Claim 3)

[0815] The system according to claim 1, characterized in that it uses a generative AI model for emotion recognition to improve the accuracy of the data.

[0816] "Application example 2 when combining with an emotional engine"

[0817] (Claim 1)

[0818] It is an information processing device for collecting real-time data.

[0819] An analysis means for analyzing data collected by the aforementioned information processing device to detect signs of a crime occurring,

[0820] A means of analyzing user emotions,

[0821] A notification means that generates a warning and suggests countermeasures based on the signs detected by the analysis means and emotion analysis means,

[0822] A system that includes this.

[0823] (Claim 2)

[0824] The system according to claim 1, wherein a monitoring system, a communication network, and sensors are used as means for collecting data within a region.

[0825] (Claim 3)

[0826] The system according to claim 1, further comprising data processing means for performing risk assessment by comparing patterns in past crime data with real-time data and the results of sentiment data analysis. [Explanation of Symbols]

[0827] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. It is an information processing device for collecting real-time data. An analysis means for analyzing data collected by the aforementioned information processing device to detect signs of a crime occurring, A notification means that generates a warning based on the signs detected by the analysis means and presents countermeasures, A system that includes this.

2. The system according to claim 1, wherein a security monitoring device, an information and communication network, and environmental sensors are used as means for collecting data within the region.

3. The system according to claim 1, further comprising data processing means for comparing patterns in past crime data with analysis results of real-time data and performing risk assessment.

Images