System

Abstract

A system is provided.SOLUTION: A system comprising: means for obtaining current location information from a user terminal; means for receiving location information transmitted from the user terminal; means for generating an optimal course of action using past incident data and real-time information; and means for transmitting the generated course of action to the user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Recently, there has been an increase in the number of sudden violent crimes, but there is a problem of a lack of immediate and appropriate information to take appropriate action in such cases. In particular, although news and police information is updated quickly, it is difficult for ordinary citizens to obtain this information in real time and take appropriate action. Furthermore, there is no system for integrating and analyzing past incident data and real-time information, so preventive measures are insufficient. In such a situation, many people are likely to be exposed to unexpected dangers, especially during peak seasons and when traveling, resulting in social anxiety. [Means for solving the problem]

[0005] To solve this problem, the present invention provides the following means. First, a means for acquiring current location information from a user terminal is used. Then, a means for receiving the location information transmitted from the user terminal at a server is provided. Furthermore, a means for generating an optimal action guideline using past incident data and real-time information is provided on the server. By transmitting this generated action guideline to the user terminal, the user can obtain accurate information in real time and take appropriate action. Furthermore, by providing a means for integrating and analyzing past incident data and real-time information, a more accurate action guideline is generated. This mechanism enables users to quickly acquire specific and appropriate action guideline and strengthen self-defense.

[0006] "User terminal" refers to a device that allows a user to connect to the Internet and run applications.

[0007] "Location information" refers to geographical information represented by the latitude and longitude of a specific point.

[0008] "Receiving" refers to the act of taking in data sent from outside.

[0009] "Means" refers to devices or methods used to achieve a particular purpose.

[0010] "Past incident data" refers to data that records information about violent crimes that have occurred in the past.

[0011] "Real-time information" refers to up-to-date information about ongoing events.

[0012] A "code of conduct" refers to instructions that outline specific actions to take in a particular situation.

[0013] "Integration" refers to combining multiple pieces of data or information and treating them as a single entity.

[0014] "Analysis" refers to the act of examining collected data and information in detail and drawing conclusions.

[0015] A "server" refers to a computer system that provides services and data over a network.

[0016] "Generation" refers to the act of creating new data or information.

[0017] "Transmission" refers to the act of sending data from one device to another. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention provides a system that provides a user with an appropriate course of action in real time when the user encounters a violent crime. Specific embodiments of the system will be described below.

[0040] This system consists of a user device, a server, and a cloud AI model. Users access the system using a mobile app and send their location information to the system, allowing them to receive the optimal course of action for their current location.

[0041] User terminal

[0042] A user first launches a mobile app. The user's device uses its built-in GPS to obtain its current location. For example, if the user is in Shibuya, Tokyo, the device obtains the user's latitude and longitude. This location information is sent to the server via an HTTP request. The server processes this request and determines the user's current location.

[0043] server

[0044] The server receives location information sent from the user's device. Based on the received location information, the server sends a request to the cloud AI model, asking it to compare past incident data with real-time information and generate an optimal course of action. Specifically, the server sends JSON data containing location information to the cloud AI model and receives a course of action from the cloud AI model.

[0045] Cloud AI Model

[0046] The cloud AI model integrates and analyzes past incident data and real-time information based on location information received from the server. Past incident data includes records and trends of crimes that have occurred in the past. Real-time information incorporates current news and breaking reports from the police. Based on this information, the cloud AI model generates specific guidelines for users to act safely. For example, guidelines such as "Evacuate to the nearest building and do not move until safety is confirmed."

[0047] Sending guidelines to user terminals

[0048] The server sends the action guidelines received from the cloud AI model to the user's device. The user's device receives this information and displays it on the mobile app. The user can confirm the action guidelines and act safely.

[0049] Specific examples

[0050] A concrete example of this system is shown below. For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation has occurred. When the user launches the app, the device detects their current location and sends that information to the server. The server receives the location information and queries the cloud AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The server receives this information and sends it to the user's device. The user can check the specific course of action on the app and take safe actions.

[0051] As described above, the present invention enables a user to take appropriate action even when encountering a sudden violent incident, thereby strengthening self-defense.

[0052] The processing flow will be explained below.

[0053] Step 1:

[0054] The user launches the mobile app, which starts the application and displays the interface for further action.

[0055] Step 2:

[0056] The device obtains its current location using its built-in GPS: the device's GPS sensor detects latitude and longitude and converts this data into a format that can be used within applications.

[0057] Step 3:

[0058] The device sends the acquired location information to the server using an HTTP POST request, and the location data (latitude and longitude) is sent to the server in JSON format.

[0059] Step 4:

[0060] The server receives the location information sent from the device, analyzes the HTTP request, and extracts the latitude and longitude data included.

[0061] Step 5:

[0062] The server creates a request to send the received location information to the cloud AI model. This request is also in JSON format and is sent to the cloud AI model along with the location information.

[0063] Step 6:

[0064] The cloud AI model integrates and analyzes past incident data and real-time information based on location information received from the server, searching for relevant records from the past incident database and simultaneously obtaining real-time news information and police bulletins.

[0065] Step 7:

[0066] The cloud AI model analyzes the acquired data and generates optimal guidelines for action, such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[0067] Step 8:

[0068] The cloud AI model returns the generated action guidelines to the server, which receives this information and prepares it to send as a response to the user device.

[0069] Step 9:

[0070] The server sends the action guidelines received from the cloud AI model to the user's device, and the action guidelines data is returned to the user's device as an HTTP response.

[0071] Step 10:

[0072] The terminal displays the guidelines received from the server, and the application displays the specific guidelines in a format that the user can immediately check.

[0073] Step 11:

[0074] The user checks the display on the device and acts according to the provided guidelines, such as moving to the nearest evacuation site, and takes self-protective action according to specific instructions.

[0075] Through these steps, users can obtain appropriate guidelines for action in real time and act safely even when they encounter a sudden violent incident.

[0076] Example 1

[0077] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0078] In modern society, when a sudden violent crime occurs, ordinary citizens are required to take swift and appropriate action to ensure safety. However, current systems have difficulty providing real-time information for users to take appropriate action, and safety is not sufficiently ensured. Therefore, a system is needed that allows users to receive optimal guidelines for action in real time based on their current location information.

[0079] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0080] In this invention, the server includes means for receiving current location information from the user terminal, means for sending a prompt message including the location information to the generative AI model, means for generating an optimal action guideline using past incident data and real-time information by the generative AI model, and means for sending the generated action guideline to the user terminal, thereby enabling the user to receive a guideline for taking safe actions in real time based on their current location information.

[0081] A "user terminal" is a communication device that is directly operated by a user, and is equipped with means for acquiring location information and performing data communication with a server.

[0082] "Location information" is data indicating the user's current geographical location, and is expressed as latitude and longitude.

[0083] "Means for receiving" refers to a device or program that has the function of receiving data or information from a sender.

[0084] A "server" is a computer system that communicates with user terminals via a network and receives, processes, and transmits data.

[0085] A "generative AI model" is an artificial intelligence system that uses past incident data and real-time information to analyze and judge specific issues and generate optimal guidelines for action.

[0086] A "prompt sentence" is text data that is input into a generative AI model and includes location information and context.

[0087] "Guidelines for Action" are messages that contain specific instructions or advice for users to act safely in specific situations.

[0088] A "transmitting means" is a device or program that has the function of sending data or information to other devices or computer systems.

[0089] "Past incident data" is a database that collects information about crimes and accidents that have occurred in the past.

[0090] "Real-time information" refers to the latest information, such as ongoing events and breaking news.

[0091] "JSON format" is an abbreviation for JavaScript Object Notation, and is a lightweight data description format used for exchanging and storing data.

[0092] The system of the present invention provides appropriate action guidelines in real time when a user encounters a violent crime. This system is composed of a user terminal, a server, and a cloud AI model. Specific embodiments of each component are described below.

[0093] User terminal

[0094] A user first launches a mobile app. This app runs on a mobile device such as a smartphone or tablet. The user device obtains its current location using its built-in GPS function. For example, if the user is in Shibuya Ward, Tokyo, the device obtains the user's latitude and longitude information. This location information is sent to the server via an HTTP request. The location information is packaged in JSON format.

[0095] server

[0096] The server receives the location information sent from the user device and sends a request to the generative AI model. When the location information arrives at the server, it is extracted and recorded in a log. The server then sends a prompt message containing the location information to the generative AI model. This prompt message instructs the AI ​​model to generate the optimal course of action using past incident data and real-time information. As a concrete example, the prompt message has the following format:

[0097] Location: Latitude 35.658581, Longitude 139.745433

[0098] Context: There is currently a hostage situation occurring in Shibuya, Tokyo. Please provide guidelines for nearby residents to act safely.

[0099] Cloud AI Model

[0100] The cloud AI model integrates and analyzes past incident data and real-time information based on location information and prompts received from the server. This AI model incorporates records of past crimes, current news, and breaking news from the police to provide the user with the optimal course of action. For example, the cloud AI model generates the following course of action:

[0101] Action Guidelines: A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed.

[0102] The generated guidelines are sent back to the server in JSON format.

[0103] Sending and displaying guidance

[0104] The server sends the guidelines received from the cloud AI model to the user's device. The device receives this information and displays it on the mobile app. The user can check the guidelines displayed on the app screen and take safe actions based on them.

[0105] Specific examples

[0106] For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation is occurring. When the user launches the app, the device detects their current location and sends that information to the server. The server receives the location information and queries the generative AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The server sends this information to the user's device, and the user can check the specific course of action on the app.

[0107] The present invention enables a user to take appropriate action in real time even when encountering a sudden violent incident, thereby strengthening self-defense.

[0108] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0109] Step 1:

[0110] A user launches a mobile app. When the user taps the app icon on their smartphone, the app launches. At this point, the app turns on the GPS function in the background and obtains the current location information (latitude and longitude). The input is the user's operation (launching the app), and the output is the current location information. Specifically, location information such as latitude 35.658581 and longitude 139.745433 is extracted.

[0111] Step 2:

[0112] The device sends the acquired current location information to the server via an HTTP request. The device converts the acquired latitude and longitude information into JSON format and sends it to the server as the payload of the HTTP request. The input is the location information obtained from GPS, and the output is an HTTP request in JSON format. Specifically, the following JSON data is generated:

[0113] {

[0114] "latitude": 35.658581,

[0115] "longitude": 139.745433

[0116] }

[0117] Step 3:

[0118] The server processes the received HTTP request and extracts the location information. The server receives the request at the " / location" endpoint and parses the latitude and longitude from the received data. The input is the JSON data sent from the device, and the output is the extracted location information. The specific operation is recorded in the log as "Location information received: latitude 35.658581, longitude 139.745433."

[0119] Step 4:

[0120] The server creates a request to the generative AI model based on the extracted location information. The server generates a prompt sentence including the location information and sends it to the cloud AI model. The input is the extracted location information, and the output is a prompt sentence for the AI ​​model. Specifically, the following prompt sentence is generated:

[0121] Location: Latitude 35.658581, Longitude 139.745433

[0122] Context: There is currently a hostage situation occurring in Shibuya, Tokyo. Please provide guidelines for nearby residents to act safely.

[0123] Step 5:

[0124] The cloud AI model receives the prompt text and performs analysis based on past incident data and real-time information. The input is the prompt text from the server, accumulated incident data, and real-time information, and the output is specific guidelines for action. Analysis is performed and specific guidelines for action that the user should take are generated. The specific actions that are generated include the following:

[0125] Action Guidelines: A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed.

[0126] Step 6:

[0127] The server checks the guidelines received from the cloud AI model and sends them to the user device. The input is the guidelines from the AI ​​model, and the output is an HTTP response to be sent to the user device. Specifically, JSON data containing the guidelines is generated and sent to the user device.

[0128] Step 7:

[0129] The user device analyzes the guidelines received from the server and displays them on the mobile app. The input is the JSON data of the guidelines from the server, and the output is the display on the app screen. Specifically, the app screen displays a message saying, "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed."

[0130] The above is the specific processing flow of the program of this system.

[0131] (Application example 1)

[0132] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0133] The lack of a system that allows autonomous vehicles to take prompt and appropriate action when they encounter a sudden incident or danger is a major issue. In particular, providing and implementing real-time information is difficult for autonomous vehicles, making it difficult to ensure user safety.

[0134] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0135] In this invention, the server includes means for acquiring current location information from the user terminal, means for receiving the location information transmitted from the user terminal, means for generating an optimal course of action using past incident data and real-time information, means for transmitting the generated course of action to the user terminal, and means for executing the course of action in real time in the autonomous vehicle based on the acquired location information, thereby enabling the autonomous vehicle to quickly and appropriately avoid danger and take safe actions.

[0136] A "user terminal" is an electronic device carried by a user that acquires current location information and transmits it to the system.

[0137] "Location information" is information including the current latitude and longitude of the user terminal.

[0138] A "server" is a computer system that receives location information sent from a user device and communicates with the cloud AI model.

[0139] "Past incident data" refers to data that includes records of crimes and accidents that have occurred in the past and their trends.

[0140] "Real-time information" means information that is updated immediately, including current news and breaking police reports.

[0141] "Guidelines for action" are specific instructions and advice for users to act safely.

[0142] An "autonomous vehicle" is a vehicle that can drive autonomously using AI and sensor technology.

[0143] "Autonomous driving algorithms" are the programs and procedures that enable an autonomous vehicle to operate safely and efficiently.

[0144] A "cloud AI model" is an artificial intelligence model that exists on the cloud and analyzes past incident data and real-time information to generate guidelines for action.

[0145] The "means for transmitting to the user terminal" is a function by which the server transmits the generated action guidelines to the user terminal via communication.

[0146] "Displayed in real time" means that the course of action is displayed immediately on the user terminal.

[0147] "Means for executing action guidelines in real time" refers to a function that enables an autonomous vehicle to immediately take appropriate action based on the generated action guidelines.

[0148] The present invention provides a system for providing an appropriate course of action in real time when a user encounters a violent crime, and is applied to an autonomous vehicle. A specific embodiment of this system will be described below.

[0149] System configuration

[0150] This system consists of the following hardware and software:

[0151] User terminal: An electronic device carried by a user that has a built-in GPS module for acquiring location information.

[0152] Server: A computer system that receives location information sent from user devices and communicates with the cloud AI model.

[0153] Cloud AI model: An artificial intelligence model that analyzes past incident data and real-time information to generate action guidelines.

[0154] Self-driving vehicle: A vehicle that can drive autonomously using AI and sensor technology.

[0155] Overall system operation

[0156] 1. Acquisition and transmission of location information

[0157] The user device uses its built-in GPS module to obtain its current location. For example, if the user is in a particular city, it obtains the latitude and longitude information. This location information is then sent to the server via an HTTP request.

[0158] 2. Receiving and analyzing location information

[0159] The server receives location information sent from the user's device and sends a request to the cloud AI model based on the received location information, asking it to integrate past incident data with real-time information to generate the optimal course of action.

[0160] 3. Creating guidelines for action

[0161] The cloud AI model integrates and analyzes past incident data and real-time information based on location information received from the server. Past incident data includes records and trends of past crimes, while real-time information incorporates current news and breaking reports from the police. Based on this information, the cloud AI model generates specific guidelines for users to act safely.

[0162] 4. Implementation of the Code of Conduct

[0163] The server sends the action guidelines received from the cloud AI model to the user's device and simultaneously to the self-driving vehicle. The user's device receives this information and displays it in real time. The self-driving vehicle immediately and automatically executes the optimal avoidance action (e.g., selecting a detour or moving to a safe stopping location) based on the received action guidelines.

[0164] Specific examples

[0165] For example, if a hostage situation occurs while an autonomous vehicle is driving within a city, the system operates as follows: The user's device detects its current location and sends that information to the server. The server then queries the cloud AI model based on the location information and generates a course of action: "A hostage situation is occurring within the city. Please evacuate to the nearest safe stopping area immediately and wait until safety is confirmed." Based on this, the autonomous vehicle immediately moves to a safe area, and the course of action is displayed in real time on the user's device.

[0166] Prompt Sentence Examples

[0167] An example of a prompt sent to the cloud AI model when an incident occurs is, "An emergency has occurred at the current location. Please generate the optimal course of action for the autonomous vehicle."

[0168] In this way, the system of the present invention enables autonomous vehicles to take safe and appropriate actions in real time, ensuring user safety.

[0169] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0170] Step 1:

[0171] The user device obtains its current location information using the built-in GPS module. In this process, the user device generates latitude and longitude data. Specifically, the GPS module measures the current location of the user device and prepares the location information as data. The input is a GPS signal, and the output is latitude and longitude location data.

[0172] Step 2:

[0173] The user device sends the acquired location information to the server. This transmission is performed via an HTTP request. The location information data is processed and sent to the server. Specifically, the location information is converted into JSON format and sent to the server via the HTTP protocol. The input is the location information data, and the output is the location information sent to the server.

[0174] Step 3:

[0175] The server receives the location information sent from the user device. In this process, the server obtains the location information and uses it for the next step of processing. Specifically, the server analyzes the received HTTP request and extracts the location information. The input is the HTTP request, and the output is location information data.

[0176] Step 4:

[0177] The server sends a request to the cloud AI model based on the received location information. This request includes location information, which the cloud AI model uses to analyze past incident data and real-time information. Specifically, the server sends the location information in JSON format to the cloud AI model. The input is location data, and the output is a request to the cloud AI model.

[0178] Step 5:

[0179] The cloud AI model integrates and analyzes past incident data and real-time information based on the received location information to generate optimal action guidelines. Specifically, the cloud AI model inputs location information and analyzes a database of past incidents and current real-time information. The inputs are location information, past incident data, and real-time information, and the output is optimal action guidelines.

[0180] Step 6:

[0181] The server sends the action guidelines received from the cloud AI model to the user device. This transmission allows the user device to obtain the action guidelines. Specifically, the server sends the action guideline data to the user device in JSON format. The input is the action guideline data, and the output is the action guidelines sent to the user device.

[0182] Step 7:

[0183] The user device displays the received action guidelines in real time, allowing the user to take appropriate action. Specifically, the user device analyzes the action guidelines and displays them on the screen. The input is the action guidelines data, and the output is the displayed action guidelines.

[0184] Step 8:

[0185] The server sends the action guidelines received from the cloud AI model to the autonomous vehicle. This transmission allows the autonomous vehicle to obtain the action guidelines. Specifically, the server sends the action guidelines data to the autonomous vehicle in JSON format. The input is the action guidelines data, and the output is the action guidelines sent to the autonomous vehicle.

[0186] Step 9:

[0187] The autonomous vehicle executes optimal avoidance actions in real time based on the received action guidelines. Examples include selecting a detour route or moving to a safe stopping location. Specifically, the autonomous driving algorithm analyzes the action guidelines and controls the vehicle's operation in accordance with them. The input is the action guidelines data, and the output is the executed avoidance action.

[0188] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0189] The present invention provides a system that provides an appropriate course of action in real time when a user encounters a violent crime, and further adjusts the course of action by recognizing the user's emotions. A specific embodiment of this system will be described below.

[0190] This system is composed of a user device, a server, a cloud AI model, and an emotion engine. Users access the system using a mobile app and send their location and emotion information to the system. This allows users to receive the optimal course of action based on their current location and emotional state.

[0191] User terminal

[0192] A user first launches a mobile app. The user's device acquires current location information using its built-in GPS function. At the same time, the device's camera and biometric sensors are used to acquire the user's emotional information. For example, if the user is in Shibuya Ward, Tokyo, the device acquires the user's latitude and longitude information, and then analyzes the user's facial expressions and heart rate to determine their emotional state. This location information and emotional information are sent to the server via an HTTP request. The server processes this request and determines the user's current location and emotional state.

[0193] server

[0194] The server receives location information and emotion information sent from the user's device. Based on the received data, the server sends a request to the cloud AI model, asking it to compare past incident data with real-time information and generate an optimal course of action. Specifically, the server sends JSON data containing location information and emotion information to the cloud AI model and receives a course of action from the cloud AI model.

[0195] Cloud AI Model

[0196] The cloud AI model integrates and analyzes past incident data and real-time information based on location and emotion information received from the server. It searches for relevant records from the past incident database and simultaneously obtains real-time news information and police bulletins. Based on this information, the cloud AI model generates specific guidelines for users to act safely. For example, if a user feels anxious, it will prescribe specific actions for the user, such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[0197] Emotion Engine

[0198] Furthermore, the emotion engine analyzes the user's facial expressions and physiological data (heart rate, breathing rate, etc.) to identify the user's emotional state. This emotional information is reflected in the generation of action guidelines by the cloud AI model. For example, if the user is feeling fear or tension, the emotion engine will detect this and generate action guidelines that include a message of encouragement to reassure the user.

[0199] Sending guidelines to user terminals

[0200] The server sends the guidelines received from the cloud AI model and emotion engine to the user's device. The user's device receives this information and displays it on the mobile app. The user can confirm the guidelines and act safely.

[0201] Specific examples

[0202] A concrete example of this system is shown below. For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation has occurred. When the user launches the app, the device detects their current location and analyzes the user's emotions using the emotion engine. The server receives the location and emotion information and queries the cloud AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The emotion engine senses the user's anxiety and adds a message of encouragement: "It's okay, please rest assured and follow instructions." The server receives this information and sends it to the user's device. The user can check the specific course of action and the encouragement message on the app and take safe actions.

[0203] As described above, the present invention enables users to take appropriate actions when they encounter a sudden violent incident, thereby strengthening their self-defense. Furthermore, by providing guidelines that reflect the user's emotional state, it is possible to provide a sense of psychological security.

[0204] The processing flow will be explained below.

[0205] Step 1:

[0206] The user launches the mobile app, which starts the application and displays the interface for further action.

[0207] Step 2:

[0208] The device obtains its current location using its built-in GPS: the device's GPS sensor detects latitude and longitude and converts this data into a format that can be used within applications.

[0209] Step 3:

[0210] The device uses a camera and biometric sensors to acquire information about the user's emotions. For example, the camera recognizes the user's face and analyzes their facial expressions. Biometric sensors also acquire heart rate and breathing rate to determine the user's emotional state.

[0211] Step 4:

[0212] The device sends the acquired location information and emotion information to the server using an HTTP POST request, and the location information and emotion information are sent to the server in JSON format.

[0213] Step 5:

[0214] The server receives the location information and emotion information sent from the device, analyzes the HTTP request, and extracts the included latitude and longitude data and emotion information.

[0215] Step 6:

[0216] The server creates a request to send the received location and emotion information to the cloud AI model. This request is also in JSON format and is sent to the cloud AI model along with the location and emotion information.

[0217] Step 7:

[0218] The cloud-based AI model integrates and analyzes past incident data and real-time information based on location and emotion information received from the server, searching for relevant records from the past incident database and simultaneously obtaining real-time news information and police bulletins.

[0219] Step 8:

[0220] The cloud AI model analyzes the acquired data and generates optimal guidelines for action, such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[0221] Step 9:

[0222] The emotion engine shares the user's emotional information and adds a complementary message of encouragement to the generated course of action. For example, if the user's emotional state is judged to be anxious, the engine adds the message "Don't worry, just follow the instructions."

[0223] Step 10:

[0224] The cloud AI model and emotion engine then send the generated guidelines and support messages back to the server, which receives this information and prepares it for transmission to the user's device as a response.

[0225] Step 11:

[0226] The server sends the guidelines and support messages received from the cloud AI model and emotion engine to the user's device. The guidelines and support messages are returned to the user's device as an HTTP response.

[0227] Step 12:

[0228] The device displays the guidelines and support messages received from the server. The specific guidelines and support messages are displayed on the application in a format that the user can immediately check.

[0229] Step 13:

[0230] The user checks the display on the device and acts in accordance with the provided guidelines and support messages, for example, by taking self-protective actions according to specific instructions, such as moving to the nearest evacuation site.

[0231] Through these steps, users can obtain appropriate guidelines for action in real time and act safely even when they encounter a sudden violent incident. In addition, the provision of supportive messages tailored to the user's emotional state increases their psychological sense of security.

[0232] Example 2

[0233] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0234] In modern society, the risk of users encountering sudden violent crimes is increasing, necessitating support for taking prompt and appropriate action. However, conventional systems only grasp the user's current location information and do not provide support based on the user's emotional state. This makes it difficult for users to take appropriate action and fails to provide a sense of psychological security. To address these issues, the present invention aims to utilize data including the user's emotional information to provide appropriate guidelines for action and support messages in real time.

[0235] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0236] In this invention, the server includes means for acquiring current location information from the mobile device, means for acquiring user emotional information from the mobile device, means for transmitting the acquired location information and emotional information to a cloud AI model, means for the cloud AI model to generate an optimal course of action using past incident data and real-time information, and means for the emotion engine to analyze the emotional information and add a support message to the course of action. This allows the user to obtain an appropriate course of action based on the current location information and emotional information, enabling them to act safely. In addition, receiving a support message tailored to the user's emotional state also provides a sense of psychological security.

[0237] A "mobile terminal" is an electronic device that a user carries and uses, and includes devices such as smartphones and tablets.

[0238] "Current location information" is latitude and longitude data obtained using the built-in GPS function of the mobile device, and is information that indicates the user's current location.

[0239] "Emotion information" is information indicating the emotional state of the user that is obtained by analyzing physiological data such as facial expressions and heart rate.

[0240] The "server" is a central processing unit that receives location and emotion information sent from mobile devices and sends the data to the cloud AI model.

[0241] The "Cloud AI Model" is an artificial intelligence model with an algorithm that compares past incident data and real-time information based on location information and emotional information received from a server to generate optimal guidelines for action.

[0242] "Past incident data" is a database containing information on incidents that have occurred in the past, and is data that records information such as the type of incident, the location of the incident, and how it was handled.

[0243] "Real-time information" refers to information that allows you to understand the latest situation, such as ongoing incidents and police bulletins.

[0244] "Guidelines for action" are specific instructions for actions that users should take, generated by the cloud AI model, and include specific steps and advice for ensuring safety.

[0245] The "Emotion Engine" is a system that analyzes the user's facial expressions and physiological data to identify emotional information, and adds encouraging messages to the guidelines for action generated by the cloud AI model.

[0246] A "support message" is a message that is generated by the emotion engine based on the user's emotional state and is intended to give a sense of security.

[0247] An "HTTP request" is a protocol request used to communicate data between a server and a mobile device over the Internet.

[0248] "JSON data" is data expressed in JavaScript Object Notation format, a format used for exchanging structured data.

[0249] The present invention provides a system that provides appropriate guidelines for action in real time when a user encounters a sudden violent incident, and further adjusts the guidelines of action by recognizing the user's emotions. How to specifically implement the present invention will be described below.

[0250] System Overview

[0251] This system is composed of a mobile device, a server, a cloud AI model, and an emotion engine. Users access the system using a mobile app and send their location and emotion information to the system, allowing them to receive the optimal course of action based on their current situation.

[0252] Mobile devices

[0253] A user first launches a mobile app. The mobile device acquires its current location using its built-in GPS. At the same time, the device's camera and biometric sensors are used to acquire the user's emotional information. The location information includes latitude and longitude, and the emotional information includes the user's facial expression and heart rate. This information is then sent to the server via an HTTP request.

[0254] server

[0255] The server receives location information and emotion information sent from the mobile device. Based on the received data, the server sends a request to the cloud AI model to compare past incident data with real-time information and generate an optimal course of action. Specifically, the server sends data including location information and emotion information to the cloud AI model and receives a course of action from the cloud AI model.

[0256] Cloud AI Model

[0257] The cloud AI model integrates and analyzes past incident data and real-time information based on location and emotion information received from the server. It searches for relevant records from the past incident database and simultaneously obtains real-time news information and police bulletins. Based on this information, the cloud AI model generates specific guidelines for users to act safely.

[0258] Emotion Engine

[0259] The emotion engine analyzes the user's facial expressions and physiological data (heart rate, breathing rate, etc.) to identify the user's emotional state. This emotional information is reflected in the generation of action guidelines by the cloud AI model. If the user is feeling fear or tension, the emotion engine will detect this and generate action guidelines that include a message of encouragement to reassure the user.

[0260] Sending guidelines to user terminals

[0261] The server sends the guidelines received from the cloud AI model and emotion engine to the user's device. The user's device receives this information and displays it on the mobile app. The user can confirm the guidelines and act safely.

[0262] Specific examples

[0263] For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation is occurring. When the user launches the app, the device detects their current location and analyzes the user's emotions using the emotion engine. The server receives the location and emotion information and queries the cloud AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The emotion engine senses the user's anxiety and adds a message of encouragement: "It's okay, please don't worry and follow instructions." The server sends this information to the user's device, and the user can view the specific course of action and the encouragement message on the app and take safe actions.

[0264] An example of a prompt is as follows:

[0265] User's current location: Shibuya-ku, Tokyo

[0266] User's emotional state: Anxiety

[0267] Past incident data: Shibuya Ward hostage incident

[0268] Real-time information: Police news: "Hostage incident occurring in Shibuya Ward"

[0269] Generate guidelines: Give specific instructions for safe behavior that users should take.

[0270] The present invention enables users to take appropriate action when they encounter a sudden violent incident, thereby strengthening their self-defense. Furthermore, by providing guidelines that reflect the user's emotional state, it is possible to provide a sense of psychological security.

[0271] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0272] Step 1:

[0273] The user launches the mobile app.

[0274] Specific actions: Tap the app icon on your smartphone's home screen to launch the app.

[0275] Input: User action (tapping the app icon).

[0276] Output: App launch.

[0277] Step 2:

[0278] The device obtains the current location.

[0279] Specific operation: Enables the device's built-in GPS function and obtains the user's current location in latitude and longitude.

[0280] Input: Location data from a GPS sensor.

[0281] Output: The location information obtained (e.g., latitude 35.659, longitude 139.702).

[0282] Step 3:

[0283] The device collects emotional information.

[0284] How it works: The camera takes a picture of the user's face and the biometric sensor measures their heart rate. From this data, facial expressions and heart rate are analyzed to determine their emotional state.

[0285] Input: Camera footage, biometric sensor data.

[0286] Output: Identified emotional information (e.g., facial expression anxiety, heart rate 100 bpm).

[0287] Step 4:

[0288] The location information and emotion information acquired by the device are sent to the server via an HTTP request.

[0289] Specific operation: Data including location information and emotion information is packaged in the body of an HTTP request and sent to the server.

[0290] Input: location and emotion.

[0291] Output: The request sent to the server.

[0292] Step 5:

[0293] The server receives an HTTP request from the mobile device.

[0294] What it does: The server listens for HTTP requests and analyzes the received data to extract location and emotion information.

[0295] Input: HTTP request.

[0296] Output: Extracted location and emotion information.

[0297] Step 6:

[0298] The server sends location and emotion information to the cloud AI model.

[0299] Specific operation: Convert location information and emotion information into JSON format and send a request to the cloud AI model.

[0300] Input: Extracted location and emotion information.

[0301] Output: The request sent to the cloud AI model.

[0302] Step 7:

[0303] A cloud AI model receives the request and performs analysis by integrating historical incident data with real-time information.

[0304] How it works: The cloud AI model searches past incident data, retrieves real-time news and police alerts, and uses this data to match location and emotion information.

[0305] Inputs: location information, emotion information, historical incident data, real-time information.

[0306] Output: Consolidated analysis results.

[0307] Step 8:

[0308] Cloud AI models generate optimal courses of action.

[0309] Specific actions: Based on the analysis results, specific guidelines for the user to take are generated.

[0310] Input: Consolidated analysis results.

[0311] Output: Generated course of action (e.g., Take shelter in a nearby building now and avoid going outside until it is safe).

[0312] Step 9:

[0313] The emotion engine generates a support message based on the emotional information.

[0314] How it works: The emotion engine analyzes the user's emotional state and creates a reassuring, supportive message.

[0315] Input: Emotion information.

[0316] Output: Encouraging message (e.g., "Don't worry, just follow the instructions.").

[0317] Step 10:

[0318] The server sends guidelines and support messages to the user terminal.

[0319] Specific operation: Compile a code of conduct and a message of encouragement and send it to the user's device as an HTTP response.

[0320] Input: Guidelines for action, messages of encouragement.

[0321] Output: HTTP response to the user's device.

[0322] Step 11:

[0323] The user's device receives the guidelines and support messages and displays them on the app.

[0324] Specific operation: Analyze the received data and display guidelines and messages of encouragement on the screen.

[0325] Input: The action plan and encouragement message received as an HTTP response.

[0326] Output: The information displayed on the app screen.

[0327] Through the above steps, the system provides users with appropriate guidelines for action and supportive messages tailored to their emotions in real time, providing a sense of psychological security while ensuring their safety.

[0328] (Application example 2)

[0329] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0330] Conventional security systems have difficulty providing appropriate guidance in real time when users encounter dangerous situations. Furthermore, they do not provide guidance that reflects the user's emotional state, preventing users from feeling psychologically secure. Furthermore, there is a lack of technology to display instructions in a format suitable for wearable devices such as smart glasses.

[0331] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring current location information from the user terminal, means for analyzing emotional information acquired from the user terminal, means for receiving location information and emotional information transmitted from the user terminal, means for generating an optimal guideline of action using past incident data and real-time information, means for adjusting the generated guideline of action based on the emotional state, and means for transmitting the generated guideline of action to the user terminal. This enables the user to respond quickly to dangerous situations and act safely and with peace of mind.

[0332] A "user terminal" is a portable information terminal that a user can carry around and that is capable of acquiring and transmitting location information and emotion information.

[0333] "Location information" is data indicating the current latitude and longitude of the user terminal.

[0334] "Emotional information" is information obtained by analyzing the user's physiological and psychological state, such as facial expressions and heart rate.

[0335] The "analyzing means" refers to a technical means for acquiring and analyzing the user's emotional information to identify the user's emotional state.

[0336] "Past incident data" is a database that compiles information about incidents that have occurred in the past.

[0337] "Real-time information" means up-to-date information about ongoing events or situations.

[0338] The "means for generating" refers to the technical means for generating optimal guidelines for action based on the acquired location information and emotional information.

[0339] "Adjustment means" refers to the ability to appropriately change or complement already generated courses of action based on emotional information.

[0340] The "action guidelines" are specific activity guides for users to act safely.

[0341] "Displaying on a display" means visually presenting the generated action guidelines on the screen of the user terminal.

[0342] The present invention provides a system that, when a user encounters a dangerous situation, provides an optimal course of action in real time and adjusts the course of action by analyzing the user's emotional state. A specific embodiment of this system will be described below.

[0343] System configuration

[0344] This system is composed of a user terminal, a server, a cloud AI model, and an emotion engine.

[0345] User terminal

[0346] The user terminal is a wearable device such as smart glasses. This terminal is equipped with a GPS function, a camera, and a biometric sensor, and can acquire current location information and emotional information in real time. For example, the GPS function can be used to acquire latitude and longitude, and the camera and biometric sensor can be used to analyze facial expressions and heart rate to identify emotional information.

[0347] server

[0348] The server receives location and emotional information from the user's device and generates optimal guidelines based on the cloud AI model. The server sends this data to the cloud in JSON format and compares it with past incident data and real-time information to generate guidelines. It also has the function of adjusting guidelines based on the user's emotional information.

[0349] Cloud AI Model

[0350] The cloud AI model integrates past incident data and real-time information using location and emotion information received from the server. Based on the analysis results, it generates specific guidelines for users to act safely. For example, if a user is clearly feeling anxious or scared, it generates guidelines such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[0351] Emotion Engine

[0352] The emotion engine analyzes the user's emotional state based on physiological data such as facial expressions and heart rate obtained from the user's device's camera and biometric sensors. The generated emotional information is sent to a cloud AI model and reflected in the generation and adjustment of action guidelines. The emotion engine can also provide encouraging messages that reflect the user's psychological state.

[0353] Specific examples

[0354] Suppose a user is attending an event in Shinjuku when a commotion breaks out in the crowd, causing their heart rate to rise and anxiety to rise. The smart glasses worn by the user use their GPS to obtain their current location and their camera and heart rate sensor to analyze their emotional information. This data is sent to a server, where a cloud AI model and emotion engine generate a guideline of action, such as "Please evacuate to a nearby building," and a message of encouragement, such as "It's okay, please act with peace of mind." These messages are displayed in real time on the smart glasses' display, and the user follows the instructions to take safe actions.

[0355] Prompt Sentence Examples

[0356] Latitude: 35.6895

[0357] Longitude: 139.6917

[0358] Emotional state: anxiety

[0359] Heart rate: 110

[0360] In this way, the present invention can prevent users from encountering danger, encourage safe behavior, and provide a sense of psychological security.

[0361] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0362] Step 1:

[0363] The user's device acquires current location information and collects emotional information. Specifically, it uses the device's GPS function to acquire latitude and longitude data, and uses the camera and biometric sensors to acquire physiological data such as the user's facial expressions and heart rate. This allows the user's device to obtain current location information (latitude and longitude) and emotional information (facial expression analysis results, heart rate).

[0364] Input: None

[0365] Output: Location information (latitude and longitude), emotional information (facial expression analysis results, heart rate)

[0366] Step 2:

[0367] The user device sends the acquired location information and emotion information to the server. The device converts this data into JSON format and sends it to the server via an HTTP request. At this time, data in the form of a prompt message is generated.

[0368] Input: Location information (latitude and longitude), emotional information (facial expression analysis results, heart rate)

[0369] Output: HTTP request to the server (data in JSON format)

[0370] Step 3:

[0371] The server receives the data sent from the user terminal and sends it to the cloud AI model. The server generates a prompt sentence including the received location information and emotion information and sends a request to the cloud AI model.

[0372] Input: HTTP request from the user device (JSON format data)

[0373] Output: Request to cloud AI model (prompt sentence including location and emotion information)

[0374] Step 4:

[0375] Based on the data received, the cloud AI model compares past incident data with real-time information to generate optimal guidelines for action. The cloud AI model integrates and analyzes location information and emotional information to generate specific guidelines for action that will enable users to act safely.

[0376] Input: Request to cloud AI model (prompt sentence)

[0377] Output: Generated course of action

[0378] Step 5:

[0379] The emotion engine adjusts the course of action based on the user's emotional information and adds a supportive message as needed. The emotion engine analyzes the received course of action and includes an optimal supportive message to alleviate the user's anxiety and fear.

[0380] Input: Generated course of action

[0381] Output: Coordinated course of action and encouraging message

[0382] Step 6:

[0383] The server sends the adjusted action plan to the user terminal. The server generates data including the adjusted action plan and a support message, and sends it to the user terminal as an HTTP response.

[0384] Input: Coordinated course of action and message of encouragement

[0385] Output: HTTP response to the user's device (adjusted guidelines and support message)

[0386] Step 7:

[0387] The user device processes the data received from the server and displays it on the smart glasses display. The device analyzes the received guidelines and support messages and visually presents them on the display.

[0388] Input: HTTP response from the server (adjusted course of action and support message)

[0389] Output: Guidelines and support messages displayed on the screen

[0390] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0391] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0392] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0393] [Second embodiment]

[0394] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0395] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0396] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0397] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0398] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0399] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0400] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0401] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0402] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0403] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0404] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0405] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0406] The present invention provides a system that provides a user with an appropriate course of action in real time when the user encounters a violent crime. Specific embodiments of the system will be described below.

[0407] This system consists of a user device, a server, and a cloud AI model. Users access the system using a mobile app and send their location information to the system, allowing them to receive the optimal course of action for their current location.

[0408] User terminal

[0409] A user first launches a mobile app. The user's device uses its built-in GPS to obtain its current location. For example, if the user is in Shibuya, Tokyo, the device obtains the user's latitude and longitude. This location information is sent to the server via an HTTP request. The server processes this request and determines the user's current location.

[0410] server

[0411] The server receives location information sent from the user's device. Based on the received location information, the server sends a request to the cloud AI model, asking it to compare past incident data with real-time information and generate an optimal course of action. Specifically, the server sends JSON data containing location information to the cloud AI model and receives a course of action from the cloud AI model.

[0412] Cloud AI Model

[0413] The cloud AI model integrates and analyzes past incident data and real-time information based on location information received from the server. Past incident data includes records and trends of crimes that have occurred in the past. Real-time information incorporates current news and breaking reports from the police. Based on this information, the cloud AI model generates specific guidelines for users to act safely. For example, guidelines such as "Evacuate to the nearest building and do not move until safety is confirmed."

[0414] Sending guidelines to user terminals

[0415] The server sends the action guidelines received from the cloud AI model to the user's device. The user's device receives this information and displays it on the mobile app. The user can confirm the action guidelines and act safely.

[0416] Specific examples

[0417] A concrete example of this system is shown below. For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation has occurred. When the user launches the app, the device detects their current location and sends that information to the server. The server receives the location information and queries the cloud AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The server receives this information and sends it to the user's device. The user can check the specific course of action on the app and take safe actions.

[0418] As described above, the present invention enables a user to take appropriate action even when encountering a sudden violent incident, thereby strengthening self-defense.

[0419] The processing flow will be explained below.

[0420] Step 1:

[0421] The user launches the mobile app, which starts the application and displays the interface for further action.

[0422] Step 2:

[0423] The device obtains its current location using its built-in GPS: the device's GPS sensor detects latitude and longitude and converts this data into a format that can be used within applications.

[0424] Step 3:

[0425] The device sends the acquired location information to the server using an HTTP POST request, and the location data (latitude and longitude) is sent to the server in JSON format.

[0426] Step 4:

[0427] The server receives the location information sent from the device, analyzes the HTTP request, and extracts the latitude and longitude data included.

[0428] Step 5:

[0429] The server creates a request to send the received location information to the cloud AI model. This request is also in JSON format and is sent to the cloud AI model along with the location information.

[0430] Step 6:

[0431] The cloud AI model integrates and analyzes past incident data and real-time information based on location information received from the server, searching for relevant records from the past incident database and simultaneously obtaining real-time news information and police bulletins.

[0432] Step 7:

[0433] The cloud AI model analyzes the acquired data and generates optimal guidelines for action, such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[0434] Step 8:

[0435] The cloud AI model returns the generated action guidelines to the server, which receives this information and prepares it to send as a response to the user device.

[0436] Step 9:

[0437] The server sends the action guidelines received from the cloud AI model to the user's device, and the action guidelines data is returned to the user's device as an HTTP response.

[0438] Step 10:

[0439] The terminal displays the guidelines received from the server, and the application displays the specific guidelines in a format that the user can immediately check.

[0440] Step 11:

[0441] The user checks the display on the device and acts according to the provided guidelines, such as moving to the nearest evacuation site, and takes self-protective action according to specific instructions.

[0442] Through these steps, users can obtain appropriate guidelines for action in real time and act safely even when they encounter a sudden violent incident.

[0443] Example 1

[0444] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0445] In modern society, when a sudden violent crime occurs, ordinary citizens are required to take swift and appropriate action to ensure safety. However, current systems have difficulty providing real-time information for users to take appropriate action, and safety is not sufficiently ensured. Therefore, a system is needed that allows users to receive optimal guidelines for action in real time based on their current location information.

[0446] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0447] In this invention, the server includes means for receiving current location information from the user terminal, means for sending a prompt message including the location information to the generative AI model, means for generating an optimal action guideline using past incident data and real-time information by the generative AI model, and means for sending the generated action guideline to the user terminal, thereby enabling the user to receive a guideline for taking safe actions in real time based on their current location information.

[0448] A "user terminal" is a communication device that is directly operated by a user, and is equipped with means for acquiring location information and performing data communication with a server.

[0449] "Location information" is data indicating the user's current geographical location, and is expressed as latitude and longitude.

[0450] "Means for receiving" refers to a device or program that has the function of receiving data or information from a sender.

[0451] A "server" is a computer system that communicates with user terminals via a network and receives, processes, and transmits data.

[0452] A "generative AI model" is an artificial intelligence system that uses past incident data and real-time information to analyze and judge specific issues and generate optimal guidelines for action.

[0453] A "prompt sentence" is text data that is input into a generative AI model and includes location information and context.

[0454] "Guidelines for Action" are messages that contain specific instructions or advice for users to act safely in specific situations.

[0455] A "transmitting means" is a device or program that has the function of sending data or information to other devices or computer systems.

[0456] "Past incident data" is a database that collects information about crimes and accidents that have occurred in the past.

[0457] "Real-time information" refers to the latest information, such as ongoing events and breaking news.

[0458] "JSON format" is an abbreviation for JavaScript Object Notation, and is a lightweight data description format used for exchanging and storing data.

[0459] The system of the present invention provides appropriate action guidelines in real time when a user encounters a violent crime. This system is composed of a user terminal, a server, and a cloud AI model. Specific embodiments of each component are described below.

[0460] User terminal

[0461] A user first launches a mobile app. This app runs on a mobile device such as a smartphone or tablet. The user device obtains its current location using its built-in GPS function. For example, if the user is in Shibuya Ward, Tokyo, the device obtains the user's latitude and longitude information. This location information is sent to the server via an HTTP request. The location information is packaged in JSON format.

[0462] server

[0463] The server receives the location information sent from the user device and sends a request to the generative AI model. When the location information arrives at the server, it is extracted and recorded in a log. The server then sends a prompt message containing the location information to the generative AI model. This prompt message instructs the AI ​​model to generate the optimal course of action using past incident data and real-time information. As a concrete example, the prompt message has the following format:

[0464] Location: Latitude 35.658581, Longitude 139.745433

[0465] Context: There is currently a hostage situation occurring in Shibuya, Tokyo. Please provide guidelines for nearby residents to act safely.

[0466] Cloud AI Model

[0467] The cloud AI model integrates and analyzes past incident data and real-time information based on location information and prompts received from the server. This AI model incorporates records of past crimes, current news, and breaking news from the police to provide the user with the optimal course of action. For example, the cloud AI model generates the following course of action:

[0468] Action Guidelines: A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed.

[0469] The generated guidelines are sent back to the server in JSON format.

[0470] Sending and displaying guidance

[0471] The server sends the guidelines received from the cloud AI model to the user's device. The device receives this information and displays it on the mobile app. The user can check the guidelines displayed on the app screen and take safe actions based on them.

[0472] Specific examples

[0473] For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation is occurring. When the user launches the app, the device detects their current location and sends that information to the server. The server receives the location information and queries the generative AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The server sends this information to the user's device, and the user can check the specific course of action on the app.

[0474] The present invention enables a user to take appropriate action in real time even when encountering a sudden violent incident, thereby strengthening self-defense.

[0475] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0476] Step 1:

[0477] A user launches a mobile app. When the user taps the app icon on their smartphone, the app launches. At this point, the app turns on the GPS function in the background and obtains the current location information (latitude and longitude). The input is the user's operation (launching the app), and the output is the current location information. Specifically, location information such as latitude 35.658581 and longitude 139.745433 is extracted.

[0478] Step 2:

[0479] The device sends the acquired current location information to the server via an HTTP request. The device converts the acquired latitude and longitude information into JSON format and sends it to the server as the payload of the HTTP request. The input is the location information obtained from GPS, and the output is an HTTP request in JSON format. Specifically, the following JSON data is generated:

[0480] {

[0481] "latitude": 35.658581,

[0482] "longitude": 139.745433

[0483] }

[0484] Step 3:

[0485] The server processes the received HTTP request and extracts the location information. The server receives the request at the " / location" endpoint and parses the latitude and longitude from the received data. The input is the JSON data sent from the device, and the output is the extracted location information. The specific operation is recorded in the log as "Location information received: latitude 35.658581, longitude 139.745433."

[0486] Step 4:

[0487] The server creates a request to the generative AI model based on the extracted location information. The server generates a prompt sentence including the location information and sends it to the cloud AI model. The input is the extracted location information, and the output is a prompt sentence for the AI ​​model. Specifically, the following prompt sentence is generated:

[0488] Location: Latitude 35.658581, Longitude 139.745433

[0489] Context: There is currently a hostage situation occurring in Shibuya, Tokyo. Please provide guidelines for nearby residents to act safely.

[0490] Step 5:

[0491] The cloud AI model receives the prompt text and performs analysis based on past incident data and real-time information. The input is the prompt text from the server, accumulated incident data, and real-time information, and the output is specific guidelines for action. Analysis is performed and specific guidelines for action that the user should take are generated. The specific actions that are generated include the following:

[0492] Action Guidelines: A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed.

[0493] Step 6:

[0494] The server checks the guidelines received from the cloud AI model and sends them to the user device. The input is the guidelines from the AI ​​model, and the output is an HTTP response to be sent to the user device. Specifically, JSON data containing the guidelines is generated and sent to the user device.

[0495] Step 7:

[0496] The user device analyzes the guidelines received from the server and displays them on the mobile app. The input is the JSON data of the guidelines from the server, and the output is the display on the app screen. Specifically, the app screen displays a message saying, "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed."

[0497] The above is the specific processing flow of the program of this system.

[0498] (Application example 1)

[0499] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0500] The lack of a system that allows autonomous vehicles to take prompt and appropriate action when they encounter a sudden incident or danger is a major issue. In particular, providing and implementing real-time information is difficult for autonomous vehicles, making it difficult to ensure user safety.

[0501] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0502] In this invention, the server includes means for acquiring current location information from the user terminal, means for receiving the location information transmitted from the user terminal, means for generating an optimal course of action using past incident data and real-time information, means for transmitting the generated course of action to the user terminal, and means for executing the course of action in real time in the autonomous vehicle based on the acquired location information, thereby enabling the autonomous vehicle to quickly and appropriately avoid danger and take safe actions.

[0503] A "user terminal" is an electronic device carried by a user that acquires current location information and transmits it to the system.

[0504] "Location information" is information including the current latitude and longitude of the user terminal.

[0505] A "server" is a computer system that receives location information sent from a user device and communicates with the cloud AI model.

[0506] "Past incident data" refers to data that includes records of crimes and accidents that have occurred in the past and their trends.

[0507] "Real-time information" means information that is updated immediately, including current news and breaking police reports.

[0508] "Guidelines for action" are specific instructions and advice for users to act safely.

[0509] An "autonomous vehicle" is a vehicle that can drive autonomously using AI and sensor technology.

[0510] "Autonomous driving algorithms" are the programs and procedures that enable an autonomous vehicle to operate safely and efficiently.

[0511] A "cloud AI model" is an artificial intelligence model that exists on the cloud and analyzes past incident data and real-time information to generate guidelines for action.

[0512] The "means for transmitting to the user terminal" is a function by which the server transmits the generated action guidelines to the user terminal via communication.

[0513] "Displayed in real time" means that the course of action is displayed immediately on the user terminal.

[0514] "Means for executing action guidelines in real time" refers to a function that enables an autonomous vehicle to immediately take appropriate action based on the generated action guidelines.

[0515] The present invention provides a system for providing an appropriate course of action in real time when a user encounters a violent crime, and is applied to an autonomous vehicle. A specific embodiment of this system will be described below.

[0516] System configuration

[0517] This system consists of the following hardware and software:

[0518] User terminal: An electronic device carried by a user that has a built-in GPS module for acquiring location information.

[0519] Server: A computer system that receives location information sent from user devices and communicates with the cloud AI model.

[0520] Cloud AI model: An artificial intelligence model that analyzes past incident data and real-time information to generate action guidelines.

[0521] Self-driving vehicle: A vehicle that can drive autonomously using AI and sensor technology.

[0522] Overall system operation

[0523] 1. Acquisition and transmission of location information

[0524] The user device uses its built-in GPS module to obtain its current location. For example, if the user is in a particular city, it obtains the latitude and longitude information. This location information is then sent to the server via an HTTP request.

[0525] 2. Receiving and analyzing location information

[0526] The server receives location information sent from the user's device and sends a request to the cloud AI model based on the received location information, asking it to integrate past incident data with real-time information to generate the optimal course of action.

[0527] 3. Creating guidelines for action

[0528] The cloud AI model integrates and analyzes past incident data and real-time information based on location information received from the server. Past incident data includes records and trends of past crimes, while real-time information incorporates current news and breaking reports from the police. Based on this information, the cloud AI model generates specific guidelines for users to act safely.

[0529] 4. Implementation of the Code of Conduct

[0530] The server sends the action guidelines received from the cloud AI model to the user's device and simultaneously to the self-driving vehicle. The user's device receives this information and displays it in real time. The self-driving vehicle immediately and automatically executes the optimal avoidance action (e.g., selecting a detour or moving to a safe stopping location) based on the received action guidelines.

[0531] Specific examples

[0532] For example, if a hostage situation occurs while an autonomous vehicle is driving within a city, the system operates as follows: The user's device detects its current location and sends that information to the server. The server then queries the cloud AI model based on the location information and generates a course of action: "A hostage situation is occurring within the city. Please evacuate to the nearest safe stopping area immediately and wait until safety is confirmed." Based on this, the autonomous vehicle immediately moves to a safe area, and the course of action is displayed in real time on the user's device.

[0533] Prompt Sentence Examples

[0534] An example of a prompt sent to the cloud AI model when an incident occurs is, "An emergency has occurred at the current location. Please generate the optimal course of action for the autonomous vehicle."

[0535] In this way, the system of the present invention enables autonomous vehicles to take safe and appropriate actions in real time, ensuring user safety.

[0536] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0537] Step 1:

[0538] The user device obtains its current location information using the built-in GPS module. In this process, the user device generates latitude and longitude data. Specifically, the GPS module measures the current location of the user device and prepares the location information as data. The input is a GPS signal, and the output is latitude and longitude location data.

[0539] Step 2:

[0540] The user device sends the acquired location information to the server. This transmission is performed via an HTTP request. The location information data is processed and sent to the server. Specifically, the location information is converted into JSON format and sent to the server via the HTTP protocol. The input is the location information data, and the output is the location information sent to the server.

[0541] Step 3:

[0542] The server receives the location information sent from the user device. In this process, the server obtains the location information and uses it for the next step of processing. Specifically, the server analyzes the received HTTP request and extracts the location information. The input is the HTTP request, and the output is location information data.

[0543] Step 4:

[0544] The server sends a request to the cloud AI model based on the received location information. This request includes location information, which the cloud AI model uses to analyze past incident data and real-time information. Specifically, the server sends the location information in JSON format to the cloud AI model. The input is location data, and the output is a request to the cloud AI model.

[0545] Step 5:

[0546] The cloud AI model integrates and analyzes past incident data and real-time information based on the received location information to generate optimal action guidelines. Specifically, the cloud AI model inputs location information and analyzes a database of past incidents and current real-time information. The inputs are location information, past incident data, and real-time information, and the output is optimal action guidelines.

[0547] Step 6:

[0548] The server sends the action guidelines received from the cloud AI model to the user device. This transmission allows the user device to obtain the action guidelines. Specifically, the server sends the action guideline data to the user device in JSON format. The input is the action guideline data, and the output is the action guidelines sent to the user device.

[0549] Step 7:

[0550] The user device displays the received action guidelines in real time, allowing the user to take appropriate action. Specifically, the user device analyzes the action guidelines and displays them on the screen. The input is the action guidelines data, and the output is the displayed action guidelines.

[0551] Step 8:

[0552] The server sends the action guidelines received from the cloud AI model to the autonomous vehicle. This transmission allows the autonomous vehicle to obtain the action guidelines. Specifically, the server sends the action guidelines data to the autonomous vehicle in JSON format. The input is the action guidelines data, and the output is the action guidelines sent to the autonomous vehicle.

[0553] Step 9:

[0554] The autonomous vehicle executes optimal avoidance actions in real time based on the received action guidelines. Examples include selecting a detour route or moving to a safe stopping location. Specifically, the autonomous driving algorithm analyzes the action guidelines and controls the vehicle's operation in accordance with them. The input is the action guidelines data, and the output is the executed avoidance action.

[0555] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0556] The present invention provides a system that provides an appropriate course of action in real time when a user encounters a violent crime, and further adjusts the course of action by recognizing the user's emotions. A specific embodiment of this system will be described below.

[0557] This system is composed of a user device, a server, a cloud AI model, and an emotion engine. Users access the system using a mobile app and send their location and emotion information to the system. This allows users to receive the optimal course of action based on their current location and emotional state.

[0558] User terminal

[0559] A user first launches a mobile app. The user's device acquires current location information using its built-in GPS function. At the same time, the device's camera and biometric sensors are used to acquire the user's emotional information. For example, if the user is in Shibuya Ward, Tokyo, the device acquires the user's latitude and longitude information, and then analyzes the user's facial expressions and heart rate to determine their emotional state. This location information and emotional information are sent to the server via an HTTP request. The server processes this request and determines the user's current location and emotional state.

[0560] server

[0561] The server receives location information and emotion information sent from the user's device. Based on the received data, the server sends a request to the cloud AI model, asking it to compare past incident data with real-time information and generate an optimal course of action. Specifically, the server sends JSON data containing location information and emotion information to the cloud AI model and receives a course of action from the cloud AI model.

[0562] Cloud AI Model

[0563] The cloud AI model integrates and analyzes past incident data and real-time information based on location and emotion information received from the server. It searches for relevant records from the past incident database and simultaneously obtains real-time news information and police bulletins. Based on this information, the cloud AI model generates specific guidelines for users to act safely. For example, if a user feels anxious, it will prescribe specific actions for the user, such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[0564] Emotion Engine

[0565] Furthermore, the emotion engine analyzes the user's facial expressions and physiological data (heart rate, breathing rate, etc.) to identify the user's emotional state. This emotional information is reflected in the generation of action guidelines by the cloud AI model. For example, if the user is feeling fear or tension, the emotion engine will detect this and generate action guidelines that include a message of encouragement to reassure the user.

[0566] Sending guidelines to user terminals

[0567] The server sends the guidelines received from the cloud AI model and emotion engine to the user's device. The user's device receives this information and displays it on the mobile app. The user can confirm the guidelines and act safely.

[0568] Specific examples

[0569] A concrete example of this system is shown below. For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation has occurred. When the user launches the app, the device detects their current location and analyzes the user's emotions using the emotion engine. The server receives the location and emotion information and queries the cloud AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The emotion engine senses the user's anxiety and adds a message of encouragement: "It's okay, please rest assured and follow instructions." The server receives this information and sends it to the user's device. The user can check the specific course of action and the encouragement message on the app and take safe actions.

[0570] As described above, the present invention enables users to take appropriate actions when they encounter a sudden violent incident, thereby strengthening their self-defense. Furthermore, by providing guidelines that reflect the user's emotional state, it is possible to provide a sense of psychological security.

[0571] The processing flow will be explained below.

[0572] Step 1:

[0573] The user launches the mobile app, which starts the application and displays the interface for further action.

[0574] Step 2:

[0575] The device obtains its current location using its built-in GPS: the device's GPS sensor detects latitude and longitude and converts this data into a format that can be used within applications.

[0576] Step 3:

[0577] The device uses a camera and biometric sensors to acquire information about the user's emotions. For example, the camera recognizes the user's face and analyzes their facial expressions. Biometric sensors also acquire heart rate and breathing rate to determine the user's emotional state.

[0578] Step 4:

[0579] The device sends the acquired location information and emotion information to the server using an HTTP POST request, and the location information and emotion information are sent to the server in JSON format.

[0580] Step 5:

[0581] The server receives the location information and emotion information sent from the device, analyzes the HTTP request, and extracts the included latitude and longitude data and emotion information.

[0582] Step 6:

[0583] The server creates a request to send the received location and emotion information to the cloud AI model. This request is also in JSON format and is sent to the cloud AI model along with the location and emotion information.

[0584] Step 7:

[0585] The cloud-based AI model integrates and analyzes past incident data and real-time information based on location and emotion information received from the server, searching for relevant records from the past incident database and simultaneously obtaining real-time news information and police bulletins.

[0586] Step 8:

[0587] The cloud AI model analyzes the acquired data and generates optimal guidelines for action, such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[0588] Step 9:

[0589] The emotion engine shares the user's emotional information and adds a complementary message of encouragement to the generated course of action. For example, if the user's emotional state is judged to be anxious, the engine adds the message "Don't worry, just follow the instructions."

[0590] Step 10:

[0591] The cloud AI model and emotion engine then send the generated guidelines and support messages back to the server, which receives this information and prepares it for transmission to the user's device as a response.

[0592] Step 11:

[0593] The server sends the guidelines and support messages received from the cloud AI model and emotion engine to the user's device. The guidelines and support messages are returned to the user's device as an HTTP response.

[0594] Step 12:

[0595] The device displays the guidelines and support messages received from the server. The specific guidelines and support messages are displayed on the application in a format that the user can immediately check.

[0596] Step 13:

[0597] The user checks the display on the device and acts in accordance with the provided guidelines and support messages, for example, by taking self-protective actions according to specific instructions, such as moving to the nearest evacuation site.

[0598] Through these steps, users can obtain appropriate guidelines for action in real time and act safely even when they encounter a sudden violent incident. In addition, the provision of supportive messages tailored to the user's emotional state increases their psychological sense of security.

[0599] Example 2

[0600] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0601] In modern society, the risk of users encountering sudden violent crimes is increasing, necessitating support for taking prompt and appropriate action. However, conventional systems only grasp the user's current location information and do not provide support based on the user's emotional state. This makes it difficult for users to take appropriate action and fails to provide a sense of psychological security. To address these issues, the present invention aims to utilize data including the user's emotional information to provide appropriate guidelines for action and support messages in real time.

[0602] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0603] In this invention, the server includes means for acquiring current location information from the mobile device, means for acquiring user emotional information from the mobile device, means for transmitting the acquired location information and emotional information to a cloud AI model, means for the cloud AI model to generate an optimal course of action using past incident data and real-time information, and means for the emotion engine to analyze the emotional information and add a support message to the course of action. This allows the user to obtain an appropriate course of action based on the current location information and emotional information, enabling them to act safely. In addition, receiving a support message tailored to the user's emotional state also provides a sense of psychological security.

[0604] A "mobile terminal" is an electronic device that a user carries and uses, and includes devices such as smartphones and tablets.

[0605] "Current location information" is latitude and longitude data obtained using the built-in GPS function of the mobile device, and is information that indicates the user's current location.

[0606] "Emotion information" is information indicating the emotional state of the user that is obtained by analyzing physiological data such as facial expressions and heart rate.

[0607] The "server" is a central processing unit that receives location and emotion information sent from mobile devices and sends the data to the cloud AI model.

[0608] The "Cloud AI Model" is an artificial intelligence model with an algorithm that compares past incident data and real-time information based on location information and emotional information received from a server to generate optimal guidelines for action.

[0609] "Past incident data" is a database containing information on incidents that have occurred in the past, and is data that records information such as the type of incident, the location of the incident, and how it was handled.

[0610] "Real-time information" refers to information that allows you to understand the latest situation, such as ongoing incidents and police bulletins.

[0611] "Guidelines for action" are specific instructions for actions that users should take, generated by the cloud AI model, and include specific steps and advice for ensuring safety.

[0612] The "Emotion Engine" is a system that analyzes the user's facial expressions and physiological data to identify emotional information, and adds encouraging messages to the guidelines for action generated by the cloud AI model.

[0613] A "support message" is a message that is generated by the emotion engine based on the user's emotional state and is intended to give a sense of security.

[0614] An "HTTP request" is a protocol request used to communicate data between a server and a mobile device over the Internet.

[0615] "JSON data" is data expressed in JavaScript Object Notation format, a format used for exchanging structured data.

[0616] The present invention provides a system that provides appropriate guidelines for action in real time when a user encounters a sudden violent incident, and further adjusts the guidelines of action by recognizing the user's emotions. How to specifically implement the present invention will be described below.

[0617] System Overview

[0618] This system is composed of a mobile device, a server, a cloud AI model, and an emotion engine. Users access the system using a mobile app and send their location and emotion information to the system, allowing them to receive the optimal course of action based on their current situation.

[0619] Mobile devices

[0620] A user first launches a mobile app. The mobile device acquires its current location using its built-in GPS. At the same time, the device's camera and biometric sensors are used to acquire the user's emotional information. The location information includes latitude and longitude, and the emotional information includes the user's facial expression and heart rate. This information is then sent to the server via an HTTP request.

[0621] server

[0622] The server receives location information and emotion information sent from the mobile device. Based on the received data, the server sends a request to the cloud AI model to compare past incident data with real-time information and generate an optimal course of action. Specifically, the server sends data including location information and emotion information to the cloud AI model and receives a course of action from the cloud AI model.

[0623] Cloud AI Model

[0624] The cloud AI model integrates and analyzes past incident data and real-time information based on location and emotion information received from the server. It searches for relevant records from the past incident database and simultaneously obtains real-time news information and police bulletins. Based on this information, the cloud AI model generates specific guidelines for users to act safely.

[0625] Emotion Engine

[0626] The emotion engine analyzes the user's facial expressions and physiological data (heart rate, breathing rate, etc.) to identify the user's emotional state. This emotional information is reflected in the generation of action guidelines by the cloud AI model. If the user is feeling fear or tension, the emotion engine will detect this and generate action guidelines that include a message of encouragement to reassure the user.

[0627] Sending guidelines to user terminals

[0628] The server sends the guidelines received from the cloud AI model and emotion engine to the user's device. The user's device receives this information and displays it on the mobile app. The user can confirm the guidelines and act safely.

[0629] Specific examples

[0630] For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation is occurring. When the user launches the app, the device detects their current location and analyzes the user's emotions using the emotion engine. The server receives the location and emotion information and queries the cloud AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The emotion engine senses the user's anxiety and adds a message of encouragement: "It's okay, please don't worry and follow instructions." The server sends this information to the user's device, and the user can view the specific course of action and the encouragement message on the app and take safe actions.

[0631] An example of a prompt is as follows:

[0632] User's current location: Shibuya-ku, Tokyo

[0633] User's emotional state: Anxiety

[0634] Past incident data: Shibuya Ward hostage incident

[0635] Real-time information: Police news: "Hostage incident occurring in Shibuya Ward"

[0636] Generate guidelines: Give specific instructions for safe behavior that users should take.

[0637] The present invention enables users to take appropriate action when they encounter a sudden violent incident, thereby strengthening their self-defense. Furthermore, by providing guidelines that reflect the user's emotional state, it is possible to provide a sense of psychological security.

[0638] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0639] Step 1:

[0640] The user launches the mobile app.

[0641] Specific actions: Tap the app icon on your smartphone's home screen to launch the app.

[0642] Input: User action (tapping the app icon).

[0643] Output: App launch.

[0644] Step 2:

[0645] The device obtains the current location.

[0646] Specific operation: Enables the device's built-in GPS function and obtains the user's current location in latitude and longitude.

[0647] Input: Location data from a GPS sensor.

[0648] Output: The location information obtained (e.g., latitude 35.659, longitude 139.702).

[0649] Step 3:

[0650] The device collects emotional information.

[0651] How it works: The camera takes a picture of the user's face and the biometric sensor measures their heart rate. From this data, facial expressions and heart rate are analyzed to determine their emotional state.

[0652] Input: Camera footage, biometric sensor data.

[0653] Output: Identified emotional information (e.g., facial expression anxiety, heart rate 100 bpm).

[0654] Step 4:

[0655] The location information and emotion information acquired by the device are sent to the server via an HTTP request.

[0656] Specific operation: Data including location information and emotion information is packaged in the body of an HTTP request and sent to the server.

[0657] Input: location and emotion.

[0658] Output: The request sent to the server.

[0659] Step 5:

[0660] The server receives an HTTP request from the mobile device.

[0661] What it does: The server listens for HTTP requests and analyzes the received data to extract location and emotion information.

[0662] Input: HTTP request.

[0663] Output: Extracted location and emotion information.

[0664] Step 6:

[0665] The server sends location and emotion information to the cloud AI model.

[0666] Specific operation: Convert location information and emotion information into JSON format and send a request to the cloud AI model.

[0667] Input: Extracted location and emotion information.

[0668] Output: The request sent to the cloud AI model.

[0669] Step 7:

[0670] A cloud AI model receives the request and performs analysis by integrating historical incident data with real-time information.

[0671] How it works: The cloud AI model searches past incident data, retrieves real-time news and police alerts, and uses this data to match location and emotion information.

[0672] Inputs: location information, emotion information, historical incident data, real-time information.

[0673] Output: Consolidated analysis results.

[0674] Step 8:

[0675] Cloud AI models generate optimal courses of action.

[0676] Specific actions: Based on the analysis results, specific guidelines for the user to take are generated.

[0677] Input: Consolidated analysis results.

[0678] Output: Generated course of action (e.g., Take shelter in a nearby building now and avoid going outside until it is safe).

[0679] Step 9:

[0680] The emotion engine generates a support message based on the emotional information.

[0681] How it works: The emotion engine analyzes the user's emotional state and creates a reassuring, supportive message.

[0682] Input: Emotion information.

[0683] Output: Encouraging message (e.g., "Don't worry, just follow the instructions.").

[0684] Step 10:

[0685] The server sends guidelines and support messages to the user terminal.

[0686] Specific operation: Compile a code of conduct and a message of encouragement and send it to the user's device as an HTTP response.

[0687] Input: Guidelines for action, messages of encouragement.

[0688] Output: HTTP response to the user's device.

[0689] Step 11:

[0690] The user's device receives the guidelines and support messages and displays them on the app.

[0691] Specific operation: Analyze the received data and display guidelines and messages of encouragement on the screen.

[0692] Input: The action plan and encouragement message received as an HTTP response.

[0693] Output: The information displayed on the app screen.

[0694] Through the above steps, the system provides users with appropriate guidelines for action and supportive messages tailored to their emotions in real time, providing a sense of psychological security while ensuring their safety.

[0695] (Application example 2)

[0696] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0697] Conventional security systems have difficulty providing appropriate guidance in real time when users encounter dangerous situations. Furthermore, they do not provide guidance that reflects the user's emotional state, preventing users from feeling psychologically secure. Furthermore, there is a lack of technology to display instructions in a format suitable for wearable devices such as smart glasses.

[0698] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring current location information from the user terminal, means for analyzing emotional information acquired from the user terminal, means for receiving location information and emotional information transmitted from the user terminal, means for generating an optimal guideline of action using past incident data and real-time information, means for adjusting the generated guideline of action based on the emotional state, and means for transmitting the generated guideline of action to the user terminal. This enables the user to respond quickly to dangerous situations and act safely and with peace of mind.

[0699] A "user terminal" is a portable information terminal that a user can carry around and that is capable of acquiring and transmitting location information and emotion information.

[0700] "Location information" is data indicating the current latitude and longitude of the user terminal.

[0701] "Emotional information" is information obtained by analyzing the user's physiological and psychological state, such as facial expressions and heart rate.

[0702] The "analyzing means" refers to a technical means for acquiring and analyzing the user's emotional information to identify the user's emotional state.

[0703] "Past incident data" is a database that compiles information about incidents that have occurred in the past.

[0704] "Real-time information" means up-to-date information about ongoing events or situations.

[0705] The "means for generating" refers to the technical means for generating optimal guidelines for action based on the acquired location information and emotional information.

[0706] "Adjustment means" refers to the ability to appropriately change or complement already generated courses of action based on emotional information.

[0707] The "action guidelines" are specific activity guides for users to act safely.

[0708] "Displaying on a display" means visually presenting the generated action guidelines on the screen of the user terminal.

[0709] The present invention provides a system that, when a user encounters a dangerous situation, provides an optimal course of action in real time and adjusts the course of action by analyzing the user's emotional state. A specific embodiment of this system will be described below.

[0710] System configuration

[0711] This system is composed of a user terminal, a server, a cloud AI model, and an emotion engine.

[0712] User terminal

[0713] The user terminal is a wearable device such as smart glasses. This terminal is equipped with a GPS function, a camera, and a biometric sensor, and can acquire current location information and emotional information in real time. For example, the GPS function can be used to acquire latitude and longitude, and the camera and biometric sensor can be used to analyze facial expressions and heart rate to identify emotional information.

[0714] server

[0715] The server receives location and emotional information from the user's device and generates optimal guidelines based on the cloud AI model. The server sends this data to the cloud in JSON format and compares it with past incident data and real-time information to generate guidelines. It also has the function of adjusting guidelines based on the user's emotional information.

[0716] Cloud AI Model

[0717] The cloud AI model integrates past incident data and real-time information using location and emotion information received from the server. Based on the analysis results, it generates specific guidelines for users to act safely. For example, if a user is clearly feeling anxious or scared, it generates guidelines such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[0718] Emotion Engine

[0719] The emotion engine analyzes the user's emotional state based on physiological data such as facial expressions and heart rate obtained from the user's device's camera and biometric sensors. The generated emotional information is sent to a cloud AI model and reflected in the generation and adjustment of action guidelines. The emotion engine can also provide encouraging messages that reflect the user's psychological state.

[0720] Specific examples

[0721] Suppose a user is attending an event in Shinjuku when a commotion breaks out in the crowd, causing their heart rate to rise and anxiety to rise. The smart glasses worn by the user use their GPS to obtain their current location and their camera and heart rate sensor to analyze their emotional information. This data is sent to a server, where a cloud AI model and emotion engine generate a guideline of action, such as "Please evacuate to a nearby building," and a message of encouragement, such as "It's okay, please act with peace of mind." These messages are displayed in real time on the smart glasses' display, and the user follows the instructions to take safe actions.

[0722] Prompt Sentence Examples

[0723] Latitude: 35.6895

[0724] Longitude: 139.6917

[0725] Emotional state: anxiety

[0726] Heart rate: 110

[0727] In this way, the present invention can prevent users from encountering danger, encourage safe behavior, and provide a sense of psychological security.

[0728] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0729] Step 1:

[0730] The user's device acquires current location information and collects emotional information. Specifically, it uses the device's GPS function to acquire latitude and longitude data, and uses the camera and biometric sensors to acquire physiological data such as the user's facial expressions and heart rate. This allows the user's device to obtain current location information (latitude and longitude) and emotional information (facial expression analysis results, heart rate).

[0731] Input: None

[0732] Output: Location information (latitude and longitude), emotional information (facial expression analysis results, heart rate)

[0733] Step 2:

[0734] The user device sends the acquired location information and emotion information to the server. The device converts this data into JSON format and sends it to the server via an HTTP request. At this time, data in the form of a prompt message is generated.

[0735] Input: Location information (latitude and longitude), emotional information (facial expression analysis results, heart rate)

[0736] Output: HTTP request to the server (data in JSON format)

[0737] Step 3:

[0738] The server receives the data sent from the user terminal and sends it to the cloud AI model. The server generates a prompt sentence including the received location information and emotion information and sends a request to the cloud AI model.

[0739] Input: HTTP request from the user device (JSON format data)

[0740] Output: Request to cloud AI model (prompt sentence including location and emotion information)

[0741] Step 4:

[0742] Based on the data received, the cloud AI model compares past incident data with real-time information to generate optimal guidelines for action. The cloud AI model integrates and analyzes location information and emotional information to generate specific guidelines for action that will enable users to act safely.

[0743] Input: Request to cloud AI model (prompt sentence)

[0744] Output: Generated course of action

[0745] Step 5:

[0746] The emotion engine adjusts the course of action based on the user's emotional information and adds a supportive message as needed. The emotion engine analyzes the received course of action and includes an optimal supportive message to alleviate the user's anxiety and fear.

[0747] Input: Generated course of action

[0748] Output: Coordinated course of action and encouraging message

[0749] Step 6:

[0750] The server sends the adjusted action plan to the user terminal. The server generates data including the adjusted action plan and a support message, and sends it to the user terminal as an HTTP response.

[0751] Input: Coordinated course of action and message of encouragement

[0752] Output: HTTP response to the user's device (adjusted guidelines and support message)

[0753] Step 7:

[0754] The user device processes the data received from the server and displays it on the smart glasses display. The device analyzes the received guidelines and support messages and visually presents them on the display.

[0755] Input: HTTP response from the server (adjusted course of action and support message)

[0756] Output: Guidelines and support messages displayed on the screen

[0757] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0758] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0759] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0760] [Third embodiment]

[0761] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0762] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0763] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0764] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0765] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0766] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0767] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0768] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0769] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0770] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0771] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0772] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0773] The present invention provides a system that provides a user with an appropriate course of action in real time when the user encounters a violent crime. Specific embodiments of the system will be described below.

[0774] This system consists of a user device, a server, and a cloud AI model. Users access the system using a mobile app and send their location information to the system, allowing them to receive the optimal course of action for their current location.

[0775] User terminal

[0776] A user first launches a mobile app. The user's device uses its built-in GPS to obtain its current location. For example, if the user is in Shibuya, Tokyo, the device obtains the user's latitude and longitude. This location information is sent to the server via an HTTP request. The server processes this request and determines the user's current location.

[0777] server

[0778] The server receives location information sent from the user's device. Based on the received location information, the server sends a request to the cloud AI model, asking it to compare past incident data with real-time information and generate an optimal course of action. Specifically, the server sends JSON data containing location information to the cloud AI model and receives a course of action from the cloud AI model.

[0779] Cloud AI Model

[0780] The cloud AI model integrates and analyzes past incident data and real-time information based on location information received from the server. Past incident data includes records and trends of crimes that have occurred in the past. Real-time information incorporates current news and breaking reports from the police. Based on this information, the cloud AI model generates specific guidelines for users to act safely. For example, guidelines such as "Evacuate to the nearest building and do not move until safety is confirmed."

[0781] Sending guidelines to user terminals

[0782] The server sends the action guidelines received from the cloud AI model to the user's device. The user's device receives this information and displays it on the mobile app. The user can confirm the action guidelines and act safely.

[0783] Specific examples

[0784] A concrete example of this system is shown below. For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation has occurred. When the user launches the app, the device detects their current location and sends that information to the server. The server receives the location information and queries the cloud AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The server receives this information and sends it to the user's device. The user can check the specific course of action on the app and take safe actions.

[0785] As described above, the present invention enables a user to take appropriate action even when encountering a sudden violent incident, thereby strengthening self-defense.

[0786] The processing flow will be explained below.

[0787] Step 1:

[0788] The user launches the mobile app, which starts the application and displays the interface for further action.

[0789] Step 2:

[0790] The device obtains its current location using its built-in GPS: the device's GPS sensor detects latitude and longitude and converts this data into a format that can be used within applications.

[0791] Step 3:

[0792] The device sends the acquired location information to the server using an HTTP POST request, and the location data (latitude and longitude) is sent to the server in JSON format.

[0793] Step 4:

[0794] The server receives the location information sent from the device, analyzes the HTTP request, and extracts the latitude and longitude data included.

[0795] Step 5:

[0796] The server creates a request to send the received location information to the cloud AI model. This request is also in JSON format and is sent to the cloud AI model along with the location information.

[0797] Step 6:

[0798] The cloud AI model integrates and analyzes past incident data and real-time information based on location information received from the server, searching for relevant records from the past incident database and simultaneously obtaining real-time news information and police bulletins.

[0799] Step 7:

[0800] The cloud AI model analyzes the acquired data and generates optimal guidelines for action, such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[0801] Step 8:

[0802] The cloud AI model returns the generated action guidelines to the server, which receives this information and prepares it to send as a response to the user device.

[0803] Step 9:

[0804] The server sends the action guidelines received from the cloud AI model to the user's device, and the action guidelines data is returned to the user's device as an HTTP response.

[0805] Step 10:

[0806] The terminal displays the guidelines received from the server, and the application displays the specific guidelines in a format that the user can immediately check.

[0807] Step 11:

[0808] The user checks the display on the device and acts according to the provided guidelines, such as moving to the nearest evacuation site, and takes self-protective action according to specific instructions.

[0809] Through these steps, users can obtain appropriate guidelines for action in real time and act safely even when they encounter a sudden violent incident.

[0810] Example 1

[0811] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0812] In modern society, when a sudden violent crime occurs, ordinary citizens are required to take swift and appropriate action to ensure safety. However, current systems have difficulty providing real-time information for users to take appropriate action, and safety is not sufficiently ensured. Therefore, a system is needed that allows users to receive optimal guidelines for action in real time based on their current location information.

[0813] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0814] In this invention, the server includes means for receiving current location information from the user terminal, means for sending a prompt message including the location information to the generative AI model, means for generating an optimal action guideline using past incident data and real-time information by the generative AI model, and means for sending the generated action guideline to the user terminal, thereby enabling the user to receive a guideline for taking safe actions in real time based on their current location information.

[0815] A "user terminal" is a communication device that is directly operated by a user, and is equipped with means for acquiring location information and performing data communication with a server.

[0816] "Location information" is data indicating the user's current geographical location, and is expressed as latitude and longitude.

[0817] "Means for receiving" refers to a device or program that has the function of receiving data or information from a sender.

[0818] A "server" is a computer system that communicates with user terminals via a network and receives, processes, and transmits data.

[0819] A "generative AI model" is an artificial intelligence system that uses past incident data and real-time information to analyze and judge specific issues and generate optimal guidelines for action.

[0820] A "prompt sentence" is text data that is input into a generative AI model and includes location information and context.

[0821] "Guidelines for Action" are messages that contain specific instructions or advice for users to act safely in specific situations.

[0822] A "transmitting means" is a device or program that has the function of sending data or information to other devices or computer systems.

[0823] "Past incident data" is a database that collects information about crimes and accidents that have occurred in the past.

[0824] "Real-time information" refers to the latest information, such as ongoing events and breaking news.

[0825] "JSON format" is an abbreviation for JavaScript Object Notation, and is a lightweight data description format used for exchanging and storing data.

[0826] The system of the present invention provides appropriate action guidelines in real time when a user encounters a violent crime. This system is composed of a user terminal, a server, and a cloud AI model. Specific embodiments of each component are described below.

[0827] User terminal

[0828] A user first launches a mobile app. This app runs on a mobile device such as a smartphone or tablet. The user device obtains its current location using its built-in GPS function. For example, if the user is in Shibuya Ward, Tokyo, the device obtains the user's latitude and longitude information. This location information is sent to the server via an HTTP request. The location information is packaged in JSON format.

[0829] server

[0830] The server receives the location information sent from the user device and sends a request to the generative AI model. When the location information arrives at the server, it is extracted and recorded in a log. The server then sends a prompt message containing the location information to the generative AI model. This prompt message instructs the AI ​​model to generate the optimal course of action using past incident data and real-time information. As a concrete example, the prompt message has the following format:

[0831] Location: Latitude 35.658581, Longitude 139.745433

[0832] Context: There is currently a hostage situation occurring in Shibuya, Tokyo. Please provide guidelines for nearby residents to act safely.

[0833] Cloud AI Model

[0834] The cloud AI model integrates and analyzes past incident data and real-time information based on location information and prompts received from the server. This AI model incorporates records of past crimes, current news, and breaking news from the police to provide the user with the optimal course of action. For example, the cloud AI model generates the following course of action:

[0835] Action Guidelines: A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed.

[0836] The generated guidelines are sent back to the server in JSON format.

[0837] Sending and displaying guidance

[0838] The server sends the guidelines received from the cloud AI model to the user's device. The device receives this information and displays it on the mobile app. The user can check the guidelines displayed on the app screen and take safe actions based on them.

[0839] Specific examples

[0840] For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation is occurring. When the user launches the app, the device detects their current location and sends that information to the server. The server receives the location information and queries the generative AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The server sends this information to the user's device, and the user can check the specific course of action on the app.

[0841] The present invention enables a user to take appropriate action in real time even when encountering a sudden violent incident, thereby strengthening self-defense.

[0842] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0843] Step 1:

[0844] A user launches a mobile app. When the user taps the app icon on their smartphone, the app launches. At this point, the app turns on the GPS function in the background and obtains the current location information (latitude and longitude). The input is the user's operation (launching the app), and the output is the current location information. Specifically, location information such as latitude 35.658581 and longitude 139.745433 is extracted.

[0845] Step 2:

[0846] The device sends the acquired current location information to the server via an HTTP request. The device converts the acquired latitude and longitude information into JSON format and sends it to the server as the payload of the HTTP request. The input is the location information obtained from GPS, and the output is an HTTP request in JSON format. Specifically, the following JSON data is generated:

[0847] {

[0848] "latitude": 35.658581,

[0849] "longitude": 139.745433

[0850] }

[0851] Step 3:

[0852] The server processes the received HTTP request and extracts the location information. The server receives the request at the " / location" endpoint and parses the latitude and longitude from the received data. The input is the JSON data sent from the device, and the output is the extracted location information. The specific operation is recorded in the log as "Location information received: latitude 35.658581, longitude 139.745433."

[0853] Step 4:

[0854] The server creates a request to the generative AI model based on the extracted location information. The server generates a prompt sentence including the location information and sends it to the cloud AI model. The input is the extracted location information, and the output is a prompt sentence for the AI ​​model. Specifically, the following prompt sentence is generated:

[0855] Location: Latitude 35.658581, Longitude 139.745433

[0856] Context: There is currently a hostage situation occurring in Shibuya, Tokyo. Please provide guidelines for nearby residents to act safely.

[0857] Step 5:

[0858] The cloud AI model receives the prompt text and performs analysis based on past incident data and real-time information. The input is the prompt text from the server, accumulated incident data, and real-time information, and the output is specific guidelines for action. Analysis is performed and specific guidelines for action that the user should take are generated. The specific actions that are generated include the following:

[0859] Action Guidelines: A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed.

[0860] Step 6:

[0861] The server checks the guidelines received from the cloud AI model and sends them to the user device. The input is the guidelines from the AI ​​model, and the output is an HTTP response to be sent to the user device. Specifically, JSON data containing the guidelines is generated and sent to the user device.

[0862] Step 7:

[0863] The user device analyzes the guidelines received from the server and displays them on the mobile app. The input is the JSON data of the guidelines from the server, and the output is the display on the app screen. Specifically, the app screen displays a message saying, "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed."

[0864] The above is the specific processing flow of the program of this system.

[0865] (Application example 1)

[0866] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0867] The lack of a system that allows autonomous vehicles to take prompt and appropriate action when they encounter a sudden incident or danger is a major issue. In particular, providing and implementing real-time information is difficult for autonomous vehicles, making it difficult to ensure user safety.

[0868] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0869] In this invention, the server includes means for acquiring current location information from the user terminal, means for receiving the location information transmitted from the user terminal, means for generating an optimal course of action using past incident data and real-time information, means for transmitting the generated course of action to the user terminal, and means for executing the course of action in real time in the autonomous vehicle based on the acquired location information, thereby enabling the autonomous vehicle to quickly and appropriately avoid danger and take safe actions.

[0870] A "user terminal" is an electronic device carried by a user that acquires current location information and transmits it to the system.

[0871] "Location information" is information including the current latitude and longitude of the user terminal.

[0872] A "server" is a computer system that receives location information sent from a user device and communicates with the cloud AI model.

[0873] "Past incident data" refers to data that includes records of crimes and accidents that have occurred in the past and their trends.

[0874] "Real-time information" means information that is updated immediately, including current news and breaking police reports.

[0875] "Guidelines for action" are specific instructions and advice for users to act safely.

[0876] An "autonomous vehicle" is a vehicle that can drive autonomously using AI and sensor technology.

[0877] "Autonomous driving algorithms" are the programs and procedures that enable an autonomous vehicle to operate safely and efficiently.

[0878] A "cloud AI model" is an artificial intelligence model that exists on the cloud and analyzes past incident data and real-time information to generate guidelines for action.

[0879] The "means for transmitting to the user terminal" is a function by which the server transmits the generated action guidelines to the user terminal via communication.

[0880] "Displayed in real time" means that the course of action is displayed immediately on the user terminal.

[0881] "Means for executing action guidelines in real time" refers to a function that enables an autonomous vehicle to immediately take appropriate action based on the generated action guidelines.

[0882] The present invention provides a system for providing an appropriate course of action in real time when a user encounters a violent crime, and is applied to an autonomous vehicle. A specific embodiment of this system will be described below.

[0883] System configuration

[0884] This system consists of the following hardware and software:

[0885] User terminal: An electronic device carried by a user that has a built-in GPS module for acquiring location information.

[0886] Server: A computer system that receives location information sent from user devices and communicates with the cloud AI model.

[0887] Cloud AI model: An artificial intelligence model that analyzes past incident data and real-time information to generate action guidelines.

[0888] Self-driving vehicle: A vehicle that can drive autonomously using AI and sensor technology.

[0889] Overall system operation

[0890] 1. Acquisition and transmission of location information

[0891] The user device uses its built-in GPS module to obtain its current location. For example, if the user is in a particular city, it obtains the latitude and longitude information. This location information is then sent to the server via an HTTP request.

[0892] 2. Receiving and analyzing location information

[0893] The server receives location information sent from the user's device and sends a request to the cloud AI model based on the received location information, asking it to integrate past incident data with real-time information to generate the optimal course of action.

[0894] 3. Creating guidelines for action

[0895] The cloud AI model integrates and analyzes past incident data and real-time information based on location information received from the server. Past incident data includes records and trends of past crimes, while real-time information incorporates current news and breaking reports from the police. Based on this information, the cloud AI model generates specific guidelines for users to act safely.

[0896] 4. Implementation of the Code of Conduct

[0897] The server sends the action guidelines received from the cloud AI model to the user's device and simultaneously to the self-driving vehicle. The user's device receives this information and displays it in real time. The self-driving vehicle immediately and automatically executes the optimal avoidance action (e.g., selecting a detour or moving to a safe stopping location) based on the received action guidelines.

[0898] Specific examples

[0899] For example, if a hostage situation occurs while an autonomous vehicle is driving within a city, the system operates as follows: The user's device detects its current location and sends that information to the server. The server then queries the cloud AI model based on the location information and generates a course of action: "A hostage situation is occurring within the city. Please evacuate to the nearest safe stopping area immediately and wait until safety is confirmed." Based on this, the autonomous vehicle immediately moves to a safe area, and the course of action is displayed in real time on the user's device.

[0900] Prompt Sentence Examples

[0901] An example of a prompt sent to the cloud AI model when an incident occurs is, "An emergency has occurred at the current location. Please generate the optimal course of action for the autonomous vehicle."

[0902] In this way, the system of the present invention enables autonomous vehicles to take safe and appropriate actions in real time, ensuring user safety.

[0903] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0904] Step 1:

[0905] The user device obtains its current location information using the built-in GPS module. In this process, the user device generates latitude and longitude data. Specifically, the GPS module measures the current location of the user device and prepares the location information as data. The input is a GPS signal, and the output is latitude and longitude location data.

[0906] Step 2:

[0907] The user device sends the acquired location information to the server. This transmission is performed via an HTTP request. The location information data is processed and sent to the server. Specifically, the location information is converted into JSON format and sent to the server via the HTTP protocol. The input is the location information data, and the output is the location information sent to the server.

[0908] Step 3:

[0909] The server receives the location information sent from the user device. In this process, the server obtains the location information and uses it for the next step of processing. Specifically, the server analyzes the received HTTP request and extracts the location information. The input is the HTTP request, and the output is location information data.

[0910] Step 4:

[0911] The server sends a request to the cloud AI model based on the received location information. This request includes location information, which the cloud AI model uses to analyze past incident data and real-time information. Specifically, the server sends the location information in JSON format to the cloud AI model. The input is location data, and the output is a request to the cloud AI model.

[0912] Step 5:

[0913] The cloud AI model integrates and analyzes past incident data and real-time information based on the received location information to generate optimal action guidelines. Specifically, the cloud AI model inputs location information and analyzes a database of past incidents and current real-time information. The inputs are location information, past incident data, and real-time information, and the output is optimal action guidelines.

[0914] Step 6:

[0915] The server sends the action guidelines received from the cloud AI model to the user device. This transmission allows the user device to obtain the action guidelines. Specifically, the server sends the action guideline data to the user device in JSON format. The input is the action guideline data, and the output is the action guidelines sent to the user device.

[0916] Step 7:

[0917] The user device displays the received action guidelines in real time, allowing the user to take appropriate action. Specifically, the user device analyzes the action guidelines and displays them on the screen. The input is the action guidelines data, and the output is the displayed action guidelines.

[0918] Step 8:

[0919] The server sends the action guidelines received from the cloud AI model to the autonomous vehicle. This transmission allows the autonomous vehicle to obtain the action guidelines. Specifically, the server sends the action guidelines data to the autonomous vehicle in JSON format. The input is the action guidelines data, and the output is the action guidelines sent to the autonomous vehicle.

[0920] Step 9:

[0921] The autonomous vehicle executes optimal avoidance actions in real time based on the received action guidelines. Examples include selecting a detour route or moving to a safe stopping location. Specifically, the autonomous driving algorithm analyzes the action guidelines and controls the vehicle's operation in accordance with them. The input is the action guidelines data, and the output is the executed avoidance action.

[0922] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0923] The present invention provides a system that provides an appropriate course of action in real time when a user encounters a violent crime, and further adjusts the course of action by recognizing the user's emotions. A specific embodiment of this system will be described below.

[0924] This system is composed of a user device, a server, a cloud AI model, and an emotion engine. Users access the system using a mobile app and send their location and emotion information to the system. This allows users to receive the optimal course of action based on their current location and emotional state.

[0925] User terminal

[0926] A user first launches a mobile app. The user's device acquires current location information using its built-in GPS function. At the same time, the device's camera and biometric sensors are used to acquire the user's emotional information. For example, if the user is in Shibuya Ward, Tokyo, the device acquires the user's latitude and longitude information, and then analyzes the user's facial expressions and heart rate to determine their emotional state. This location information and emotional information are sent to the server via an HTTP request. The server processes this request and determines the user's current location and emotional state.

[0927] server

[0928] The server receives location information and emotion information sent from the user's device. Based on the received data, the server sends a request to the cloud AI model, asking it to compare past incident data with real-time information and generate an optimal course of action. Specifically, the server sends JSON data containing location information and emotion information to the cloud AI model and receives a course of action from the cloud AI model.

[0929] Cloud AI Model

[0930] The cloud AI model integrates and analyzes past incident data and real-time information based on location and emotion information received from the server. It searches for relevant records from the past incident database and simultaneously obtains real-time news information and police bulletins. Based on this information, the cloud AI model generates specific guidelines for users to act safely. For example, if a user feels anxious, it will prescribe specific actions for the user, such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[0931] Emotion Engine

[0932] Furthermore, the emotion engine analyzes the user's facial expressions and physiological data (heart rate, breathing rate, etc.) to identify the user's emotional state. This emotional information is reflected in the generation of action guidelines by the cloud AI model. For example, if the user is feeling fear or tension, the emotion engine will detect this and generate action guidelines that include a message of encouragement to reassure the user.

[0933] Sending guidelines to user terminals

[0934] The server sends the guidelines received from the cloud AI model and emotion engine to the user's device. The user's device receives this information and displays it on the mobile app. The user can confirm the guidelines and act safely.

[0935] Specific examples

[0936] A concrete example of this system is shown below. For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation has occurred. When the user launches the app, the device detects their current location and analyzes the user's emotions using the emotion engine. The server receives the location and emotion information and queries the cloud AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The emotion engine senses the user's anxiety and adds a message of encouragement: "It's okay, please rest assured and follow instructions." The server receives this information and sends it to the user's device. The user can check the specific course of action and the encouragement message on the app and take safe actions.

[0937] As described above, the present invention enables users to take appropriate actions when they encounter a sudden violent incident, thereby strengthening their self-defense. Furthermore, by providing guidelines that reflect the user's emotional state, it is possible to provide a sense of psychological security.

[0938] The processing flow will be explained below.

[0939] Step 1:

[0940] The user launches the mobile app, which starts the application and displays the interface for further action.

[0941] Step 2:

[0942] The device obtains its current location using its built-in GPS: the device's GPS sensor detects latitude and longitude and converts this data into a format that can be used within applications.

[0943] Step 3:

[0944] The device uses a camera and biometric sensors to acquire information about the user's emotions. For example, the camera recognizes the user's face and analyzes their facial expressions. Biometric sensors also acquire heart rate and breathing rate to determine the user's emotional state.

[0945] Step 4:

[0946] The device sends the acquired location information and emotion information to the server using an HTTP POST request, and the location information and emotion information are sent to the server in JSON format.

[0947] Step 5:

[0948] The server receives the location information and emotion information sent from the device, analyzes the HTTP request, and extracts the included latitude and longitude data and emotion information.

[0949] Step 6:

[0950] The server creates a request to send the received location and emotion information to the cloud AI model. This request is also in JSON format and is sent to the cloud AI model along with the location and emotion information.

[0951] Step 7:

[0952] The cloud-based AI model integrates and analyzes past incident data and real-time information based on location and emotion information received from the server, searching for relevant records from the past incident database and simultaneously obtaining real-time news information and police bulletins.

[0953] Step 8:

[0954] The cloud AI model analyzes the acquired data and generates optimal guidelines for action, such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[0955] Step 9:

[0956] The emotion engine shares the user's emotional information and adds a complementary message of encouragement to the generated course of action. For example, if the user's emotional state is judged to be anxious, the engine adds the message "Don't worry, just follow the instructions."

[0957] Step 10:

[0958] The cloud AI model and emotion engine then send the generated guidelines and support messages back to the server, which receives this information and prepares it for transmission to the user's device as a response.

[0959] Step 11:

[0960] The server sends the guidelines and support messages received from the cloud AI model and emotion engine to the user's device. The guidelines and support messages are returned to the user's device as an HTTP response.

[0961] Step 12:

[0962] The device displays the guidelines and support messages received from the server. The specific guidelines and support messages are displayed on the application in a format that the user can immediately check.

[0963] Step 13:

[0964] The user checks the display on the device and acts in accordance with the provided guidelines and support messages, for example, by taking self-protective actions according to specific instructions, such as moving to the nearest evacuation site.

[0965] Through these steps, users can obtain appropriate guidelines for action in real time and act safely even when they encounter a sudden violent incident. In addition, the provision of supportive messages tailored to the user's emotional state increases their psychological sense of security.

[0966] Example 2

[0967] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0968] In modern society, the risk of users encountering sudden violent crimes is increasing, necessitating support for taking prompt and appropriate action. However, conventional systems only grasp the user's current location information and do not provide support based on the user's emotional state. This makes it difficult for users to take appropriate action and fails to provide a sense of psychological security. To address these issues, the present invention aims to utilize data including the user's emotional information to provide appropriate guidelines for action and support messages in real time.

[0969] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0970] In this invention, the server includes means for acquiring current location information from the mobile device, means for acquiring user emotional information from the mobile device, means for transmitting the acquired location information and emotional information to a cloud AI model, means for the cloud AI model to generate an optimal course of action using past incident data and real-time information, and means for the emotion engine to analyze the emotional information and add a support message to the course of action. This allows the user to obtain an appropriate course of action based on the current location information and emotional information, enabling them to act safely. In addition, receiving a support message tailored to the user's emotional state also provides a sense of psychological security.

[0971] A "mobile terminal" is an electronic device that a user carries and uses, and includes devices such as smartphones and tablets.

[0972] "Current location information" is latitude and longitude data obtained using the built-in GPS function of the mobile device, and is information that indicates the user's current location.

[0973] "Emotion information" is information indicating the emotional state of the user that is obtained by analyzing physiological data such as facial expressions and heart rate.

[0974] The "server" is a central processing unit that receives location and emotion information sent from mobile devices and sends the data to the cloud AI model.

[0975] The "Cloud AI Model" is an artificial intelligence model with an algorithm that compares past incident data and real-time information based on location information and emotional information received from a server to generate optimal guidelines for action.

[0976] "Past incident data" is a database containing information on incidents that have occurred in the past, and is data that records information such as the type of incident, the location of the incident, and how it was handled.

[0977] "Real-time information" refers to information that allows you to understand the latest situation, such as ongoing incidents and police bulletins.

[0978] "Guidelines for action" are specific instructions for actions that users should take, generated by the cloud AI model, and include specific steps and advice for ensuring safety.

[0979] The "Emotion Engine" is a system that analyzes the user's facial expressions and physiological data to identify emotional information, and adds encouraging messages to the guidelines for action generated by the cloud AI model.

[0980] A "support message" is a message that is generated by the emotion engine based on the user's emotional state and is intended to give a sense of security.

[0981] An "HTTP request" is a protocol request used to communicate data between a server and a mobile device over the Internet.

[0982] "JSON data" is data expressed in JavaScript Object Notation format, a format used for exchanging structured data.

[0983] The present invention provides a system that provides appropriate guidelines for action in real time when a user encounters a sudden violent incident, and further adjusts the guidelines of action by recognizing the user's emotions. How to specifically implement the present invention will be described below.

[0984] System Overview

[0985] This system is composed of a mobile device, a server, a cloud AI model, and an emotion engine. Users access the system using a mobile app and send their location and emotion information to the system, allowing them to receive the optimal course of action based on their current situation.

[0986] Mobile devices

[0987] A user first launches a mobile app. The mobile device acquires its current location using its built-in GPS. At the same time, the device's camera and biometric sensors are used to acquire the user's emotional information. The location information includes latitude and longitude, and the emotional information includes the user's facial expression and heart rate. This information is then sent to the server via an HTTP request.

[0988] server

[0989] The server receives location information and emotion information sent from the mobile device. Based on the received data, the server sends a request to the cloud AI model to compare past incident data with real-time information and generate an optimal course of action. Specifically, the server sends data including location information and emotion information to the cloud AI model and receives a course of action from the cloud AI model.

[0990] Cloud AI Model

[0991] The cloud AI model integrates and analyzes past incident data and real-time information based on location and emotion information received from the server. It searches for relevant records from the past incident database and simultaneously obtains real-time news information and police bulletins. Based on this information, the cloud AI model generates specific guidelines for users to act safely.

[0992] Emotion Engine

[0993] The emotion engine analyzes the user's facial expressions and physiological data (heart rate, breathing rate, etc.) to identify the user's emotional state. This emotional information is reflected in the generation of action guidelines by the cloud AI model. If the user is feeling fear or tension, the emotion engine will detect this and generate action guidelines that include a message of encouragement to reassure the user.

[0994] Sending guidelines to user terminals

[0995] The server sends the guidelines received from the cloud AI model and emotion engine to the user's device. The user's device receives this information and displays it on the mobile app. The user can confirm the guidelines and act safely.

[0996] Specific examples

[0997] For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation is occurring. When the user launches the app, the device detects their current location and analyzes the user's emotions using the emotion engine. The server receives the location and emotion information and queries the cloud AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The emotion engine senses the user's anxiety and adds a message of encouragement: "It's okay, please don't worry and follow instructions." The server sends this information to the user's device, and the user can view the specific course of action and the encouragement message on the app and take safe actions.

[0998] An example of a prompt is as follows:

[0999] User's current location: Shibuya-ku, Tokyo

[1000] User's emotional state: Anxiety

[1001] Past incident data: Shibuya Ward hostage incident

[1002] Real-time information: Police news: "Hostage incident occurring in Shibuya Ward"

[1003] Generate guidelines: Give specific instructions for safe behavior that users should take.

[1004] The present invention enables users to take appropriate action when they encounter a sudden violent incident, thereby strengthening their self-defense. Furthermore, by providing guidelines that reflect the user's emotional state, it is possible to provide a sense of psychological security.

[1005] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1006] Step 1:

[1007] The user launches the mobile app.

[1008] Specific actions: Tap the app icon on your smartphone's home screen to launch the app.

[1009] Input: User action (tapping the app icon).

[1010] Output: App launch.

[1011] Step 2:

[1012] The device obtains the current location.

[1013] Specific operation: Enables the device's built-in GPS function and obtains the user's current location in latitude and longitude.

[1014] Input: Location data from a GPS sensor.

[1015] Output: The location information obtained (e.g., latitude 35.659, longitude 139.702).

[1016] Step 3:

[1017] The device collects emotional information.

[1018] How it works: The camera takes a picture of the user's face and the biometric sensor measures their heart rate. From this data, facial expressions and heart rate are analyzed to determine their emotional state.

[1019] Input: Camera footage, biometric sensor data.

[1020] Output: Identified emotional information (e.g., facial expression anxiety, heart rate 100 bpm).

[1021] Step 4:

[1022] The location information and emotion information acquired by the device are sent to the server via an HTTP request.

[1023] Specific operation: Data including location information and emotion information is packaged in the body of an HTTP request and sent to the server.

[1024] Input: location and emotion.

[1025] Output: The request sent to the server.

[1026] Step 5:

[1027] The server receives an HTTP request from the mobile device.

[1028] What it does: The server listens for HTTP requests and analyzes the received data to extract location and emotion information.

[1029] Input: HTTP request.

[1030] Output: Extracted location and emotion information.

[1031] Step 6:

[1032] The server sends location and emotion information to the cloud AI model.

[1033] Specific operation: Convert location information and emotion information into JSON format and send a request to the cloud AI model.

[1034] Input: Extracted location and emotion information.

[1035] Output: The request sent to the cloud AI model.

[1036] Step 7:

[1037] A cloud AI model receives the request and performs analysis by integrating historical incident data with real-time information.

[1038] How it works: The cloud AI model searches past incident data, retrieves real-time news and police alerts, and uses this data to match location and emotion information.

[1039] Inputs: location information, emotion information, historical incident data, real-time information.

[1040] Output: Consolidated analysis results.

[1041] Step 8:

[1042] Cloud AI models generate optimal courses of action.

[1043] Specific actions: Based on the analysis results, specific guidelines for the user to take are generated.

[1044] Input: Consolidated analysis results.

[1045] Output: Generated course of action (e.g., Take shelter in a nearby building now and avoid going outside until it is safe).

[1046] Step 9:

[1047] The emotion engine generates a support message based on the emotional information.

[1048] How it works: The emotion engine analyzes the user's emotional state and creates a reassuring, supportive message.

[1049] Input: Emotion information.

[1050] Output: Encouraging message (e.g., "Don't worry, just follow the instructions.").

[1051] Step 10:

[1052] The server sends guidelines and support messages to the user terminal.

[1053] Specific operation: Compile a code of conduct and a message of encouragement and send it to the user's device as an HTTP response.

[1054] Input: Guidelines for action, messages of encouragement.

[1055] Output: HTTP response to the user's device.

[1056] Step 11:

[1057] The user's device receives the guidelines and support messages and displays them on the app.

[1058] Specific operation: Analyze the received data and display guidelines and messages of encouragement on the screen.

[1059] Input: The action plan and encouragement message received as an HTTP response.

[1060] Output: The information displayed on the app screen.

[1061] Through the above steps, the system provides users with appropriate guidelines for action and supportive messages tailored to their emotions in real time, providing a sense of psychological security while ensuring their safety.

[1062] (Application example 2)

[1063] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1064] Conventional security systems have difficulty providing appropriate guidance in real time when users encounter dangerous situations. Furthermore, they do not provide guidance that reflects the user's emotional state, preventing users from feeling psychologically secure. Furthermore, there is a lack of technology to display instructions in a format suitable for wearable devices such as smart glasses.

[1065] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring current location information from the user terminal, means for analyzing emotional information acquired from the user terminal, means for receiving location information and emotional information transmitted from the user terminal, means for generating an optimal guideline of action using past incident data and real-time information, means for adjusting the generated guideline of action based on the emotional state, and means for transmitting the generated guideline of action to the user terminal. This enables the user to respond quickly to dangerous situations and act safely and with peace of mind.

[1066] A "user terminal" is a portable information terminal that a user can carry around and that is capable of acquiring and transmitting location information and emotion information.

[1067] "Location information" is data indicating the current latitude and longitude of the user terminal.

[1068] "Emotional information" is information obtained by analyzing the user's physiological and psychological state, such as facial expressions and heart rate.

[1069] The "analyzing means" refers to a technical means for acquiring and analyzing the user's emotional information to identify the user's emotional state.

[1070] "Past incident data" is a database that compiles information about incidents that have occurred in the past.

[1071] "Real-time information" means up-to-date information about ongoing events or situations.

[1072] The "means for generating" refers to the technical means for generating optimal guidelines for action based on the acquired location information and emotional information.

[1073] "Adjustment means" refers to the ability to appropriately change or complement already generated courses of action based on emotional information.

[1074] The "action guidelines" are specific activity guides for users to act safely.

[1075] "Displaying on a display" means visually presenting the generated action guidelines on the screen of the user terminal.

[1076] The present invention provides a system that, when a user encounters a dangerous situation, provides an optimal course of action in real time and adjusts the course of action by analyzing the user's emotional state. A specific embodiment of this system will be described below.

[1077] System configuration

[1078] This system is composed of a user terminal, a server, a cloud AI model, and an emotion engine.

[1079] User terminal

[1080] The user terminal is a wearable device such as smart glasses. This terminal is equipped with a GPS function, a camera, and a biometric sensor, and can acquire current location information and emotional information in real time. For example, the GPS function can be used to acquire latitude and longitude, and the camera and biometric sensor can be used to analyze facial expressions and heart rate to identify emotional information.

[1081] server

[1082] The server receives location and emotional information from the user's device and generates optimal guidelines based on the cloud AI model. The server sends this data to the cloud in JSON format and compares it with past incident data and real-time information to generate guidelines. It also has the function of adjusting guidelines based on the user's emotional information.

[1083] Cloud AI Model

[1084] The cloud AI model integrates past incident data and real-time information using location and emotion information received from the server. Based on the analysis results, it generates specific guidelines for users to act safely. For example, if a user is clearly feeling anxious or scared, it generates guidelines such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[1085] Emotion Engine

[1086] The emotion engine analyzes the user's emotional state based on physiological data such as facial expressions and heart rate obtained from the user's device's camera and biometric sensors. The generated emotional information is sent to a cloud AI model and reflected in the generation and adjustment of action guidelines. The emotion engine can also provide encouraging messages that reflect the user's psychological state.

[1087] Specific examples

[1088] Suppose a user is attending an event in Shinjuku when a commotion breaks out in the crowd, causing their heart rate to rise and anxiety to rise. The smart glasses worn by the user use their GPS to obtain their current location and their camera and heart rate sensor to analyze their emotional information. This data is sent to a server, where a cloud AI model and emotion engine generate a guideline of action, such as "Please evacuate to a nearby building," and a message of encouragement, such as "It's okay, please act with peace of mind." These messages are displayed in real time on the smart glasses' display, and the user follows the instructions to take safe actions.

[1089] Prompt Sentence Examples

[1090] Latitude: 35.6895

[1091] Longitude: 139.6917

[1092] Emotional state: anxiety

[1093] Heart rate: 110

[1094] In this way, the present invention can prevent users from encountering danger, encourage safe behavior, and provide a sense of psychological security.

[1095] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1096] Step 1:

[1097] The user's device acquires current location information and collects emotional information. Specifically, it uses the device's GPS function to acquire latitude and longitude data, and uses the camera and biometric sensors to acquire physiological data such as the user's facial expressions and heart rate. This allows the user's device to obtain current location information (latitude and longitude) and emotional information (facial expression analysis results, heart rate).

[1098] Input: None

[1099] Output: Location information (latitude and longitude), emotional information (facial expression analysis results, heart rate)

[1100] Step 2:

[1101] The user device sends the acquired location information and emotion information to the server. The device converts this data into JSON format and sends it to the server via an HTTP request. At this time, data in the form of a prompt message is generated.

[1102] Input: Location information (latitude and longitude), emotional information (facial expression analysis results, heart rate)

[1103] Output: HTTP request to the server (data in JSON format)

[1104] Step 3:

[1105] The server receives the data sent from the user terminal and sends it to the cloud AI model. The server generates a prompt sentence including the received location information and emotion information and sends a request to the cloud AI model.

[1106] Input: HTTP request from the user device (JSON format data)

[1107] Output: Request to cloud AI model (prompt sentence including location and emotion information)

[1108] Step 4:

[1109] Based on the data received, the cloud AI model compares past incident data with real-time information to generate optimal guidelines for action. The cloud AI model integrates and analyzes location information and emotional information to generate specific guidelines for action that will enable users to act safely.

[1110] Input: Request to cloud AI model (prompt sentence)

[1111] Output: Generated course of action

[1112] Step 5:

[1113] The emotion engine adjusts the course of action based on the user's emotional information and adds a supportive message as needed. The emotion engine analyzes the received course of action and includes an optimal supportive message to alleviate the user's anxiety and fear.

[1114] Input: Generated course of action

[1115] Output: Coordinated course of action and encouraging message

[1116] Step 6:

[1117] The server sends the adjusted action plan to the user terminal. The server generates data including the adjusted action plan and a support message, and sends it to the user terminal as an HTTP response.

[1118] Input: Coordinated course of action and message of encouragement

[1119] Output: HTTP response to the user's device (adjusted guidelines and support message)

[1120] Step 7:

[1121] The user device processes the data received from the server and displays it on the smart glasses display. The device analyzes the received guidelines and support messages and visually presents them on the display.

[1122] Input: HTTP response from the server (adjusted course of action and support message)

[1123] Output: Guidelines and support messages displayed on the screen

[1124] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1125] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1126] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1127] [Fourth embodiment]

[1128] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1129] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1130] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1131] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1132] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1133] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1134] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1135] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1136] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1137] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1138] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1139] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1140] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1141] The present invention provides a system that provides a user with an appropriate course of action in real time when the user encounters a violent crime. Specific embodiments of the system will be described below.

[1142] This system consists of a user device, a server, and a cloud AI model. Users access the system using a mobile app and send their location information to the system, allowing them to receive the optimal course of action for their current location.

[1143] User terminal

[1144] A user first launches a mobile app. The user's device uses its built-in GPS to obtain its current location. For example, if the user is in Shibuya, Tokyo, the device obtains the user's latitude and longitude. This location information is sent to the server via an HTTP request. The server processes this request and determines the user's current location.

[1145] server

[1146] The server receives location information sent from the user's device. Based on the received location information, the server sends a request to the cloud AI model, asking it to compare past incident data with real-time information and generate an optimal course of action. Specifically, the server sends JSON data containing location information to the cloud AI model and receives a course of action from the cloud AI model.

[1147] Cloud AI Model

[1148] The cloud AI model integrates and analyzes past incident data and real-time information based on location information received from the server. Past incident data includes records and trends of crimes that have occurred in the past. Real-time information incorporates current news and breaking reports from the police. Based on this information, the cloud AI model generates specific guidelines for users to act safely. For example, guidelines such as "Evacuate to the nearest building and do not move until safety is confirmed."

[1149] Sending guidelines to user terminals

[1150] The server sends the action guidelines received from the cloud AI model to the user's device. The user's device receives this information and displays it on the mobile app. The user can confirm the action guidelines and act safely.

[1151] Specific examples

[1152] A concrete example of this system is shown below. For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation has occurred. When the user launches the app, the device detects their current location and sends that information to the server. The server receives the location information and queries the cloud AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The server receives this information and sends it to the user's device. The user can check the specific course of action on the app and take safe actions.

[1153] As described above, the present invention enables a user to take appropriate action even when encountering a sudden violent incident, thereby strengthening self-defense.

[1154] The processing flow will be explained below.

[1155] Step 1:

[1156] The user launches the mobile app, which starts the application and displays the interface for further action.

[1157] Step 2:

[1158] The device obtains its current location using its built-in GPS: the device's GPS sensor detects latitude and longitude and converts this data into a format that can be used within applications.

[1159] Step 3:

[1160] The device sends the acquired location information to the server using an HTTP POST request, and the location data (latitude and longitude) is sent to the server in JSON format.

[1161] Step 4:

[1162] The server receives the location information sent from the device, analyzes the HTTP request, and extracts the latitude and longitude data included.

[1163] Step 5:

[1164] The server creates a request to send the received location information to the cloud AI model. This request is also in JSON format and is sent to the cloud AI model along with the location information.

[1165] Step 6:

[1166] The cloud AI model integrates and analyzes past incident data and real-time information based on location information received from the server, searching for relevant records from the past incident database and simultaneously obtaining real-time news information and police bulletins.

[1167] Step 7:

[1168] The cloud AI model analyzes the acquired data and generates optimal guidelines for action, such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[1169] Step 8:

[1170] The cloud AI model returns the generated action guidelines to the server, which receives this information and prepares it to send as a response to the user device.

[1171] Step 9:

[1172] The server sends the action guidelines received from the cloud AI model to the user's device, and the action guidelines data is returned to the user's device as an HTTP response.

[1173] Step 10:

[1174] The terminal displays the guidelines received from the server, and the application displays the specific guidelines in a format that the user can immediately check.

[1175] Step 11:

[1176] The user checks the display on the device and acts according to the provided guidelines, such as moving to the nearest evacuation site, and takes self-protective action according to specific instructions.

[1177] Through these steps, users can obtain appropriate guidelines for action in real time and act safely even when they encounter a sudden violent incident.

[1178] Example 1

[1179] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1180] In modern society, when a sudden violent crime occurs, ordinary citizens are required to take swift and appropriate action to ensure safety. However, current systems have difficulty providing real-time information for users to take appropriate action, and safety is not sufficiently ensured. Therefore, a system is needed that allows users to receive optimal guidelines for action in real time based on their current location information.

[1181] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1182] In this invention, the server includes means for receiving current location information from the user terminal, means for sending a prompt message including the location information to the generative AI model, means for generating an optimal action guideline using past incident data and real-time information by the generative AI model, and means for sending the generated action guideline to the user terminal, thereby enabling the user to receive a guideline for taking safe actions in real time based on their current location information.

[1183] A "user terminal" is a communication device that is directly operated by a user, and is equipped with means for acquiring location information and performing data communication with a server.

[1184] "Location information" is data indicating the user's current geographical location, and is expressed as latitude and longitude.

[1185] "Means for receiving" refers to a device or program that has the function of receiving data or information from a sender.

[1186] A "server" is a computer system that communicates with user terminals via a network and receives, processes, and transmits data.

[1187] A "generative AI model" is an artificial intelligence system that uses past incident data and real-time information to analyze and judge specific issues and generate optimal guidelines for action.

[1188] A "prompt sentence" is text data that is input into a generative AI model and includes location information and context.

[1189] "Guidelines for Action" are messages that contain specific instructions or advice for users to act safely in specific situations.

[1190] A "transmitting means" is a device or program that has the function of sending data or information to other devices or computer systems.

[1191] "Past incident data" is a database that collects information about crimes and accidents that have occurred in the past.

[1192] "Real-time information" refers to the latest information, such as ongoing events and breaking news.

[1193] "JSON format" is an abbreviation for JavaScript Object Notation, and is a lightweight data description format used for exchanging and storing data.

[1194] The system of the present invention provides appropriate action guidelines in real time when a user encounters a violent crime. This system is composed of a user terminal, a server, and a cloud AI model. Specific embodiments of each component are described below.

[1195] User terminal

[1196] A user first launches a mobile app. This app runs on a mobile device such as a smartphone or tablet. The user device obtains its current location using its built-in GPS function. For example, if the user is in Shibuya Ward, Tokyo, the device obtains the user's latitude and longitude information. This location information is sent to the server via an HTTP request. The location information is packaged in JSON format.

[1197] server

[1198] The server receives the location information sent from the user device and sends a request to the generative AI model. When the location information arrives at the server, it is extracted and recorded in a log. The server then sends a prompt message containing the location information to the generative AI model. This prompt message instructs the AI ​​model to generate the optimal course of action using past incident data and real-time information. As a concrete example, the prompt message has the following format:

[1199] Location: Latitude 35.658581, Longitude 139.745433

[1200] Context: There is currently a hostage situation occurring in Shibuya, Tokyo. Please provide guidelines for nearby residents to act safely.

[1201] Cloud AI Model

[1202] The cloud AI model integrates and analyzes past incident data and real-time information based on location information and prompts received from the server. This AI model incorporates records of past crimes, current news, and breaking news from the police to provide the user with the optimal course of action. For example, the cloud AI model generates the following course of action:

[1203] Action Guidelines: A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed.

[1204] The generated guidelines are sent back to the server in JSON format.

[1205] Sending and displaying guidance

[1206] The server sends the guidelines received from the cloud AI model to the user's device. The device receives this information and displays it on the mobile app. The user can check the guidelines displayed on the app screen and take safe actions based on them.

[1207] Specific examples

[1208] For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation is occurring. When the user launches the app, the device detects their current location and sends that information to the server. The server receives the location information and queries the generative AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The server sends this information to the user's device, and the user can check the specific course of action on the app.

[1209] The present invention enables a user to take appropriate action in real time even when encountering a sudden violent incident, thereby strengthening self-defense.

[1210] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1211] Step 1:

[1212] A user launches a mobile app. When the user taps the app icon on their smartphone, the app launches. At this point, the app turns on the GPS function in the background and obtains the current location information (latitude and longitude). The input is the user's operation (launching the app), and the output is the current location information. Specifically, location information such as latitude 35.658581 and longitude 139.745433 is extracted.

[1213] Step 2:

[1214] The device sends the acquired current location information to the server via an HTTP request. The device converts the acquired latitude and longitude information into JSON format and sends it to the server as the payload of the HTTP request. The input is the location information obtained from GPS, and the output is an HTTP request in JSON format. Specifically, the following JSON data is generated:

[1215] {

[1216] "latitude": 35.658581,

[1217] "longitude": 139.745433

[1218] }

[1219] Step 3:

[1220] The server processes the received HTTP request and extracts the location information. The server receives the request at the " / location" endpoint and parses the latitude and longitude from the received data. The input is the JSON data sent from the device, and the output is the extracted location information. The specific operation is recorded in the log as "Location information received: latitude 35.658581, longitude 139.745433."

[1221] Step 4:

[1222] The server creates a request to the generative AI model based on the extracted location information. The server generates a prompt sentence including the location information and sends it to the cloud AI model. The input is the extracted location information, and the output is a prompt sentence for the AI ​​model. Specifically, the following prompt sentence is generated:

[1223] Location: Latitude 35.658581, Longitude 139.745433

[1224] Context: There is currently a hostage situation occurring in Shibuya, Tokyo. Please provide guidelines for nearby residents to act safely.

[1225] Step 5:

[1226] The cloud AI model receives the prompt text and performs analysis based on past incident data and real-time information. The input is the prompt text from the server, accumulated incident data, and real-time information, and the output is specific guidelines for action. Analysis is performed and specific guidelines for action that the user should take are generated. The specific actions that are generated include the following:

[1227] Action Guidelines: A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed.

[1228] Step 6:

[1229] The server checks the guidelines received from the cloud AI model and sends them to the user device. The input is the guidelines from the AI ​​model, and the output is an HTTP response to be sent to the user device. Specifically, JSON data containing the guidelines is generated and sent to the user device.

[1230] Step 7:

[1231] The user device analyzes the guidelines received from the server and displays them on the mobile app. The input is the JSON data of the guidelines from the server, and the output is the display on the app screen. Specifically, the app screen displays a message saying, "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed."

[1232] The above is the specific processing flow of the program of this system.

[1233] (Application example 1)

[1234] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1235] The lack of a system that allows autonomous vehicles to take prompt and appropriate action when they encounter a sudden incident or danger is a major issue. In particular, providing and implementing real-time information is difficult for autonomous vehicles, making it difficult to ensure user safety.

[1236] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1237] In this invention, the server includes means for acquiring current location information from the user terminal, means for receiving the location information transmitted from the user terminal, means for generating an optimal course of action using past incident data and real-time information, means for transmitting the generated course of action to the user terminal, and means for executing the course of action in real time in the autonomous vehicle based on the acquired location information, thereby enabling the autonomous vehicle to quickly and appropriately avoid danger and take safe actions.

[1238] A "user terminal" is an electronic device carried by a user that acquires current location information and transmits it to the system.

[1239] "Location information" is information including the current latitude and longitude of the user terminal.

[1240] A "server" is a computer system that receives location information sent from a user device and communicates with the cloud AI model.

[1241] "Past incident data" refers to data that includes records of crimes and accidents that have occurred in the past and their trends.

[1242] "Real-time information" means information that is updated immediately, including current news and breaking police reports.

[1243] "Guidelines for action" are specific instructions and advice for users to act safely.

[1244] An "autonomous vehicle" is a vehicle that can drive autonomously using AI and sensor technology.

[1245] "Autonomous driving algorithms" are the programs and procedures that enable an autonomous vehicle to operate safely and efficiently.

[1246] A "cloud AI model" is an artificial intelligence model that exists on the cloud and analyzes past incident data and real-time information to generate guidelines for action.

[1247] The "means for transmitting to the user terminal" is a function by which the server transmits the generated action guidelines to the user terminal via communication.

[1248] "Displayed in real time" means that the course of action is displayed immediately on the user terminal.

[1249] "Means for executing action guidelines in real time" refers to a function that enables an autonomous vehicle to immediately take appropriate action based on the generated action guidelines.

[1250] The present invention provides a system for providing an appropriate course of action in real time when a user encounters a violent crime, and is applied to an autonomous vehicle. A specific embodiment of this system will be described below.

[1251] System configuration

[1252] This system consists of the following hardware and software:

[1253] User terminal: An electronic device carried by a user that has a built-in GPS module for acquiring location information.

[1254] Server: A computer system that receives location information sent from user devices and communicates with the cloud AI model.

[1255] Cloud AI model: An artificial intelligence model that analyzes past incident data and real-time information to generate action guidelines.

[1256] Self-driving vehicle: A vehicle that can drive autonomously using AI and sensor technology.

[1257] Overall system operation

[1258] 1. Acquisition and transmission of location information

[1259] The user device uses its built-in GPS module to obtain its current location. For example, if the user is in a particular city, it obtains the latitude and longitude information. This location information is then sent to the server via an HTTP request.

[1260] 2. Receiving and analyzing location information

[1261] The server receives location information sent from the user's device and sends a request to the cloud AI model based on the received location information, asking it to integrate past incident data with real-time information to generate the optimal course of action.

[1262] 3. Creating guidelines for action

[1263] The cloud AI model integrates and analyzes past incident data and real-time information based on location information received from the server. Past incident data includes records and trends of past crimes, while real-time information incorporates current news and breaking reports from the police. Based on this information, the cloud AI model generates specific guidelines for users to act safely.

[1264] 4. Implementation of the Code of Conduct

[1265] The server sends the action guidelines received from the cloud AI model to the user's device and simultaneously to the self-driving vehicle. The user's device receives this information and displays it in real time. The self-driving vehicle immediately and automatically executes the optimal avoidance action (e.g., selecting a detour or moving to a safe stopping location) based on the received action guidelines.

[1266] Specific examples

[1267] For example, if a hostage situation occurs while an autonomous vehicle is driving within a city, the system operates as follows: The user's device detects its current location and sends that information to the server. The server then queries the cloud AI model based on the location information and generates a course of action: "A hostage situation is occurring within the city. Please evacuate to the nearest safe stopping area immediately and wait until safety is confirmed." Based on this, the autonomous vehicle immediately moves to a safe area, and the course of action is displayed in real time on the user's device.

[1268] Prompt Sentence Examples

[1269] An example of a prompt sent to the cloud AI model when an incident occurs is, "An emergency has occurred at the current location. Please generate the optimal course of action for the autonomous vehicle."

[1270] In this way, the system of the present invention enables autonomous vehicles to take safe and appropriate actions in real time, ensuring user safety.

[1271] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1272] Step 1:

[1273] The user device obtains its current location information using the built-in GPS module. In this process, the user device generates latitude and longitude data. Specifically, the GPS module measures the current location of the user device and prepares the location information as data. The input is a GPS signal, and the output is latitude and longitude location data.

[1274] Step 2:

[1275] The user device sends the acquired location information to the server. This transmission is performed via an HTTP request. The location information data is processed and sent to the server. Specifically, the location information is converted into JSON format and sent to the server via the HTTP protocol. The input is the location information data, and the output is the location information sent to the server.

[1276] Step 3:

[1277] The server receives the location information sent from the user device. In this process, the server obtains the location information and uses it for the next step of processing. Specifically, the server analyzes the received HTTP request and extracts the location information. The input is the HTTP request, and the output is location information data.

[1278] Step 4:

[1279] The server sends a request to the cloud AI model based on the received location information. This request includes location information, which the cloud AI model uses to analyze past incident data and real-time information. Specifically, the server sends the location information in JSON format to the cloud AI model. The input is location data, and the output is a request to the cloud AI model.

[1280] Step 5:

[1281] The cloud AI model integrates and analyzes past incident data and real-time information based on the received location information to generate optimal action guidelines. Specifically, the cloud AI model inputs location information and analyzes a database of past incidents and current real-time information. The inputs are location information, past incident data, and real-time information, and the output is optimal action guidelines.

[1282] Step 6:

[1283] The server sends the action guidelines received from the cloud AI model to the user device. This transmission allows the user device to obtain the action guidelines. Specifically, the server sends the action guideline data to the user device in JSON format. The input is the action guideline data, and the output is the action guidelines sent to the user device.

[1284] Step 7:

[1285] The user device displays the received action guidelines in real time, allowing the user to take appropriate action. Specifically, the user device analyzes the action guidelines and displays them on the screen. The input is the action guidelines data, and the output is the displayed action guidelines.

[1286] Step 8:

[1287] The server sends the action guidelines received from the cloud AI model to the autonomous vehicle. This transmission allows the autonomous vehicle to obtain the action guidelines. Specifically, the server sends the action guidelines data to the autonomous vehicle in JSON format. The input is the action guidelines data, and the output is the action guidelines sent to the autonomous vehicle.

[1288] Step 9:

[1289] The autonomous vehicle executes optimal avoidance actions in real time based on the received action guidelines. Examples include selecting a detour route or moving to a safe stopping location. Specifically, the autonomous driving algorithm analyzes the action guidelines and controls the vehicle's operation in accordance with them. The input is the action guidelines data, and the output is the executed avoidance action.

[1290] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1291] The present invention provides a system that provides an appropriate course of action in real time when a user encounters a violent crime, and further adjusts the course of action by recognizing the user's emotions. A specific embodiment of this system will be described below.

[1292] This system is composed of a user device, a server, a cloud AI model, and an emotion engine. Users access the system using a mobile app and send their location and emotion information to the system. This allows users to receive the optimal course of action based on their current location and emotional state.

[1293] User terminal

[1294] A user first launches a mobile app. The user's device acquires current location information using its built-in GPS function. At the same time, the device's camera and biometric sensors are used to acquire the user's emotional information. For example, if the user is in Shibuya Ward, Tokyo, the device acquires the user's latitude and longitude information, and then analyzes the user's facial expressions and heart rate to determine their emotional state. This location information and emotional information are sent to the server via an HTTP request. The server processes this request and determines the user's current location and emotional state.

[1295] server

[1296] The server receives location information and emotion information sent from the user's device. Based on the received data, the server sends a request to the cloud AI model, asking it to compare past incident data with real-time information and generate an optimal course of action. Specifically, the server sends JSON data containing location information and emotion information to the cloud AI model and receives a course of action from the cloud AI model.

[1297] Cloud AI Model

[1298] The cloud AI model integrates and analyzes past incident data and real-time information based on location and emotion information received from the server. It searches for relevant records from the past incident database and simultaneously obtains real-time news information and police bulletins. Based on this information, the cloud AI model generates specific guidelines for users to act safely. For example, if a user feels anxious, it will prescribe specific actions for the user, such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[1299] Emotion Engine

[1300] Furthermore, the emotion engine analyzes the user's facial expressions and physiological data (heart rate, breathing rate, etc.) to identify the user's emotional state. This emotional information is reflected in the generation of action guidelines by the cloud AI model. For example, if the user is feeling fear or tension, the emotion engine will detect this and generate action guidelines that include a message of encouragement to reassure the user.

[1301] Sending guidelines to user terminals

[1302] The server sends the guidelines received from the cloud AI model and emotion engine to the user's device. The user's device receives this information and displays it on the mobile app. The user can confirm the guidelines and act safely.

[1303] Specific examples

[1304] A concrete example of this system is shown below. For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation has occurred. When the user launches the app, the device detects their current location and analyzes the user's emotions using the emotion engine. The server receives the location and emotion information and queries the cloud AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The emotion engine senses the user's anxiety and adds a message of encouragement: "It's okay, please rest assured and follow instructions." The server receives this information and sends it to the user's device. The user can check the specific course of action and the encouragement message on the app and take safe actions.

[1305] As described above, the present invention enables users to take appropriate actions when they encounter a sudden violent incident, thereby strengthening their self-defense. Furthermore, by providing guidelines that reflect the user's emotional state, it is possible to provide a sense of psychological security.

[1306] The processing flow will be explained below.

[1307] Step 1:

[1308] The user launches the mobile app, which starts the application and displays the interface for further action.

[1309] Step 2:

[1310] The device obtains its current location using its built-in GPS: the device's GPS sensor detects latitude and longitude and converts this data into a format that can be used within applications.

[1311] Step 3:

[1312] The device uses a camera and biometric sensors to acquire information about the user's emotions. For example, the camera recognizes the user's face and analyzes their facial expressions. Biometric sensors also acquire heart rate and breathing rate to determine the user's emotional state.

[1313] Step 4:

[1314] The device sends the acquired location information and emotion information to the server using an HTTP POST request, and the location information and emotion information are sent to the server in JSON format.

[1315] Step 5:

[1316] The server receives the location information and emotion information sent from the device, analyzes the HTTP request, and extracts the included latitude and longitude data and emotion information.

[1317] Step 6:

[1318] The server creates a request to send the received location and emotion information to the cloud AI model. This request is also in JSON format and is sent to the cloud AI model along with the location and emotion information.

[1319] Step 7:

[1320] The cloud-based AI model integrates and analyzes past incident data and real-time information based on location and emotion information received from the server, searching for relevant records from the past incident database and simultaneously obtaining real-time news information and police bulletins.

[1321] Step 8:

[1322] The cloud AI model analyzes the acquired data and generates optimal guidelines for action, such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[1323] Step 9:

[1324] The emotion engine shares the user's emotional information and adds a complementary message of encouragement to the generated course of action. For example, if the user's emotional state is judged to be anxious, the engine adds the message "Don't worry, just follow the instructions."

[1325] Step 10:

[1326] The cloud AI model and emotion engine then send the generated guidelines and support messages back to the server, which receives this information and prepares it for transmission to the user's device as a response.

[1327] Step 11:

[1328] The server sends the guidelines and support messages received from the cloud AI model and emotion engine to the user's device. The guidelines and support messages are returned to the user's device as an HTTP response.

[1329] Step 12:

[1330] The device displays the guidelines and support messages received from the server. The specific guidelines and support messages are displayed on the application in a format that the user can immediately check.

[1331] Step 13:

[1332] The user checks the display on the device and acts in accordance with the provided guidelines and support messages, for example, by taking self-protective actions according to specific instructions, such as moving to the nearest evacuation site.

[1333] Through these steps, users can obtain appropriate guidelines for action in real time and act safely even when they encounter a sudden violent incident. In addition, the provision of supportive messages tailored to the user's emotional state increases their psychological sense of security.

[1334] Example 2

[1335] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1336] In modern society, the risk of users encountering sudden violent crimes is increasing, necessitating support for taking prompt and appropriate action. However, conventional systems only grasp the user's current location information and do not provide support based on the user's emotional state. This makes it difficult for users to take appropriate action and fails to provide a sense of psychological security. To address these issues, the present invention aims to utilize data including the user's emotional information to provide appropriate guidelines for action and support messages in real time.

[1337] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1338] In this invention, the server includes means for acquiring current location information from the mobile device, means for acquiring user emotional information from the mobile device, means for transmitting the acquired location information and emotional information to a cloud AI model, means for the cloud AI model to generate an optimal course of action using past incident data and real-time information, and means for the emotion engine to analyze the emotional information and add a support message to the course of action. This allows the user to obtain an appropriate course of action based on the current location information and emotional information, enabling them to act safely. In addition, receiving a support message tailored to the user's emotional state also provides a sense of psychological security.

[1339] A "mobile terminal" is an electronic device that a user carries and uses, and includes devices such as smartphones and tablets.

[1340] "Current location information" is latitude and longitude data obtained using the built-in GPS function of the mobile device, and is information that indicates the user's current location.

[1341] "Emotion information" is information indicating the emotional state of the user that is obtained by analyzing physiological data such as facial expressions and heart rate.

[1342] The "server" is a central processing unit that receives location and emotion information sent from mobile devices and sends the data to the cloud AI model.

[1343] The "Cloud AI Model" is an artificial intelligence model with an algorithm that compares past incident data and real-time information based on location information and emotional information received from a server to generate optimal guidelines for action.

[1344] "Past incident data" is a database containing information on incidents that have occurred in the past, and is data that records information such as the type of incident, the location of the incident, and how it was handled.

[1345] "Real-time information" refers to information that allows you to understand the latest situation, such as ongoing incidents and police bulletins.

[1346] "Guidelines for action" are specific instructions for actions that users should take, generated by the cloud AI model, and include specific steps and advice for ensuring safety.

[1347] The "Emotion Engine" is a system that analyzes the user's facial expressions and physiological data to identify emotional information, and adds encouraging messages to the guidelines for action generated by the cloud AI model.

[1348] A "support message" is a message that is generated by the emotion engine based on the user's emotional state and is intended to give a sense of security.

[1349] An "HTTP request" is a protocol request used to communicate data between a server and a mobile device over the Internet.

[1350] "JSON data" is data expressed in JavaScript Object Notation format, a format used for exchanging structured data.

[1351] The present invention provides a system that provides appropriate guidelines for action in real time when a user encounters a sudden violent incident, and further adjusts the guidelines of action by recognizing the user's emotions. How to specifically implement the present invention will be described below.

[1352] System Overview

[1353] This system is composed of a mobile device, a server, a cloud AI model, and an emotion engine. Users access the system using a mobile app and send their location and emotion information to the system, allowing them to receive the optimal course of action based on their current situation.

[1354] Mobile devices

[1355] A user first launches a mobile app. The mobile device acquires its current location using its built-in GPS. At the same time, the device's camera and biometric sensors are used to acquire the user's emotional information. The location information includes latitude and longitude, and the emotional information includes the user's facial expression and heart rate. This information is then sent to the server via an HTTP request.

[1356] server

[1357] The server receives location information and emotion information sent from the mobile device. Based on the received data, the server sends a request to the cloud AI model to compare past incident data with real-time information and generate an optimal course of action. Specifically, the server sends data including location information and emotion information to the cloud AI model and receives a course of action from the cloud AI model.

[1358] Cloud AI Model

[1359] The cloud AI model integrates and analyzes past incident data and real-time information based on location and emotion information received from the server. It searches for relevant records from the past incident database and simultaneously obtains real-time news information and police bulletins. Based on this information, the cloud AI model generates specific guidelines for users to act safely.

[1360] Emotion Engine

[1361] The emotion engine analyzes the user's facial expressions and physiological data (heart rate, breathing rate, etc.) to identify the user's emotional state. This emotional information is reflected in the generation of action guidelines by the cloud AI model. If the user is feeling fear or tension, the emotion engine will detect this and generate action guidelines that include a message of encouragement to reassure the user.

[1362] Sending guidelines to user terminals

[1363] The server sends the guidelines received from the cloud AI model and emotion engine to the user's device. The user's device receives this information and displays it on the mobile app. The user can confirm the guidelines and act safely.

[1364] Specific examples

[1365] For example, suppose a user is in Shibuya Ward, Tokyo, and a hostage situation is occurring. When the user launches the app, the device detects their current location and analyzes the user's emotions using the emotion engine. The server receives the location and emotion information and queries the cloud AI model. Based on past incident data and real-time information, the cloud AI model generates a course of action: "A hostage situation is occurring in Shibuya Ward. Please evacuate to a nearby building immediately and avoid going outside until safety is confirmed." The emotion engine senses the user's anxiety and adds a message of encouragement: "It's okay, please don't worry and follow instructions." The server sends this information to the user's device, and the user can view the specific course of action and the encouragement message on the app and take safe actions.

[1366] An example of a prompt is as follows:

[1367] User's current location: Shibuya-ku, Tokyo

[1368] User's emotional state: Anxiety

[1369] Past incident data: Shibuya Ward hostage incident

[1370] Real-time information: Police news: "Hostage incident occurring in Shibuya Ward"

[1371] Generate guidelines: Give specific instructions for safe behavior that users should take.

[1372] The present invention enables users to take appropriate action when they encounter a sudden violent incident, thereby strengthening their self-defense. Furthermore, by providing guidelines that reflect the user's emotional state, it is possible to provide a sense of psychological security.

[1373] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1374] Step 1:

[1375] The user launches the mobile app.

[1376] Specific actions: Tap the app icon on your smartphone's home screen to launch the app.

[1377] Input: User action (tapping the app icon).

[1378] Output: App launch.

[1379] Step 2:

[1380] The device obtains the current location.

[1381] Specific operation: Enables the device's built-in GPS function and obtains the user's current location in latitude and longitude.

[1382] Input: Location data from a GPS sensor.

[1383] Output: The location information obtained (e.g., latitude 35.659, longitude 139.702).

[1384] Step 3:

[1385] The device collects emotional information.

[1386] How it works: The camera takes a picture of the user's face and the biometric sensor measures their heart rate. From this data, facial expressions and heart rate are analyzed to determine their emotional state.

[1387] Input: Camera footage, biometric sensor data.

[1388] Output: Identified emotional information (e.g., facial expression anxiety, heart rate 100 bpm).

[1389] Step 4:

[1390] The location information and emotion information acquired by the device are sent to the server via an HTTP request.

[1391] Specific operation: Data including location information and emotion information is packaged in the body of an HTTP request and sent to the server.

[1392] Input: location and emotion.

[1393] Output: The request sent to the server.

[1394] Step 5:

[1395] The server receives an HTTP request from the mobile device.

[1396] What it does: The server listens for HTTP requests and analyzes the received data to extract location and emotion information.

[1397] Input: HTTP request.

[1398] Output: Extracted location and emotion information.

[1399] Step 6:

[1400] The server sends location and emotion information to the cloud AI model.

[1401] Specific operation: Convert location information and emotion information into JSON format and send a request to the cloud AI model.

[1402] Input: Extracted location and emotion information.

[1403] Output: The request sent to the cloud AI model.

[1404] Step 7:

[1405] A cloud AI model receives the request and performs analysis by integrating historical incident data with real-time information.

[1406] How it works: The cloud AI model searches past incident data, retrieves real-time news and police alerts, and uses this data to match location and emotion information.

[1407] Inputs: location information, emotion information, historical incident data, real-time information.

[1408] Output: Consolidated analysis results.

[1409] Step 8:

[1410] Cloud AI models generate optimal courses of action.

[1411] Specific actions: Based on the analysis results, specific guidelines for the user to take are generated.

[1412] Input: Consolidated analysis results.

[1413] Output: Generated course of action (e.g., Take shelter in a nearby building now and avoid going outside until it is safe).

[1414] Step 9:

[1415] The emotion engine generates a support message based on the emotional information.

[1416] How it works: The emotion engine analyzes the user's emotional state and creates a reassuring, supportive message.

[1417] Input: Emotion information.

[1418] Output: Encouraging message (e.g., "Don't worry, just follow the instructions.").

[1419] Step 10:

[1420] The server sends guidelines and support messages to the user terminal.

[1421] Specific operation: Compile a code of conduct and a message of encouragement and send it to the user's device as an HTTP response.

[1422] Input: Guidelines for action, messages of encouragement.

[1423] Output: HTTP response to the user's device.

[1424] Step 11:

[1425] The user's device receives the guidelines and support messages and displays them on the app.

[1426] Specific operation: Analyze the received data and display guidelines and messages of encouragement on the screen.

[1427] Input: The action plan and encouragement message received as an HTTP response.

[1428] Output: The information displayed on the app screen.

[1429] Through the above steps, the system provides users with appropriate guidelines for action and supportive messages tailored to their emotions in real time, providing a sense of psychological security while ensuring their safety.

[1430] (Application example 2)

[1431] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1432] Conventional security systems have difficulty providing appropriate guidance in real time when users encounter dangerous situations. Furthermore, they do not provide guidance that reflects the user's emotional state, preventing users from feeling psychologically secure. Furthermore, there is a lack of technology to display instructions in a format suitable for wearable devices such as smart glasses.

[1433] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring current location information from the user terminal, means for analyzing emotional information acquired from the user terminal, means for receiving location information and emotional information transmitted from the user terminal, means for generating an optimal guideline of action using past incident data and real-time information, means for adjusting the generated guideline of action based on the emotional state, and means for transmitting the generated guideline of action to the user terminal. This enables the user to respond quickly to dangerous situations and act safely and with peace of mind.

[1434] A "user terminal" is a portable information terminal that a user can carry around and that is capable of acquiring and transmitting location information and emotion information.

[1435] "Location information" is data indicating the current latitude and longitude of the user terminal.

[1436] "Emotional information" is information obtained by analyzing the user's physiological and psychological state, such as facial expressions and heart rate.

[1437] The "analyzing means" refers to a technical means for acquiring and analyzing the user's emotional information to identify the user's emotional state.

[1438] "Past incident data" is a database that compiles information about incidents that have occurred in the past.

[1439] "Real-time information" means up-to-date information about ongoing events or situations.

[1440] The "means for generating" refers to the technical means for generating optimal guidelines for action based on the acquired location information and emotional information.

[1441] "Adjustment means" refers to the ability to appropriately change or complement already generated courses of action based on emotional information.

[1442] The "action guidelines" are specific activity guides for users to act safely.

[1443] "Displaying on a display" means visually presenting the generated action guidelines on the screen of the user terminal.

[1444] The present invention provides a system that, when a user encounters a dangerous situation, provides an optimal course of action in real time and adjusts the course of action by analyzing the user's emotional state. A specific embodiment of this system will be described below.

[1445] System configuration

[1446] This system is composed of a user terminal, a server, a cloud AI model, and an emotion engine.

[1447] User terminal

[1448] The user terminal is a wearable device such as smart glasses. This terminal is equipped with a GPS function, a camera, and a biometric sensor, and can acquire current location information and emotional information in real time. For example, the GPS function can be used to acquire latitude and longitude, and the camera and biometric sensor can be used to analyze facial expressions and heart rate to identify emotional information.

[1449] server

[1450] The server receives location and emotional information from the user's device and generates optimal guidelines based on the cloud AI model. The server sends this data to the cloud in JSON format and compares it with past incident data and real-time information to generate guidelines. It also has the function of adjusting guidelines based on the user's emotional information.

[1451] Cloud AI Model

[1452] The cloud AI model integrates past incident data and real-time information using location and emotion information received from the server. Based on the analysis results, it generates specific guidelines for users to act safely. For example, if a user is clearly feeling anxious or scared, it generates guidelines such as "Evacuate to the nearest building and avoid going outside until safety is confirmed."

[1453] Emotion Engine

[1454] The emotion engine analyzes the user's emotional state based on physiological data such as facial expressions and heart rate obtained from the user's device's camera and biometric sensors. The generated emotional information is sent to a cloud AI model and reflected in the generation and adjustment of action guidelines. The emotion engine can also provide encouraging messages that reflect the user's psychological state.

[1455] Specific examples

[1456] Suppose a user is attending an event in Shinjuku when a commotion breaks out in the crowd, causing their heart rate to rise and anxiety to rise. The smart glasses worn by the user use their GPS to obtain their current location and their camera and heart rate sensor to analyze their emotional information. This data is sent to a server, where a cloud AI model and emotion engine generate a guideline of action, such as "Please evacuate to a nearby building," and a message of encouragement, such as "It's okay, please act with peace of mind." These messages are displayed in real time on the smart glasses' display, and the user follows the instructions to take safe actions.

[1457] Prompt Sentence Examples

[1458] Latitude: 35.6895

[1459] Longitude: 139.6917

[1460] Emotional state: anxiety

[1461] Heart rate: 110

[1462] In this way, the present invention can prevent users from encountering danger, encourage safe behavior, and provide a sense of psychological security.

[1463] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1464] Step 1:

[1465] The user's device acquires current location information and collects emotional information. Specifically, it uses the device's GPS function to acquire latitude and longitude data, and uses the camera and biometric sensors to acquire physiological data such as the user's facial expressions and heart rate. This allows the user's device to obtain current location information (latitude and longitude) and emotional information (facial expression analysis results, heart rate).

[1466] Input: None

[1467] Output: Location information (latitude and longitude), emotional information (facial expression analysis results, heart rate)

[1468] Step 2:

[1469] The user device sends the acquired location information and emotion information to the server. The device converts this data into JSON format and sends it to the server via an HTTP request. At this time, data in the form of a prompt message is generated.

[1470] Input: Location information (latitude and longitude), emotional information (facial expression analysis results, heart rate)

[1471] Output: HTTP request to the server (data in JSON format)

[1472] Step 3:

[1473] The server receives the data sent from the user terminal and sends it to the cloud AI model. The server generates a prompt sentence including the received location information and emotion information and sends a request to the cloud AI model.

[1474] Input: HTTP request from the user device (JSON format data)

[1475] Output: Request to cloud AI model (prompt sentence including location and emotion information)

[1476] Step 4:

[1477] Based on the data received, the cloud AI model compares past incident data with real-time information to generate optimal guidelines for action. The cloud AI model integrates and analyzes location information and emotional information to generate specific guidelines for action that will enable users to act safely.

[1478] Input: Request to cloud AI model (prompt sentence)

[1479] Output: Generated course of action

[1480] Step 5:

[1481] The emotion engine adjusts the course of action based on the user's emotional information and adds a supportive message as needed. The emotion engine analyzes the received course of action and includes an optimal supportive message to alleviate the user's anxiety and fear.

[1482] Input: Generated course of action

[1483] Output: Coordinated course of action and encouraging message

[1484] Step 6:

[1485] The server sends the adjusted action plan to the user terminal. The server generates data including the adjusted action plan and a support message, and sends it to the user terminal as an HTTP response.

[1486] Input: Coordinated course of action and message of encouragement

[1487] Output: HTTP response to the user's device (adjusted guidelines and support message)

[1488] Step 7:

[1489] The user device processes the data received from the server and displays it on the smart glasses display. The device analyzes the received guidelines and support messages and visually presents them on the display.

[1490] Input: HTTP response from the server (adjusted course of action and support message)

[1491] Output: Guidelines and support messages displayed on the screen

[1492] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1493] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1494] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1495] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1496] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1497] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1498] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1499] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1500] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1501] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1502] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1503] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1504] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1505] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1506] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1507] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1508] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1509] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1510] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1511] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1512] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1513] The following is further disclosed regarding the above embodiment.

[1514] (Claim 1)

[1515] means for acquiring current location information from a user terminal;

[1516] means for receiving location information transmitted from a user terminal;

[1517] A means for generating optimal courses of action using historical incident data and real-time information;

[1518] means for transmitting the generated action guidelines to a user terminal;

[1519] A system including:

[1520] (Claim 2)

[1521] It will further equip itself with the means to integrate and analyze past incident data and real-time information,

[1522] The system according to claim 1, further comprising: generating an optimal course of action based on the analysis results.

[1523] (Claim 3)

[1524] 2. The system according to claim 1, wherein the optimal course of action is displayed in real time on a user terminal.

[1525] "Example 1"

[1526] (Claim 1)

[1527] means for acquiring current location information from a user terminal;

[1528] means for receiving location information transmitted from a user terminal;

[1529] A means for transmitting a prompt sentence including location information from the server to the generative AI model;

[1530] A means for generating optimal action plans using past incident data and real-time information through a generative AI model;

[1531] means for transmitting the generated action guidelines to a user terminal;

[1532] A system including:

[1533] (Claim 2)

[1534] It will further equip itself with the means to integrate and analyze past incident data and real-time information,

[1535] The system according to claim 1, further comprising: generating an optimal course of action based on the analysis results.

[1536] (Claim 3)

[1537] 2. The system according to claim 1, wherein the optimal course of action is displayed in real time on a user terminal.

[1538] "Application Example 1"

[1539] (Claim 1)

[1540] means for acquiring current location information from a user terminal;

[1541] means for receiving location information transmitted from a user terminal;

[1542] A means for generating optimal courses of action using historical incident data and real-time information;

[1543] means for transmitting the generated action guidelines to a user terminal;

[1544] a means for executing a course of action in real time based on the acquired location information in the autonomous vehicle;

[1545] A system including:

[1546] (Claim 2)

[1547] It will further equip itself with the means to integrate and analyze past incident data and real-time information,

[1548] The system described in claim 1 is characterized in that it applies an autonomous driving algorithm based on the results of this analysis to perform optimal actions.

[1549] (Claim 3)

[1550] 2. The system of claim 1, wherein the optimal course of action is displayed in real time on a user terminal and is automatically executed by an autonomous vehicle.

[1551] "Example 2: Combining Emotion Engines"

[1552] (Claim 1)

[1553] A means for obtaining current location information from a mobile device;

[1554] A means for acquiring user emotion information from a mobile device;

[1555] means for transmitting the acquired location information and emotion information to a server;

[1556] A means for the server to receive location information and emotion information;

[1557] A means for transmitting the location information and emotion information acquired by the server to a cloud AI model;

[1558] A cloud AI model uses historical incident data and real-time information to generate optimal courses of action; and

[1559] The emotion engine analyzes the emotion information and adds supportive messages to the action guidelines.

[1560] a means for transmitting the generated action guidelines and support messages from the server to the mobile terminal;

[1561] A system including:

[1562] (Claim 2)

[1563] It integrates and analyzes past incident data and real-time information, and generates optimal guidelines and support messages based on the analysis results and emotional information.

[1564] 10. The system of claim 1.

[1565] (Claim 3)

[1566] The optimal course of action and the support message are displayed on a mobile terminal in real time.

[1567] 10. The system of claim 1.

[1568] "Application example 2 when combining emotion engines"

[1569] (Claim 1)

[1570] means for acquiring current location information from a user terminal;

[1571] means for analyzing emotion information acquired from a user terminal;

[1572] means for receiving location information and emotion information transmitted from a user terminal;

[1573] A means for generating optimal courses of action using historical incident data and real-time information;

[1574] a means for adjusting the generated course of action based on the emotional state;

[1575] means for transmitting the generated action guidelines to a user terminal;

[1576] A system including:

[1577] (Claim 2)

[1578] It will further equip itself with the means to integrate and analyze past incident data and real-time information,

[1579] The system according to claim 1, characterized in that an optimal course of action is generated based on the analysis results and adjustments are made based on emotional information.

[1580] (Claim 3)

[1581] 2. The system according to claim 1, wherein the optimal course of action is displayed in real time on a user terminal. [Explanation of symbols]

[1582] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for acquiring current location information from a user terminal; means for receiving location information transmitted from a user terminal; A means for generating optimal courses of action using historical incident data and real-time information; means for transmitting the generated action guidelines to a user terminal; A system including:

2. It will further equip itself with the means to integrate and analyze past incident data and real-time information, The system according to claim 1, further comprising: generating an optimal course of action based on the analysis results.

3. 2. The system according to claim 1, wherein the optimal course of action is displayed in real time on a user terminal.

Images