system

Abstract

We provide the system. [Solution] A means for acquiring an image of the presented identification information, A means for analyzing the acquired image and extracting information, A means for comparing the extracted information with past verification history, A means for determining the need for reconfirmation based on the aforementioned verification results, Means for displaying the aforementioned judgment result, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a problem that it takes a great deal of time and labor to confirm identification information during police duty questioning. In particular, since the information obtained during confirmation is not properly managed, cases where the same information is confirmed again in a short period of time occur, which is not only inefficient but also a burden on the general public. Furthermore, since the accuracy of information confirmation decreases and misrecognition may occur, a faster and more accurate confirmation method is needed.

Means for Solving the Problems

[0005] This invention provides a means for acquiring and analyzing images of presented identification information to extract the information. Furthermore, it provides a system that includes means for automatically determining the need for re-verification by comparing the extracted information with past verification history. This achieves faster and more accurate verification, and at the same time reduces the burden on citizens by notifying them if re-verification is not necessary. In addition, the results obtained through these processing steps are appropriately managed in a database, which can also improve the efficiency of future verifications.

[0006] "Presented identification information" refers to data, in written or digital form, submitted by the user for verification purposes, which is used to identify an individual.

[0007] "Means for acquiring an image" refers to a device or method that captures the visual information of presented identification information as a digital image and stores it in a processable format.

[0008] "Means for analyzing images and extracting information" refers to devices or methods that perform the process of identifying necessary character and numerical information from acquired images and extracting it as electronic data.

[0009] "Means for cross-referencing past verification history" refers to devices or methods that perform a process of comparing extracted information with previous information recorded in a database to find matching data.

[0010] "Means for determining the need for reconfirmation" refers to devices or methods that perform a process based on criteria for determining whether information needs to be reconfirmed based on the verification results.

[0011] "Means for displaying judgment results" refers to devices or methods for visually showing users or police officers the judgment results regarding the need for reconfirmation.

[0012] "Means of managing a database" refers to a management system and its methods for systematically organizing, storing, updating, and deleting data, including verification history and matching results.

[0013] "Means for securely transmitting images" refers to methods and devices for transferring image data to its destination while protecting it using technologies such as encryption to prevent external attacks and data leaks. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotional engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0018] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a system for police officers to efficiently and quickly verify identification information during questioning. This system mainly consists of three components: a server, a terminal, and a user, each of which works in cooperation with the others.

[0036] The server hosts a database for managing identification information and an AI agent for image analysis. The server's role is to receive images containing identification information sent from the terminal, analyze them with the AI ​​agent, and extract the necessary information. The extracted information is then compared with the server's database, and a determination is made regarding the need for further verification based on past verification history. The server then returns this determination to the terminal.

[0037] The terminal is a device carried by police officers and begins by capturing an image of identification information provided by the user. The terminal sends this image data to a server and waits for a response from the server. It displays the matching results returned from the server and notifies the police officer if further verification is not required.

[0038] Users present their identification information to police officers during questioning. For users, the system offers the advantage of a faster verification process, saving unnecessary time.

[0039] As a concrete example, consider a scenario where a terminal takes a picture of a driver's license presented by a user and sends it to a server. Through image analysis and matching on the server, if it is determined that the same license information has been previously verified, the terminal will notify the police officer, allowing both the user and the officer to quickly leave the scene. The introduction of this system will improve the efficiency of police officers' work while simultaneously reducing unnecessary burdens for users.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The user presents their identification information to the police officer for questioning.

[0043] Step 2:

[0044] The device captures an image of the presented identification information using its camera and temporarily stores that image data.

[0045] Step 3:

[0046] The image data captured by the device is sent to the server using a secure protocol.

[0047] Step 4:

[0048] The server receives image data and passes it to an AI agent, which performs image analysis to extract identification information.

[0049] Step 5:

[0050] The server compares the extracted identification information with past verification history in the database to check for a match.

[0051] Step 6:

[0052] The server determines whether further verification is necessary based on the matching results.

[0053] Step 7:

[0054] The server sends the decision result back to the terminal.

[0055] Step 8:

[0056] The terminal displays the results from the server on the screen and informs the police officer that further verification is not necessary.

[0057] Step 9:

[0058] The user is informed that the verification is complete, and the questioning ends.

[0059] (Example 1)

[0060] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0061] The problem this invention aims to solve is to quickly and efficiently verify the identification information presented during a stop-and-frisk. Specifically, it aims to reduce the time cost and human error at the scene by automating the verification of identification information performed by police officers during stop-and-frisks, and by enabling efficient matching with verification history and decisions on whether re-verification is necessary.

[0062] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0063] In this invention, the server includes means for acquiring an image of a presented identification information medium, an analysis device for analyzing the acquired image and extracting identification information, and means for comparing the extracted identification information with verification history data. This makes it possible for police officers to quickly verify identification information at the scene and automatically determine the need for re-verification.

[0064] "Means for acquiring images of presented identification information media" refers to devices or methods for capturing image data from identification information media possessed by a user.

[0065] An "analysis device" refers to equipment or software used to analyze acquired image data and extract specific identification information from it.

[0066] "Means of matching with verification history data" refers to processes and algorithms for comparing extracted identification information with past records.

[0067] A "decision-making device" is a computational unit or method used to evaluate the need for reconfirmation of identification information based on the matching results.

[0068] A "database" is an information management system for systematically storing and managing matching results and identification information.

[0069] "Communication methods" refer to network communication technologies and protocols used to securely transmit acquired image data to information processing devices such as servers.

[0070] This invention is a system for efficiently verifying identification information during police questioning. This system mainly consists of three entities: a server, a terminal, and a user, and each entity works in cooperation with the others.

[0071] The server hosts a database for managing identification information and an AI agent for analyzing images. Image analysis is expected to utilize image processing and machine learning libraries such as OpenCV and TENSORFLOW®. Upon receiving images from a terminal, the server uses these tools to analyze them and extract identification information. The extracted information is compared with the database's matching history, and the need for further verification is automatically determined. The server then returns the results to the terminal.

[0072] The terminal is a mobile device used by police officers to capture images of identification information provided by the user. This could involve using a smartphone or a dedicated mobile device. The terminal sends the captured image data to a server and waits for a response from the server. The server then displays the verification results to the police officer, notifying them that further verification is not required. This notification may utilize audio alerts or screen displays.

[0073] Users provide identification information to police officers during questioning. The system allows users to complete the verification process quickly and leave the scene without long waits.

[0074] As a concrete example, consider a scenario where a police officer takes a picture of a driver's license presented by a user using a terminal and sends the image to a server. Through image analysis and matching with a database by the server, if it is determined that the same identification information has been previously verified, the terminal notifies the police officer and also informs the user of this information. This process allows the police officer to perform their duties quickly and efficiently, and the user can complete their work interaction smoothly.

[0075] When using a generative AI model, an example of a prompt would be: "What steps are necessary to build an AI model that extracts the necessary identification information to support the verification process of the presented driver's license?"

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] The terminal acquires an image of the identification information presented by the user. Specifically, a police officer uses the camera of a smartphone or a dedicated device to photograph the identification information medium. The input is the physical identification information medium presented by the user, and the output is the digital image data of that medium.

[0079] Step 2:

[0080] The terminal sends the acquired image data to the server. Specifically, the application on the terminal properly encrypts the image data and sends the image to the server's API endpoint using a secure communication protocol (e.g., HTTPS). The input is the digital image data acquired in step 1, and the output is a data packet in the format that the server receives.

[0081] Step 3:

[0082] The server analyzes the received image data to extract identification information. Specifically, the server first uses OpenCV to preprocess the images, and then uses TensorFlow to extract text from the acquired data. The main input is image data received from the terminal, and the output is extracted text information, such as names and identification numbers.

[0083] Step 4:

[0084] The server compares the extracted identification information with the verification history in the database. Specifically, it uses a database management system (e.g., PostgreSQL) to query and compare the extracted information with past records to find matching data. The input is the text information obtained in step 3, and the output is the verification result, such as a verified flag.

[0085] Step 5:

[0086] The server determines the need for re-verification based on the matching results. Specifically, it performs an evaluation based on factors such as the number of additions and past verification records through an algorithm based on business logic. The input is the matching result, and the output is a determination of whether or not re-verification is necessary.

[0087] Step 6:

[0088] The server sends the decision result back to the terminal. Specifically, it sends the decision result to the terminal via an API, using a protocol that ensures communication stability. The input is the decision result from step 5, and the output is a data packet to the terminal.

[0089] Step 7:

[0090] When the terminal receives results from the server, it displays that information on its screen and notifies the police officer. Specifically, a pop-up notification appears on the terminal, and an audio alert is played to the police officer. The input is the judgment result from the server, and the output is visual and auditory notification.

[0091] Step 8:

[0092] The user receives a result notification from the police officer and understands that their identification information has been verified without any problems. Specifically, the police officer verbally communicates the result and returns the identification information medium to the user. The input is the police officer's verbal explanation, and the output is the user's understanding and actions.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] In public facilities and event venues, where many people need to undergo ID verification in a short amount of time, waiting times occur, leading to a decrease in operational efficiency. This invention solves this problem by quickly and reliably verifying identification information and granting permission to pass through in such situations, thereby reducing waiting times and improving operational efficiency.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes means for acquiring an image of presented identification information, means for analyzing the acquired image and extracting information, and means for comparing the extracted information with past verification history. This enables rapid and accurate ID verification.

[0098] "Identification information" refers to information used to identify an individual, and mainly includes information such as names and photographs found on ID cards and driver's licenses.

[0099] "Means of acquiring images" refers to methods of capturing image data containing identification information in digital format using cameras, scanners, etc.

[0100] "Means of analyzing images and extracting information" refers to methods of identifying necessary information from acquired images using image analysis technology, specifically including character recognition and facial recognition.

[0101] "Methods for verifying past verification history" refer to methods of comparing past information stored in a database with current information to check for matches or discrepancies.

[0102] "Means for determining the need for reconfirmation" refers to a method of determining whether or not reconfirmation is necessary based on the verification results.

[0103] "Means for displaying the judgment result" refers to a display device or interface for visually showing the results of confirmation and judgment.

[0104] "Means for determining permission to pass through" refers to a method of determining whether to permit an individual to pass through a designated location based on extracted information and matching results.

[0105] One embodiment of this invention provides a system for police officers and security staff to efficiently verify identification information. The system mainly consists of three components: a server, a terminal, and a user.

[0106] The server hosts a database for managing identification information and an AI agent for image analysis. Specifically, it extracts facial images and text information from images obtained through image analysis using OpenCV and compares them with existing database information. Furthermore, it uses Python and Flask to perform fast and secure communication with the terminal. Through this process, it compares the information with past verification history, determines the need for re-verification, and decides whether to grant permission to proceed.

[0107] The terminals are portable devices such as smartphones and tablets carried by police officers and security staff. The terminals use their cameras to photograph the ID card or driver's license presented by the user and transmit the data to a server. Once the results are notified, they are displayed on the terminal to help the individual evacuate quickly.

[0108] The user's role is to present their identification information to police officers and security staff. The system allows users to complete the verification process quickly and stress-free.

[0109] A concrete example of this system is its use at a concert venue. In this scenario, attendees photograph their ID cards with their mobile devices, the information is analyzed on a server, and they can obtain quick entry permission.

[0110] An example of a prompt for a generative AI model might be: "How can I analyze an ID image taken by a user with a camera and verify if it matches the registered data in order to perform facial recognition and ID card matching?"

[0111] Thus, the present invention provides a means to streamline the verification of identification information in police and security settings and to enhance security by utilizing a database.

[0112] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0113] Step 1:

[0114] The terminal uses its camera to capture an image of the user's identification information (such as an ID card or driver's license). This image data becomes the terminal's input. The terminal uses a communication protocol to send the image data to the server in digital format.

[0115] Step 2:

[0116] The server receives image data sent from the terminal. Based on the input image data, it performs image analysis using OpenCV. Specifically, it applies a face recognition algorithm to detect face regions in the image and extracts the necessary information from them.

[0117] Step 3:

[0118] The server compares the extracted identification information with existing information in the database. During this comparison process, data calculations determine whether the input information matches past verification history. The result of the comparison becomes the server's output.

[0119] Step 4:

[0120] The server determines whether reconfirmation is necessary based on the matching results. In this case, if the matching results match, it decides to allow passage and does not require reconfirmation. To return the decision to the terminal, the server sends the result data to the terminal.

[0121] Step 5:

[0122] The terminal receives the judgment result sent back from the server and displays the information. Based on the displayed information, police officers and security staff inform the user that they are permitted to pass through. This allows the user to quickly pass through the designated area.

[0123] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0124] This invention provides a system that allows police officers to efficiently verify identification information during questioning while simultaneously recognizing the user's emotions. This system consists of a server, a terminal, and an emotion engine.

[0125] The server is equipped with a database and an AI agent, which performs image analysis of identification information and compares it with past history. The server analyzes the images of identification information received from the terminal and extracts the necessary information. This information is managed in the database, and the need for re-verification is determined based on the verification history. In addition, recognized emotion data is also stored in the database and used for future matching.

[0126] The terminal, designed for use by police officers, captures identification information provided by the user and transmits it to a server. The terminal also features an emotion engine that analyzes the user's facial expressions and voice in real time to recognize emotions. Based on the emotional data obtained by the emotion engine, it adjusts the verification procedures and processes, and displays instructions to the police officer as needed.

[0127] As an example, consider a scenario where a terminal takes a picture of a driver's license presented by a user and sends it to a server. The server performs image analysis, extracts identification information, and compares it with the verification history. Simultaneously, an emotion engine analyzes the user's facial expressions, and if tension or anxiety is detected, it displays a warning to the police officer. In this way, flexible responses tailored to individual situations become possible, leading to improved work efficiency for police officers and greater consideration for the user.

[0128] The following describes the processing flow.

[0129] Step 1:

[0130] The user presents their identification information to the police officer and prompts them to begin the operation.

[0131] Step 2:

[0132] The device captures an image of the presented identification information using its camera and temporarily stores that image.

[0133] Step 3:

[0134] The image data captured by the device is sent to the server via secure communication.

[0135] Step 4:

[0136] The server uses an AI agent to analyze the received image data and extract identification information.

[0137] Step 5:

[0138] The server compares the extracted identification information with past verification history in the database and searches for matching data.

[0139] Step 6:

[0140] The server determines whether further verification is necessary based on the matching results and sends the result to the terminal.

[0141] Step 7:

[0142] The device uses an emotion engine to analyze the user's facial expressions and voice in real time and acquire emotion data.

[0143] Step 8:

[0144] Based on the user's emotions recognized by the device, the system will adjust necessary procedures and alert police officers, optimizing the verification process.

[0145] Step 9:

[0146] The server records the judgment results and recognized emotion data in a database, which can be used for future questioning by police officers.

[0147] Step 10:

[0148] The terminal displays a message on the screen indicating that the procedure is complete, notifying the police officer that the verification process is finished.

[0149] Step 11:

[0150] The user is notified that the verification is complete and is quickly released from the situation.

[0151] (Example 2)

[0152] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0153] Modern police questioning requires not only verifying the presented identification information but also appropriately understanding the emotions of the person being questioned. However, in conventional systems, these processes are separated, making it difficult for police officers to respond flexibly to individual situations, resulting in challenges in operational efficiency and user consideration. In particular, there is a lack of technology to sensitively detect tension and anxiety and to autonomously respond based on that.

[0154] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0155] In this invention, the server includes means for analyzing the presented image of identification information and extracting information, means for comparing it with previous verification history, means for recognizing emotions from facial expressions and voice, and means for presenting warnings and procedural adjustments. This allows police officers to grasp the emotional state of a subject in real time and respond flexibly accordingly.

[0156] "Presented identification information" refers to documents, cards, or other media containing personally identifiable information provided by the user and used for verification.

[0157] "Image" refers to visual information acquired using a camera or scanner, and includes image data containing identification information.

[0158] "Analysis" refers to a series of processes that involve extracting necessary information from images and audio, and then understanding and processing it.

[0159] "Information extraction" refers to the process of extracting features from images and audio to obtain data for use in identification and analysis.

[0160] "Verification" refers to the process of comparing extracted information with existing databases and history to check for any matching information.

[0161] "Emotion recognition" refers to technology that determines a user's psychological state from their facial expressions, tone of voice, and other factors, and identifies their emotions.

[0162] "Adjustment" refers to the process of appropriately changing procedures and response methods based on recognized information.

[0163] This invention is a system for efficiently verifying presented identification information and recognizing user emotions during police questioning. The system mainly consists of a server, terminals, and an emotion engine.

[0164] The server is equipped with a database and an AI agent. It receives images containing identifying information from the user's terminal and performs image analysis. This analysis uses image recognition technology and algorithms. The server uses image processing libraries such as OpenCV and TensorFlow to extract the necessary identifying information from the images. The extracted information is compared with the database and past verification history. This process determines whether re-verification is necessary. Furthermore, recognized emotion data is also recorded in the database and used for subsequent data matching.

[0165] The terminal is a device carried by police officers and uses a camera to capture images of identification information presented by the user. These images are transmitted to a server via wireless communication (Wi-Fi or 4G / 5G). The terminal also features an emotion engine that analyzes the user's facial expressions and voice in real time. The emotion engine utilizes technologies such as Microsoft® Azure® Face API and Google® Cloud Vision API to recognize the user's emotional state. The results of this analysis are used to adjust the actions the police officer should take and are displayed as instructions on the terminal's screen.

[0166] A concrete example is a scenario where the terminal takes a picture of the user's driver's license and sends it to a server. The server compares the information extracted from the image with a database and returns the result to the terminal. Simultaneously, an emotion engine analyzes the user's facial expressions, and if it detects emotions such as tension or anxiety, an instruction to warn the police officer is displayed on the terminal. In this way, flexible and considerate responses during questioning become possible.

[0167] An example of a prompt for a generative AI model would be, "If the model analyzes the user's facial expression and detects tension, please display concise instructions to the police officer so that they can respond flexibly to questioning." This prompt creates a system where emotion analysis and its feedback function appropriately.

[0168] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0169] Step 1:

[0170] The device captures the identification information provided by the user. The device's camera acquires an image of the identification information, and this image becomes the input data. Specifically, the device's camera app is launched, and the identification information (e.g., driver's license) is clearly photographed using the autofocus function. The output of this process is digital image data containing the identification information.

[0171] Step 2:

[0172] The device sends the acquired image to the server. The input is the image data obtained in step 1, which is sent to the server using wireless communication (Wi-Fi or 4G / 5G). Specifically, the device uploads the image data to the server's specified endpoint using a secure protocol (e.g., HTTPS). The output of this process is the image data received by the server.

[0173] Step 3:

[0174] The server analyzes the received image and extracts identification information. The input is the image data received in step 2. The AI ​​agent installed on the server uses image recognition technology (e.g., OpenCV, TensorFlow) to extract text and facial information from the image and process the data. The output of this process is a dataset of the extracted identification information.

[0175] Step 4:

[0176] The server compares the extracted identification information with the verification history. The input is the dataset of identification information obtained in step 3. The server executes a database query to compare the identification information with past history. Specifically, the server accesses the database using Structured Query Language (SQL). The output of this process is the comparison result and a flag indicating whether re-verification is needed.

[0177] Step 5:

[0178] The device analyzes the user's facial expressions and voice in real time to recognize emotions. The input is real-time data acquired by the device's camera and microphone. The device uses an emotion engine, leveraging Microsoft Azure Face API and Google Cloud Vision API, among others, to analyze facial expressions and voice tone. The output of this process is data of the recognized emotions.

[0179] Step 6:

[0180] The terminal provides feedback to the police officer based on the emotion analysis results. The inputs are the emotion data obtained in step 5 and the matching results in step 4. The terminal displays a feedback message on the screen, providing the police officer with appropriate procedures and warnings. Specifically, the terminal's user interface displays instructions via a pop-up message to users who express tension, instructing them to respond carefully. The output of this process is the instructions the police officer receives through the terminal.

[0181] (Application Example 2)

[0182] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0183] In recent years, accurately and quickly verifying the identification information of visitors and passersby has become crucial in security services. Furthermore, understanding the emotional state of visitors is necessary to provide more appropriate responses. However, current systems lack efficient methods for simultaneously verifying identification information and recognizing emotions. This invention aims to solve these problems and provide an auxiliary system that enables security guards to respond quickly and appropriately.

[0184] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0185] In this invention, the server includes means for acquiring an image of presented identification information, means for analyzing the acquired image and extracting information, and means for recognizing emotions using the identification information and associated facial expression data. This makes it possible to determine the visitor's emotions simultaneously with verifying the identification information and to quickly provide instructions to security guards.

[0186] "Identifying information" refers to data provided to identify an individual, and typically includes information found in official documents or identification cards.

[0187] "Analyzing an image" refers to the process of processing acquired image data and extracting necessary information, typically using computer vision technology.

[0188] "Facial expression data" refers to data used to analyze a person's facial features and estimate their emotional state.

[0189] "Recognizing emotions" is the process of judging a person's emotional state from their facial expressions, voice, etc., and outputting it as a signal.

[0190] "Displaying action guidelines" means showing instructions on the device screen regarding what actions should be taken next, based on the information and analysis results obtained.

[0191] To realize this application, it is necessary to build a system in which the server, terminal, and user work together in cooperation.

[0192] The server is responsible for high-speed processing and secure database management. The server receives identification information and facial expression data transmitted from the terminal, analyzes the images using OpenCV, and performs emotion recognition using TensorFlow. Based on the analysis results, it searches the database for past history and matches any discrepancies. Through this process, the server stores newly acquired information, ensuring it is always up-to-date.

[0193] The device functions as smart glasses worn by security guards, capturing visitor identification information and facial expressions through its camera. The data captured by the camera is immediately sent to a server for analysis. This data receives real-time feedback from the server, and necessary instructions are displayed on the device's screen. For example, a green light is displayed if identity is confirmed, and a warning message is displayed if anxiety is detected in the emotions.

[0194] Users are visitors and beneficiaries of this system. By providing identification information, users can quickly analyze that information and take appropriate action.

[0195] A concrete example is security checks at large event venues. In this environment, there are many visitors, and efficient and accurate responses are required. The system helps to alleviate the burden on staff and improve security by supplementing these efforts.

[0196] The following prompt statements can be used for generative AI models.

[0197] Prompt example: "Analyze the visitor's emotional state—whether they appear smiling and comfortable, or tense and anxious—and advise the security guard accordingly."

[0198] In this way, the present invention provides a system that integrates rapid and accurate identification and sentiment analysis.

[0199] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0200] Step 1:

[0201] The device captures the user's identification information and facial expressions using its camera, acquiring the resulting image data. The input is image data, and the output is a digital image file ready for transmission to the server. The specific action performed here is to send the acquired image to the server via the network.

[0202] Step 2:

[0203] The server begins analyzing the received image data. The input is image data sent from the terminal, and the output is the results of face recognition and identification information extraction. The server uses OpenCV to analyze this data, identify faces in the image, and extract them as identification information.

[0204] Step 3:

[0205] The server compares the extracted identification information with past verification history in the database. The input is the extracted identification information, and the output is the matching result. At this stage, it quickly searches for a matching history and records the result.

[0206] Step 4:

[0207] The server recognizes emotions based on the received facial expression data. The input is facial expression data, and the output is recognized emotion information. Using TensorFlow, an emotion analysis model analyzes facial expressions in real time and determines the user's emotional state.

[0208] Step 5:

[0209] The server sends instructions to the terminal based on the matching results and sentiment recognition results. The input is the matching and sentiment analysis results, and the output is a feedback message to the terminal. Based on the generative AI model, the server creates prompt sentences and displays context-appropriate instructions on the terminal's display.

[0210] Step 6:

[0211] The terminal displays instructions received from the server as visual feedback to the security guard. Input is feedback messages from the server, and output is the instruction display on the screen. Specifically, colored lights or text messages are displayed on the terminal depending on the situation.

[0212] Therefore, users can undergo security checks quickly and appropriately.

[0213] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0214] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0215] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0216] [Second Embodiment]

[0217] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0218] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0219] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0220] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0221] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0222] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0223] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0224] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0225] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0226] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0227] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0228] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0229] This invention is a system for police officers to efficiently and quickly verify identification information during questioning. This system mainly consists of three components: a server, a terminal, and a user, each of which works in cooperation with the others.

[0230] The server hosts a database for managing identification information and an AI agent for image analysis. The server's role is to receive images containing identification information sent from the terminal, analyze them with the AI ​​agent, and extract the necessary information. The extracted information is then compared with the server's database, and a determination is made regarding the need for further verification based on past verification history. The server then returns this determination to the terminal.

[0231] The terminal is a device carried by police officers and begins by capturing an image of identification information provided by the user. The terminal sends this image data to a server and waits for a response from the server. It displays the matching results returned from the server and notifies the police officer if further verification is not required.

[0232] Users present their identification information to police officers during questioning. For users, the system offers the advantage of a faster verification process, saving unnecessary time.

[0233] As a concrete example, consider a scenario where a terminal takes a picture of a driver's license presented by a user and sends it to a server. Through image analysis and matching on the server, if it is determined that the same license information has been previously verified, the terminal will notify the police officer, allowing both the user and the officer to quickly leave the scene. The introduction of this system will improve the efficiency of police officers' work while simultaneously reducing unnecessary burdens for users.

[0234] The following describes the processing flow.

[0235] Step 1:

[0236] The user presents their identification information to the police officer for questioning.

[0237] Step 2:

[0238] The device captures an image of the presented identification information using its camera and temporarily stores that image data.

[0239] Step 3:

[0240] The image data captured by the device is sent to the server using a secure protocol.

[0241] Step 4:

[0242] The server receives image data and passes it to an AI agent, which performs image analysis to extract identification information.

[0243] Step 5:

[0244] The server compares the extracted identification information with past verification history in the database to check for a match.

[0245] Step 6:

[0246] The server determines whether further verification is necessary based on the matching results.

[0247] Step 7:

[0248] The server sends the decision result back to the terminal.

[0249] Step 8:

[0250] The terminal displays the results from the server on the screen and informs the police officer that further verification is not necessary.

[0251] Step 9:

[0252] The user is informed that the verification is complete, and the questioning ends.

[0253] (Example 1)

[0254] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0255] The problem this invention aims to solve is to quickly and efficiently verify the identification information presented during a stop-and-frisk. Specifically, it aims to reduce the time cost and human error at the scene by automating the verification of identification information performed by police officers during stop-and-frisks, and by enabling efficient matching with verification history and decisions on whether re-verification is necessary.

[0256] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0257] In this invention, the server includes means for acquiring an image of a presented identification information medium, an analysis device for analyzing the acquired image and extracting identification information, and means for comparing the extracted identification information with verification history data. This makes it possible for police officers to quickly verify identification information at the scene and automatically determine the need for re-verification.

[0258] "Means for acquiring images of presented identification information media" refers to devices or methods for capturing image data from identification information media possessed by a user.

[0259] An "analysis device" refers to equipment or software used to analyze acquired image data and extract specific identification information from it.

[0260] "Means of matching with verification history data" refers to processes and algorithms for comparing extracted identification information with past records.

[0261] A "decision-making device" is a computational unit or method used to evaluate the need for reconfirmation of identification information based on the matching results.

[0262] A "database" is an information management system for systematically storing and managing matching results and identification information.

[0263] "Communication methods" refer to network communication technologies and protocols used to securely transmit acquired image data to information processing devices such as servers.

[0264] This invention is a system for efficiently verifying identification information during police questioning. This system mainly consists of three entities: a server, a terminal, and a user, and each entity works in cooperation with the others.

[0265] The server hosts a database for managing identification information and an AI agent for analyzing images. Image processing and machine learning libraries such as OpenCV and TensorFlow are expected to be used for image analysis. Upon receiving images sent from the terminal, the server uses these tools to analyze them and extract identification information. The extracted information is compared with the database's matching history, and the need for further verification is automatically determined. The server then returns the results to the terminal.

[0266] The terminal is a mobile device used by police officers to capture images of identification information provided by the user. This could involve using a smartphone or a dedicated mobile device. The terminal sends the captured image data to a server and waits for a response from the server. The server then displays the verification results to the police officer, notifying them that further verification is not required. This notification may utilize audio alerts or screen displays.

[0267] Users provide identification information to police officers during questioning. The system allows users to complete the verification process quickly and leave the scene without long waits.

[0268] As a concrete example, consider a scenario where a police officer takes a picture of a driver's license presented by a user using a terminal and sends the image to a server. Through image analysis and matching with a database by the server, if it is determined that the same identification information has been previously verified, the terminal notifies the police officer and also informs the user of this information. This process allows the police officer to perform their duties quickly and efficiently, and the user can complete their work interaction smoothly.

[0269] When using a generative AI model, an example of a prompt would be: "What steps are necessary to build an AI model that extracts the necessary identification information to support the verification process of the presented driver's license?"

[0270] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0271] Step 1:

[0272] The terminal acquires an image of the identification information presented by the user. Specifically, a police officer uses the camera of a smartphone or a dedicated device to photograph the identification information medium. The input is the physical identification information medium presented by the user, and the output is the digital image data of that medium.

[0273] Step 2:

[0274] The terminal sends the acquired image data to the server. Specifically, the application on the terminal properly encrypts the image data and sends the image to the server's API endpoint using a secure communication protocol (e.g., HTTPS). The input is the digital image data acquired in step 1, and the output is a data packet in the format that the server receives.

[0275] Step 3:

[0276] The server analyzes the received image data to extract identification information. Specifically, the server first uses OpenCV to preprocess the images, and then uses TensorFlow to extract text from the acquired data. The main input is image data received from the terminal, and the output is extracted text information, such as names and identification numbers.

[0277] Step 4:

[0278] The server compares the extracted identification information with the verification history in the database. Specifically, it uses a database management system (e.g., PostgreSQL) to query and compare the extracted information with past records to find matching data. The input is the text information obtained in step 3, and the output is the verification result, such as a verified flag.

[0279] Step 5:

[0280] The server determines the need for re-verification based on the collation result. Specifically, through an algorithm based on business logic, an evaluation is performed based on the number of additions and past verification records, etc. The input is the collation result, and the output is a determination of whether re-verification is necessary.

[0281] Step 6:

[0282] The server returns the determination result to the terminal. Specifically, the determination result is sent to the terminal through the API, and a protocol for ensuring communication stability is used. The input is the determination result in Step 5, and the output is a data packet to the terminal.

[0283] Step 7:

[0284] When the terminal receives the result from the server, it displays that information on the screen and notifies the police officer. Specifically, a pop-up notification is displayed on the terminal and an audio alert is played for the police officer. The input is the determination result from the server, and the output is visual and auditory notifications.

[0285] Step 8:

[0286] The user receives the result notification from the police officer and understands that the identification information has been confirmed without problems. Specifically, the police officer verbally conveys the result and returns the identification information medium to the user. The input is the police officer's verbal explanation, and the output is the user's understanding and action.

[0287] (Application Example 1)

[0288] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0289] In public facilities and event venues, where many people need to undergo ID verification in a short amount of time, waiting times occur, leading to a decrease in operational efficiency. This invention solves this problem by quickly and reliably verifying identification information and granting permission to pass through in such situations, thereby reducing waiting times and improving operational efficiency.

[0290] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0291] In this invention, the server includes means for acquiring an image of presented identification information, means for analyzing the acquired image and extracting information, and means for comparing the extracted information with past verification history. This enables rapid and accurate ID verification.

[0292] "Identification information" refers to information used to identify an individual, and mainly includes information such as names and photographs found on ID cards and driver's licenses.

[0293] "Means of acquiring images" refers to methods of capturing image data containing identification information in digital format using cameras, scanners, etc.

[0294] "Means of analyzing images and extracting information" refers to methods of identifying necessary information from acquired images using image analysis technology, specifically including character recognition and facial recognition.

[0295] "Methods for verifying past verification history" refer to methods of comparing past information stored in a database with current information to check for matches or discrepancies.

[0296] "Means for determining the need for reconfirmation" refers to a method of determining whether or not reconfirmation is necessary based on the verification results.

[0297] "Means for displaying the judgment result" refers to a display device or interface for visually showing the results of confirmation and judgment.

[0298] "Means for determining permission to pass through" refers to a method of determining whether to permit an individual to pass through a designated location based on extracted information and matching results.

[0299] One embodiment of this invention provides a system for police officers and security staff to efficiently verify identification information. The system mainly consists of three components: a server, a terminal, and a user.

[0300] The server hosts a database for managing identification information and an AI agent for image analysis. Specifically, it extracts facial images and text information from images obtained through image analysis using OpenCV and compares them with existing database information. Furthermore, it uses Python and Flask to perform fast and secure communication with the terminal. Through this process, it compares the information with past verification history, determines the need for re-verification, and decides whether to grant permission to proceed.

[0301] The terminals are portable devices such as smartphones and tablets carried by police officers and security staff. The terminals use their cameras to photograph the ID card or driver's license presented by the user and transmit the data to a server. Once the results are notified, they are displayed on the terminal to help the individual evacuate quickly.

[0302] The user's role is to present their identification information to police officers and security staff. The system allows users to complete the verification process quickly and stress-free.

[0303] A concrete example of this system is its use at a concert venue. In this scenario, attendees photograph their ID cards with their mobile devices, the information is analyzed on a server, and they can obtain quick entry permission.

[0304] As an example of a prompt sentence for a generative AI model, a question such as "Please teach me how to analyze an ID image taken by a user with a camera to confirm whether it matches the registered data in order to perform face recognition and ID card verification." can be considered.

[0305] Thus, the present invention provides a means to improve the efficiency of confirming identification information at the scenes of police and security, and enhance safety by utilizing a database.

[0306] The flow of the specific process in Application Example 1 will be described with reference to FIG. 12.

[0307] Step 1:

[0308] The terminal captures an image of the identification information (ID card or driver's license) presented by the user with a camera. This image data serves as the input to the terminal. The terminal uses a communication protocol to transmit the image data to the server in digital format.

[0309] Step 2:

[0310] The server receives the image data transmitted from the terminal. Based on the input image data, it performs image analysis using OpenCV. Specifically, it applies a face recognition algorithm to detect the face region in the image and extracts the necessary information therefrom.

[0311] Step 3:

[0312] The server compares the extracted identification information with the existing information in the database. In this comparison process, it is determined by data calculation whether the input information matches the past confirmation history. The result of the comparison becomes the output of the server.

[0313] Step 4:

[0314] The server determines whether reconfirmation is necessary based on the matching results. In this case, if the matching results match, it decides to allow passage and does not require reconfirmation. To return the decision to the terminal, the server sends the result data to the terminal.

[0315] Step 5:

[0316] The terminal receives the judgment result sent back from the server and displays the information. Based on the displayed information, police officers and security staff inform the user that they are permitted to pass through. This allows the user to quickly pass through the designated area.

[0317] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0318] This invention provides a system that allows police officers to efficiently verify identification information during questioning while simultaneously recognizing the user's emotions. This system consists of a server, a terminal, and an emotion engine.

[0319] The server is equipped with a database and an AI agent, which performs image analysis of identification information and compares it with past history. The server analyzes the images of identification information received from the terminal and extracts the necessary information. This information is managed in the database, and the need for re-verification is determined based on the verification history. In addition, recognized emotion data is also stored in the database and used for future matching.

[0320] The terminal, designed for use by police officers, captures identification information provided by the user and transmits it to a server. The terminal also features an emotion engine that analyzes the user's facial expressions and voice in real time to recognize emotions. Based on the emotional data obtained by the emotion engine, it adjusts the verification procedures and processes, and displays instructions to the police officer as needed.

[0321] As an example, consider a scenario where a terminal takes a picture of a driver's license presented by a user and sends it to a server. The server performs image analysis, extracts identification information, and compares it with the verification history. Simultaneously, an emotion engine analyzes the user's facial expressions, and if tension or anxiety is detected, it displays a warning to the police officer. In this way, flexible responses tailored to individual situations become possible, leading to improved work efficiency for police officers and greater consideration for the user.

[0322] The following describes the processing flow.

[0323] Step 1:

[0324] The user presents their identification information to the police officer and prompts them to begin the operation.

[0325] Step 2:

[0326] The device captures an image of the presented identification information using its camera and temporarily stores that image.

[0327] Step 3:

[0328] The image data captured by the device is sent to the server via secure communication.

[0329] Step 4:

[0330] The server uses an AI agent to analyze the received image data and extract identification information.

[0331] Step 5:

[0332] The server compares the extracted identification information with past verification history in the database and searches for matching data.

[0333] Step 6:

[0334] The server determines whether further verification is necessary based on the matching results and sends the result to the terminal.

[0335] Step 7:

[0336] The device uses an emotion engine to analyze the user's facial expressions and voice in real time and acquire emotion data.

[0337] Step 8:

[0338] Based on the user's emotions recognized by the device, the system will adjust necessary procedures and alert police officers, optimizing the verification process.

[0339] Step 9:

[0340] The server records the judgment results and recognized emotion data in a database, which can be used for future questioning by police officers.

[0341] Step 10:

[0342] The terminal displays a message on the screen indicating that the procedure is complete, notifying the police officer that the verification process is finished.

[0343] Step 11:

[0344] The user is notified that the verification is complete and is quickly released from the situation.

[0345] (Example 2)

[0346] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0347] Modern police questioning requires not only verifying the presented identification information but also appropriately understanding the emotions of the person being questioned. However, in conventional systems, these processes are separated, making it difficult for police officers to respond flexibly to individual situations, resulting in challenges in operational efficiency and user consideration. In particular, there is a lack of technology to sensitively detect tension and anxiety and to autonomously respond based on that.

[0348] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0349] In this invention, the server includes means for analyzing the presented image of identification information and extracting information, means for comparing it with previous verification history, means for recognizing emotions from facial expressions and voice, and means for presenting warnings and procedural adjustments. This allows police officers to grasp the emotional state of a subject in real time and respond flexibly accordingly.

[0350] "Presented identification information" refers to documents, cards, or other media containing personally identifiable information provided by the user and used for verification.

[0351] "Image" refers to visual information acquired using a camera or scanner, and includes image data containing identification information.

[0352] "Analysis" refers to a series of processes that involve extracting necessary information from images and audio, and then understanding and processing it.

[0353] "Information extraction" refers to the process of extracting features from images and audio to obtain data for use in identification and analysis.

[0354] "Verification" refers to the process of comparing extracted information with existing databases and history to check for any matching information.

[0355] "Emotion recognition" refers to technology that determines a user's psychological state from their facial expressions, tone of voice, and other factors, and identifies their emotions.

[0356] "Adjustment" refers to the process of appropriately changing procedures and response methods based on recognized information.

[0357] This invention is a system for efficiently verifying presented identification information and recognizing user emotions during police questioning. The system mainly consists of a server, terminals, and an emotion engine.

[0358] The server is equipped with a database and an AI agent. It receives images containing identifying information from the user's terminal and performs image analysis. This analysis uses image recognition technology and algorithms. The server uses image processing libraries such as OpenCV and TensorFlow to extract the necessary identifying information from the images. The extracted information is compared with the database and past verification history. This process determines whether re-verification is necessary. Furthermore, recognized emotion data is also recorded in the database and used for subsequent data matching.

[0359] The terminal is a device carried by police officers and uses its camera to capture images of identification information provided by the user. These images are transmitted to a server via wireless communication (Wi-Fi or 4G / 5G). The terminal also features an emotion engine that analyzes the user's facial expressions and voice in real time. The emotion engine utilizes technologies such as Microsoft Azure Face API and Google Cloud Vision API to recognize the user's emotional state. The results of this analysis are used to adjust the actions the police officer should take and are displayed as instructions on the terminal's screen.

[0360] A concrete example is a scenario where the terminal takes a picture of the user's driver's license and sends it to a server. The server compares the information extracted from the image with a database and returns the result to the terminal. Simultaneously, an emotion engine analyzes the user's facial expressions, and if it detects emotions such as tension or anxiety, an instruction to warn the police officer is displayed on the terminal. In this way, flexible and considerate responses during questioning become possible.

[0361] An example of a prompt for a generative AI model would be, "If the model analyzes the user's facial expression and detects tension, please display concise instructions to the police officer so that they can respond flexibly to questioning." This prompt creates a system where emotion analysis and its feedback function appropriately.

[0362] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0363] Step 1:

[0364] The device captures the identification information provided by the user. The device's camera acquires an image of the identification information, and this image becomes the input data. Specifically, the device's camera app is launched, and the identification information (e.g., driver's license) is clearly photographed using the autofocus function. The output of this process is digital image data containing the identification information.

[0365] Step 2:

[0366] The device sends the acquired image to the server. The input is the image data obtained in step 1, which is sent to the server using wireless communication (Wi-Fi or 4G / 5G). Specifically, the device uploads the image data to the server's specified endpoint using a secure protocol (e.g., HTTPS). The output of this process is the image data received by the server.

[0367] Step 3:

[0368] The server analyzes the received image and extracts identification information. The input is the image data received in step 2. The AI ​​agent installed on the server uses image recognition technology (e.g., OpenCV, TensorFlow) to extract text and facial information from the image and process the data. The output of this process is a dataset of the extracted identification information.

[0369] Step 4:

[0370] The server compares the extracted identification information with the verification history. The input is the dataset of identification information obtained in step 3. The server executes a database query to compare the identification information with past history. Specifically, the server accesses the database using Structured Query Language (SQL). The output of this process is the comparison result and a flag indicating whether re-verification is needed.

[0371] Step 5:

[0372] The device analyzes the user's facial expressions and voice in real time to recognize emotions. The input is real-time data acquired by the device's camera and microphone. The device uses an emotion engine, leveraging Microsoft Azure Face API and Google Cloud Vision API, among others, to analyze facial expressions and voice tone. The output of this process is data of the recognized emotions.

[0373] Step 6:

[0374] The terminal provides feedback to the police officer based on the emotion analysis results. The inputs are the emotion data obtained in step 5 and the matching results in step 4. The terminal displays a feedback message on the screen, providing the police officer with appropriate procedures and warnings. Specifically, the terminal's user interface displays instructions via a pop-up message to users who express tension, instructing them to respond carefully. The output of this process is the instructions the police officer receives through the terminal.

[0375] (Application Example 2)

[0376] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0377] In recent years, accurately and quickly verifying the identification information of visitors and passersby has become crucial in security services. Furthermore, understanding the emotional state of visitors is necessary to provide more appropriate responses. However, current systems lack efficient methods for simultaneously verifying identification information and recognizing emotions. This invention aims to solve these problems and provide an auxiliary system that enables security guards to respond quickly and appropriately.

[0378] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0379] In this invention, the server includes means for acquiring an image of presented identification information, means for analyzing the acquired image and extracting information, and means for recognizing emotions using the identification information and associated facial expression data. This makes it possible to determine the visitor's emotions simultaneously with verifying the identification information and to quickly provide instructions to security guards.

[0380] "Identifying information" refers to data provided to identify an individual, and typically includes information found in official documents or identification cards.

[0381] "Analyzing an image" refers to the process of processing acquired image data and extracting necessary information, typically using computer vision technology.

[0382] "Facial expression data" refers to data used to analyze a person's facial features and estimate their emotional state.

[0383] "Recognizing emotions" is the process of judging a person's emotional state from their facial expressions, voice, etc., and outputting it as a signal.

[0384] "Displaying action guidelines" means showing instructions on the device screen regarding what actions should be taken next, based on the information and analysis results obtained.

[0385] To realize this application, it is necessary to build a system in which the server, terminal, and user work together in cooperation.

[0386] The server is responsible for high-speed processing and secure database management. The server receives identification information and facial expression data transmitted from the terminal, analyzes the images using OpenCV, and performs emotion recognition using TensorFlow. Based on the analysis results, it searches the database for past history and matches any discrepancies. Through this process, the server stores newly acquired information, ensuring it is always up-to-date.

[0387] The device functions as smart glasses worn by security guards, capturing visitor identification information and facial expressions through its camera. The data captured by the camera is immediately sent to a server for analysis. This data receives real-time feedback from the server, and necessary instructions are displayed on the device's screen. For example, a green light is displayed if identity is confirmed, and a warning message is displayed if anxiety is detected in the emotions.

[0388] Users are visitors and beneficiaries of this system. By providing identification information, users can quickly analyze that information and take appropriate action.

[0389] A concrete example is security checks at large event venues. In this environment, there are many visitors, and efficient and accurate responses are required. The system helps to alleviate the burden on staff and improve security by supplementing these efforts.

[0390] The following prompt statements can be used for generative AI models.

[0391] Prompt example: "Analyze the visitor's emotional state—whether they appear smiling and comfortable, or tense and anxious—and advise the security guard accordingly."

[0392] In this way, the present invention provides a system that integrates rapid and accurate identification and sentiment analysis.

[0393] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0394] Step 1:

[0395] The device captures the user's identification information and facial expressions using its camera, acquiring the resulting image data. The input is image data, and the output is a digital image file ready for transmission to the server. The specific action performed here is to send the acquired image to the server via the network.

[0396] Step 2:

[0397] The server begins analyzing the received image data. The input is image data sent from the terminal, and the output is the results of face recognition and identification information extraction. The server uses OpenCV to analyze this data, identify faces in the image, and extract them as identification information.

[0398] Step 3:

[0399] The server compares the extracted identification information with past verification history in the database. The input is the extracted identification information, and the output is the matching result. At this stage, it quickly searches for a matching history and records the result.

[0400] Step 4:

[0401] The server recognizes emotions based on the received facial expression data. The input is facial expression data, and the output is recognized emotion information. Using TensorFlow, an emotion analysis model analyzes facial expressions in real time and determines the user's emotional state.

[0402] Step 5:

[0403] The server sends instructions to the terminal based on the matching results and sentiment recognition results. The input is the matching and sentiment analysis results, and the output is a feedback message to the terminal. Based on the generative AI model, the server creates prompt sentences and displays context-appropriate instructions on the terminal's display.

[0404] Step 6:

[0405] The terminal displays instructions received from the server as visual feedback to the security guard. Input is feedback messages from the server, and output is the instruction display on the screen. Specifically, colored lights or text messages are displayed on the terminal depending on the situation.

[0406] Therefore, users can undergo security checks quickly and appropriately.

[0407] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0408] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0409] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0410] [Third Embodiment]

[0411] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0412] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0413] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0414] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0415] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0416] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0417] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0418] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0419] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0420] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0421] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0422] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0423] This invention is a system for police officers to efficiently and quickly verify identification information during questioning. This system mainly consists of three components: a server, a terminal, and a user, each of which works in cooperation with the others.

[0424] The server hosts a database for managing identification information and an AI agent for image analysis. The server's role is to receive images containing identification information sent from the terminal, analyze them with the AI ​​agent, and extract the necessary information. The extracted information is then compared with the server's database, and a determination is made regarding the need for further verification based on past verification history. The server then returns this determination to the terminal.

[0425] The terminal is a device carried by police officers and begins by capturing an image of identification information provided by the user. The terminal sends this image data to a server and waits for a response from the server. It displays the matching results returned from the server and notifies the police officer if further verification is not required.

[0426] Users present their identification information to police officers during questioning. For users, the system offers the advantage of a faster verification process, saving unnecessary time.

[0427] As a concrete example, consider a scenario where a terminal takes a picture of a driver's license presented by a user and sends it to a server. Through image analysis and matching on the server, if it is determined that the same license information has been previously verified, the terminal will notify the police officer, allowing both the user and the officer to quickly leave the scene. The introduction of this system will improve the efficiency of police officers' work while simultaneously reducing unnecessary burdens for users.

[0428] The following describes the processing flow.

[0429] Step 1:

[0430] The user presents their identification information to the police officer for questioning.

[0431] Step 2:

[0432] The device captures an image of the presented identification information using its camera and temporarily stores that image data.

[0433] Step 3:

[0434] The image data captured by the device is sent to the server using a secure protocol.

[0435] Step 4:

[0436] The server receives image data and passes it to an AI agent, which performs image analysis to extract identification information.

[0437] Step 5:

[0438] The server compares the extracted identification information with past verification history in the database to check for a match.

[0439] Step 6:

[0440] The server determines whether further verification is necessary based on the matching results.

[0441] Step 7:

[0442] The server sends the decision result back to the terminal.

[0443] Step 8:

[0444] The terminal displays the results from the server on the screen and informs the police officer that further verification is not necessary.

[0445] Step 9:

[0446] The user is informed that the verification is complete, and the questioning ends.

[0447] (Example 1)

[0448] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0449] The problem this invention aims to solve is to quickly and efficiently verify the identification information presented during a stop-and-frisk. Specifically, it aims to reduce the time cost and human error at the scene by automating the verification of identification information performed by police officers during stop-and-frisks, and by enabling efficient matching with verification history and decisions on whether re-verification is necessary.

[0450] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0451] In this invention, the server includes means for acquiring an image of a presented identification information medium, an analysis device for analyzing the acquired image and extracting identification information, and means for comparing the extracted identification information with verification history data. This makes it possible for police officers to quickly verify identification information at the scene and automatically determine the need for re-verification.

[0452] "Means for acquiring images of presented identification information media" refers to devices or methods for capturing image data from identification information media possessed by a user.

[0453] An "analysis device" refers to equipment or software used to analyze acquired image data and extract specific identification information from it.

[0454] "Means of matching with verification history data" refers to processes and algorithms for comparing extracted identification information with past records.

[0455] A "decision-making device" is a computational unit or method used to evaluate the need for reconfirmation of identification information based on the matching results.

[0456] A "database" is an information management system for systematically storing and managing matching results and identification information.

[0457] "Communication methods" refer to network communication technologies and protocols used to securely transmit acquired image data to information processing devices such as servers.

[0458] This invention is a system for efficiently verifying identification information during police questioning. This system mainly consists of three entities: a server, a terminal, and a user, and each entity works in cooperation with the others.

[0459] The server hosts a database for managing identification information and an AI agent for analyzing images. Image processing and machine learning libraries such as OpenCV and TensorFlow are expected to be used for image analysis. Upon receiving images sent from the terminal, the server uses these tools to analyze them and extract identification information. The extracted information is compared with the database's matching history, and the need for further verification is automatically determined. The server then returns the results to the terminal.

[0460] The terminal is a mobile device used by police officers to capture images of identification information provided by the user. This could involve using a smartphone or a dedicated mobile device. The terminal sends the captured image data to a server and waits for a response from the server. The server then displays the verification results to the police officer, notifying them that further verification is not required. This notification may utilize audio alerts or screen displays.

[0461] Users provide identification information to police officers during questioning. The system allows users to complete the verification process quickly and leave the scene without long waits.

[0462] As a concrete example, consider a scenario where a police officer takes a picture of a driver's license presented by a user using a terminal and sends the image to a server. Through image analysis and matching with a database by the server, if it is determined that the same identification information has been previously verified, the terminal notifies the police officer and also informs the user of this information. This process allows the police officer to perform their duties quickly and efficiently, and the user can complete their work interaction smoothly.

[0463] When using a generative AI model, an example of a prompt would be: "What steps are necessary to build an AI model that extracts the necessary identification information to support the verification process of the presented driver's license?"

[0464] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0465] Step 1:

[0466] The terminal acquires an image of the identification information presented by the user. Specifically, a police officer uses the camera of a smartphone or a dedicated device to photograph the identification information medium. The input is the physical identification information medium presented by the user, and the output is the digital image data of that medium.

[0467] Step 2:

[0468] The terminal sends the acquired image data to the server. Specifically, the application on the terminal properly encrypts the image data and sends the image to the server's API endpoint using a secure communication protocol (e.g., HTTPS). The input is the digital image data acquired in step 1, and the output is a data packet in the format that the server receives.

[0469] Step 3:

[0470] The server analyzes the received image data to extract identification information. Specifically, the server first uses OpenCV to preprocess the images, and then uses TensorFlow to extract text from the acquired data. The main input is image data received from the terminal, and the output is extracted text information, such as names and identification numbers.

[0471] Step 4:

[0472] The server compares the extracted identification information with the verification history in the database. Specifically, it uses a database management system (e.g., PostgreSQL) to query and compare the extracted information with past records to find matching data. The input is the text information obtained in step 3, and the output is the verification result, such as a verified flag.

[0473] Step 5:

[0474] The server determines the need for re-verification based on the matching results. Specifically, it performs an evaluation based on factors such as the number of additions and past verification records through an algorithm based on business logic. The input is the matching result, and the output is a determination of whether or not re-verification is necessary.

[0475] Step 6:

[0476] The server sends the decision result back to the terminal. Specifically, it sends the decision result to the terminal via an API, using a protocol that ensures communication stability. The input is the decision result from step 5, and the output is a data packet to the terminal.

[0477] Step 7:

[0478] When the terminal receives results from the server, it displays that information on its screen and notifies the police officer. Specifically, a pop-up notification appears on the terminal, and an audio alert is played to the police officer. The input is the judgment result from the server, and the output is visual and auditory notification.

[0479] Step 8:

[0480] The user receives a result notification from the police officer and understands that their identification information has been verified without any problems. Specifically, the police officer verbally communicates the result and returns the identification information medium to the user. The input is the police officer's verbal explanation, and the output is the user's understanding and actions.

[0481] (Application Example 1)

[0482] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0483] In public facilities and event venues, where many people need to undergo ID verification in a short amount of time, waiting times occur, leading to a decrease in operational efficiency. This invention solves this problem by quickly and reliably verifying identification information and granting permission to pass through in such situations, thereby reducing waiting times and improving operational efficiency.

[0484] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0485] In this invention, the server includes means for acquiring an image of presented identification information, means for analyzing the acquired image and extracting information, and means for comparing the extracted information with past verification history. This enables rapid and accurate ID verification.

[0486] "Identification information" refers to information used to identify an individual, and mainly includes information such as names and photographs found on ID cards and driver's licenses.

[0487] "Means of acquiring images" refers to methods of capturing image data containing identification information in digital format using cameras, scanners, etc.

[0488] "Means of analyzing images and extracting information" refers to methods of identifying necessary information from acquired images using image analysis technology, specifically including character recognition and facial recognition.

[0489] "Methods for verifying past verification history" refer to methods of comparing past information stored in a database with current information to check for matches or discrepancies.

[0490] "Means for determining the need for reconfirmation" refers to a method of determining whether or not reconfirmation is necessary based on the verification results.

[0491] "Means for displaying the judgment result" refers to a display device or interface for visually showing the results of confirmation and judgment.

[0492] "Means for determining permission to pass through" refers to a method of determining whether to permit an individual to pass through a designated location based on extracted information and matching results.

[0493] One embodiment of this invention provides a system for police officers and security staff to efficiently verify identification information. The system mainly consists of three components: a server, a terminal, and a user.

[0494] The server hosts a database for managing identification information and an AI agent for image analysis. Specifically, it extracts facial images and text information from images obtained through image analysis using OpenCV and compares them with existing database information. Furthermore, it uses Python and Flask to perform fast and secure communication with the terminal. Through this process, it compares the information with past verification history, determines the need for re-verification, and decides whether to grant permission to proceed.

[0495] The terminals are portable devices such as smartphones and tablets carried by police officers and security staff. The terminals use their cameras to photograph the ID card or driver's license presented by the user and transmit the data to a server. Once the results are notified, they are displayed on the terminal to help the individual evacuate quickly.

[0496] The user's role is to present their identification information to police officers and security staff. The system allows users to complete the verification process quickly and stress-free.

[0497] A concrete example of this system is its use at a concert venue. In this scenario, attendees photograph their ID cards with their mobile devices, the information is analyzed on a server, and they can obtain quick entry permission.

[0498] An example of a prompt for a generative AI model might be: "How can I analyze an ID image taken by a user with a camera and verify if it matches the registered data in order to perform facial recognition and ID card matching?"

[0499] Thus, the present invention provides a means to streamline the verification of identification information in police and security settings and to enhance security by utilizing a database.

[0500] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0501] Step 1:

[0502] The terminal uses its camera to capture an image of the user's identification information (such as an ID card or driver's license). This image data becomes the terminal's input. The terminal uses a communication protocol to send the image data to the server in digital format.

[0503] Step 2:

[0504] The server receives image data sent from the terminal. Based on the input image data, it performs image analysis using OpenCV. Specifically, it applies a face recognition algorithm to detect face regions in the image and extracts the necessary information from them.

[0505] Step 3:

[0506] The server compares the extracted identification information with existing information in the database. During this comparison process, data calculations determine whether the input information matches past verification history. The result of the comparison becomes the server's output.

[0507] Step 4:

[0508] The server determines whether reconfirmation is necessary based on the matching results. In this case, if the matching results match, it decides to allow passage and does not require reconfirmation. To return the decision to the terminal, the server sends the result data to the terminal.

[0509] Step 5:

[0510] The terminal receives the judgment result sent back from the server and displays the information. Based on the displayed information, police officers and security staff inform the user that they are permitted to pass through. This allows the user to quickly pass through the designated area.

[0511] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0512] This invention provides a system that allows police officers to efficiently verify identification information during questioning while simultaneously recognizing the user's emotions. This system consists of a server, a terminal, and an emotion engine.

[0513] The server is equipped with a database and an AI agent, which performs image analysis of identification information and compares it with past history. The server analyzes the images of identification information received from the terminal and extracts the necessary information. This information is managed in the database, and the need for re-verification is determined based on the verification history. In addition, recognized emotion data is also stored in the database and used for future matching.

[0514] The terminal, designed for use by police officers, captures identification information provided by the user and transmits it to a server. The terminal also features an emotion engine that analyzes the user's facial expressions and voice in real time to recognize emotions. Based on the emotional data obtained by the emotion engine, it adjusts the verification procedures and processes, and displays instructions to the police officer as needed.

[0515] As an example, consider a scenario where a terminal takes a picture of a driver's license presented by a user and sends it to a server. The server performs image analysis, extracts identification information, and compares it with the verification history. Simultaneously, an emotion engine analyzes the user's facial expressions, and if tension or anxiety is detected, it displays a warning to the police officer. In this way, flexible responses tailored to individual situations become possible, leading to improved work efficiency for police officers and greater consideration for the user.

[0516] The following describes the processing flow.

[0517] Step 1:

[0518] The user presents their identification information to the police officer and prompts them to begin the operation.

[0519] Step 2:

[0520] The device captures an image of the presented identification information using its camera and temporarily stores that image.

[0521] Step 3:

[0522] The image data captured by the device is sent to the server via secure communication.

[0523] Step 4:

[0524] The server uses an AI agent to analyze the received image data and extract identification information.

[0525] Step 5:

[0526] The server compares the extracted identification information with past verification history in the database and searches for matching data.

[0527] Step 6:

[0528] The server determines whether further verification is necessary based on the matching results and sends the result to the terminal.

[0529] Step 7:

[0530] The device uses an emotion engine to analyze the user's facial expressions and voice in real time and acquire emotion data.

[0531] Step 8:

[0532] Based on the user's emotions recognized by the device, the system will adjust necessary procedures and alert police officers, optimizing the verification process.

[0533] Step 9:

[0534] The server records the judgment results and recognized emotion data in a database, which can be used for future questioning by police officers.

[0535] Step 10:

[0536] The terminal displays a message on the screen indicating that the procedure is complete, notifying the police officer that the verification process is finished.

[0537] Step 11:

[0538] The user is notified that the verification is complete and is quickly released from the situation.

[0539] (Example 2)

[0540] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0541] Modern police questioning requires not only verifying the presented identification information but also appropriately understanding the emotions of the person being questioned. However, in conventional systems, these processes are separated, making it difficult for police officers to respond flexibly to individual situations, resulting in challenges in operational efficiency and user consideration. In particular, there is a lack of technology to sensitively detect tension and anxiety and to autonomously respond based on that.

[0542] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0543] In this invention, the server includes means for analyzing the presented image of identification information and extracting information, means for comparing it with previous verification history, means for recognizing emotions from facial expressions and voice, and means for presenting warnings and procedural adjustments. This allows police officers to grasp the emotional state of a subject in real time and respond flexibly accordingly.

[0544] "Presented identification information" refers to documents, cards, or other media containing personally identifiable information provided by the user and used for verification.

[0545] "Image" refers to visual information acquired using a camera or scanner, and includes image data containing identification information.

[0546] "Analysis" refers to a series of processes that involve extracting necessary information from images and audio, and then understanding and processing it.

[0547] "Information extraction" refers to the process of extracting features from images and audio to obtain data for use in identification and analysis.

[0548] "Verification" refers to the process of comparing extracted information with existing databases and history to check for any matching information.

[0549] "Emotion recognition" refers to technology that determines a user's psychological state from their facial expressions, tone of voice, and other factors, and identifies their emotions.

[0550] "Adjustment" refers to the process of appropriately changing procedures and response methods based on recognized information.

[0551] This invention is a system for efficiently verifying presented identification information and recognizing user emotions during police questioning. The system mainly consists of a server, terminals, and an emotion engine.

[0552] The server is equipped with a database and an AI agent. It receives images containing identifying information from the user's terminal and performs image analysis. This analysis uses image recognition technology and algorithms. The server uses image processing libraries such as OpenCV and TensorFlow to extract the necessary identifying information from the images. The extracted information is compared with the database and past verification history. This process determines whether re-verification is necessary. Furthermore, recognized emotion data is also recorded in the database and used for subsequent data matching.

[0553] The terminal is a device carried by police officers and uses its camera to capture images of identification information provided by the user. These images are transmitted to a server via wireless communication (Wi-Fi or 4G / 5G). The terminal also features an emotion engine that analyzes the user's facial expressions and voice in real time. The emotion engine utilizes technologies such as Microsoft Azure Face API and Google Cloud Vision API to recognize the user's emotional state. The results of this analysis are used to adjust the actions the police officer should take and are displayed as instructions on the terminal's screen.

[0554] A concrete example is a scenario where the terminal takes a picture of the user's driver's license and sends it to a server. The server compares the information extracted from the image with a database and returns the result to the terminal. Simultaneously, an emotion engine analyzes the user's facial expressions, and if it detects emotions such as tension or anxiety, an instruction to warn the police officer is displayed on the terminal. In this way, flexible and considerate responses during questioning become possible.

[0555] An example of a prompt for a generative AI model would be, "If the model analyzes the user's facial expression and detects tension, please display concise instructions to the police officer so that they can respond flexibly to questioning." This prompt creates a system where emotion analysis and its feedback function appropriately.

[0556] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0557] Step 1:

[0558] The device captures the identification information provided by the user. The device's camera acquires an image of the identification information, and this image becomes the input data. Specifically, the device's camera app is launched, and the identification information (e.g., driver's license) is clearly photographed using the autofocus function. The output of this process is digital image data containing the identification information.

[0559] Step 2:

[0560] The device sends the acquired image to the server. The input is the image data obtained in step 1, which is sent to the server using wireless communication (Wi-Fi or 4G / 5G). Specifically, the device uploads the image data to the server's specified endpoint using a secure protocol (e.g., HTTPS). The output of this process is the image data received by the server.

[0561] Step 3:

[0562] The server analyzes the received image and extracts identification information. The input is the image data received in step 2. The AI ​​agent installed on the server uses image recognition technology (e.g., OpenCV, TensorFlow) to extract text and facial information from the image and process the data. The output of this process is a dataset of the extracted identification information.

[0563] Step 4:

[0564] The server compares the extracted identification information with the verification history. The input is the dataset of identification information obtained in step 3. The server executes a database query to compare the identification information with past history. Specifically, the server accesses the database using Structured Query Language (SQL). The output of this process is the comparison result and a flag indicating whether re-verification is needed.

[0565] Step 5:

[0566] The device analyzes the user's facial expressions and voice in real time to recognize emotions. The input is real-time data acquired by the device's camera and microphone. The device uses an emotion engine, leveraging Microsoft Azure Face API and Google Cloud Vision API, among others, to analyze facial expressions and voice tone. The output of this process is data of the recognized emotions.

[0567] Step 6:

[0568] The terminal provides feedback to the police officer based on the emotion analysis results. The inputs are the emotion data obtained in step 5 and the matching results in step 4. The terminal displays a feedback message on the screen, providing the police officer with appropriate procedures and warnings. Specifically, the terminal's user interface displays instructions via a pop-up message to users who express tension, instructing them to respond carefully. The output of this process is the instructions the police officer receives through the terminal.

[0569] (Application Example 2)

[0570] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0571] In recent years, accurately and quickly verifying the identification information of visitors and passersby has become crucial in security services. Furthermore, understanding the emotional state of visitors is necessary to provide more appropriate responses. However, current systems lack efficient methods for simultaneously verifying identification information and recognizing emotions. This invention aims to solve these problems and provide an auxiliary system that enables security guards to respond quickly and appropriately.

[0572] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0573] In this invention, the server includes means for acquiring an image of presented identification information, means for analyzing the acquired image and extracting information, and means for recognizing emotions using the identification information and associated facial expression data. This makes it possible to determine the visitor's emotions simultaneously with verifying the identification information and to quickly provide instructions to security guards.

[0574] "Identifying information" refers to data provided to identify an individual, and typically includes information found in official documents or identification cards.

[0575] "Analyzing an image" refers to the process of processing acquired image data and extracting necessary information, typically using computer vision technology.

[0576] "Facial expression data" refers to data used to analyze a person's facial features and estimate their emotional state.

[0577] "Recognizing emotions" is the process of judging a person's emotional state from their facial expressions, voice, etc., and outputting it as a signal.

[0578] "Displaying action guidelines" means showing instructions on the device screen regarding what actions should be taken next, based on the information and analysis results obtained.

[0579] To realize this application, it is necessary to build a system in which the server, terminal, and user work together in cooperation.

[0580] The server is responsible for high-speed processing and secure database management. The server receives identification information and facial expression data transmitted from the terminal, analyzes the images using OpenCV, and performs emotion recognition using TensorFlow. Based on the analysis results, it searches the database for past history and matches any discrepancies. Through this process, the server stores newly acquired information, ensuring it is always up-to-date.

[0581] The device functions as smart glasses worn by security guards, capturing visitor identification information and facial expressions through its camera. The data captured by the camera is immediately sent to a server for analysis. This data receives real-time feedback from the server, and necessary instructions are displayed on the device's screen. For example, a green light is displayed if identity is confirmed, and a warning message is displayed if anxiety is detected in the emotions.

[0582] Users are visitors and beneficiaries of this system. By providing identification information, users can quickly analyze that information and take appropriate action.

[0583] A concrete example is security checks at large event venues. In this environment, there are many visitors, and efficient and accurate responses are required. The system helps to alleviate the burden on staff and improve security by supplementing these efforts.

[0584] The following prompt statements can be used for generative AI models.

[0585] Prompt example: "Analyze the visitor's emotional state—whether they appear smiling and comfortable, or tense and anxious—and advise the security guard accordingly."

[0586] In this way, the present invention provides a system that integrates rapid and accurate identification and sentiment analysis.

[0587] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0588] Step 1:

[0589] The device captures the user's identification information and facial expressions using its camera, acquiring the resulting image data. The input is image data, and the output is a digital image file ready for transmission to the server. The specific action performed here is to send the acquired image to the server via the network.

[0590] Step 2:

[0591] The server begins analyzing the received image data. The input is image data sent from the terminal, and the output is the results of face recognition and identification information extraction. The server uses OpenCV to analyze this data, identify faces in the image, and extract them as identification information.

[0592] Step 3:

[0593] The server compares the extracted identification information with past verification history in the database. The input is the extracted identification information, and the output is the matching result. At this stage, it quickly searches for a matching history and records the result.

[0594] Step 4:

[0595] The server recognizes emotions based on the received facial expression data. The input is facial expression data, and the output is recognized emotion information. Using TensorFlow, an emotion analysis model analyzes facial expressions in real time and determines the user's emotional state.

[0596] Step 5:

[0597] The server sends instructions to the terminal based on the matching results and sentiment recognition results. The input is the matching and sentiment analysis results, and the output is a feedback message to the terminal. Based on the generative AI model, the server creates prompt sentences and displays context-appropriate instructions on the terminal's display.

[0598] Step 6:

[0599] The terminal displays instructions received from the server as visual feedback to the security guard. Input is feedback messages from the server, and output is the instruction display on the screen. Specifically, colored lights or text messages are displayed on the terminal depending on the situation.

[0600] Therefore, users can undergo security checks quickly and appropriately.

[0601] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0602] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0603] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0604] [Fourth Embodiment]

[0605] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0606] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0607] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0608] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0609] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0610] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0611] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0612] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0613] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0614] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0615] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0616] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0617] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0618] This invention is a system for police officers to efficiently and quickly verify identification information during questioning. This system mainly consists of three components: a server, a terminal, and a user, each of which works in cooperation with the others.

[0619] The server hosts a database for managing identification information and an AI agent for image analysis. The server's role is to receive images containing identification information sent from the terminal, analyze them with the AI ​​agent, and extract the necessary information. The extracted information is then compared with the server's database, and a determination is made regarding the need for further verification based on past verification history. The server then returns this determination to the terminal.

[0620] The terminal is a device carried by police officers and begins by capturing an image of identification information provided by the user. The terminal sends this image data to a server and waits for a response from the server. It displays the matching results returned from the server and notifies the police officer if further verification is not required.

[0621] Users present their identification information to police officers during questioning. For users, the system offers the advantage of a faster verification process, saving unnecessary time.

[0622] As a concrete example, consider a scenario where a terminal takes a picture of a driver's license presented by a user and sends it to a server. Through image analysis and matching on the server, if it is determined that the same license information has been previously verified, the terminal will notify the police officer, allowing both the user and the officer to quickly leave the scene. The introduction of this system will improve the efficiency of police officers' work while simultaneously reducing unnecessary burdens for users.

[0623] The following describes the processing flow.

[0624] Step 1:

[0625] The user presents their identification information to the police officer for questioning.

[0626] Step 2:

[0627] The device captures an image of the presented identification information using its camera and temporarily stores that image data.

[0628] Step 3:

[0629] The image data captured by the device is sent to the server using a secure protocol.

[0630] Step 4:

[0631] The server receives image data and passes it to an AI agent, which performs image analysis to extract identification information.

[0632] Step 5:

[0633] The server compares the extracted identification information with past verification history in the database to check for a match.

[0634] Step 6:

[0635] The server determines whether further verification is necessary based on the matching results.

[0636] Step 7:

[0637] The server sends the decision result back to the terminal.

[0638] Step 8:

[0639] The terminal displays the results from the server on the screen and informs the police officer that further verification is not necessary.

[0640] Step 9:

[0641] The user is informed that the verification is complete, and the questioning ends.

[0642] (Example 1)

[0643] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0644] The problem this invention aims to solve is to quickly and efficiently verify the identification information presented during a stop-and-frisk. Specifically, it aims to reduce the time cost and human error at the scene by automating the verification of identification information performed by police officers during stop-and-frisks, and by enabling efficient matching with verification history and decisions on whether re-verification is necessary.

[0645] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0646] In this invention, the server includes means for acquiring an image of a presented identification information medium, an analysis device for analyzing the acquired image and extracting identification information, and means for comparing the extracted identification information with verification history data. This makes it possible for police officers to quickly verify identification information at the scene and automatically determine the need for re-verification.

[0647] "Means for acquiring images of presented identification information media" refers to devices or methods for capturing image data from identification information media possessed by a user.

[0648] An "analysis device" refers to equipment or software used to analyze acquired image data and extract specific identification information from it.

[0649] "Means of matching with verification history data" refers to processes and algorithms for comparing extracted identification information with past records.

[0650] A "decision-making device" is a computational unit or method used to evaluate the need for reconfirmation of identification information based on the matching results.

[0651] A "database" is an information management system for systematically storing and managing matching results and identification information.

[0652] "Communication methods" refer to network communication technologies and protocols used to securely transmit acquired image data to information processing devices such as servers.

[0653] This invention is a system for efficiently verifying identification information during police questioning. This system mainly consists of three entities: a server, a terminal, and a user, and each entity works in cooperation with the others.

[0654] The server hosts a database for managing identification information and an AI agent for analyzing images. Image processing and machine learning libraries such as OpenCV and TensorFlow are expected to be used for image analysis. Upon receiving images sent from the terminal, the server uses these tools to analyze them and extract identification information. The extracted information is compared with the database's matching history, and the need for further verification is automatically determined. The server then returns the results to the terminal.

[0655] The terminal is a mobile device used by police officers to capture images of identification information provided by the user. This could involve using a smartphone or a dedicated mobile device. The terminal sends the captured image data to a server and waits for a response from the server. The server then displays the verification results to the police officer, notifying them that further verification is not required. This notification may utilize audio alerts or screen displays.

[0656] Users provide identification information to police officers during questioning. The system allows users to complete the verification process quickly and leave the scene without long waits.

[0657] As a concrete example, consider a scenario where a police officer takes a picture of a driver's license presented by a user using a terminal and sends the image to a server. Through image analysis and matching with a database by the server, if it is determined that the same identification information has been previously verified, the terminal notifies the police officer and also informs the user of this information. This process allows the police officer to perform their duties quickly and efficiently, and the user can complete their work interaction smoothly.

[0658] When using a generative AI model, an example of a prompt would be: "What steps are necessary to build an AI model that extracts the necessary identification information to support the verification process of the presented driver's license?"

[0659] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0660] Step 1:

[0661] The terminal acquires an image of the identification information presented by the user. Specifically, a police officer uses the camera of a smartphone or a dedicated device to photograph the identification information medium. The input is the physical identification information medium presented by the user, and the output is the digital image data of that medium.

[0662] Step 2:

[0663] The terminal sends the acquired image data to the server. Specifically, the application on the terminal properly encrypts the image data and sends the image to the server's API endpoint using a secure communication protocol (e.g., HTTPS). The input is the digital image data acquired in step 1, and the output is a data packet in the format that the server receives.

[0664] Step 3:

[0665] The server analyzes the received image data to extract identification information. Specifically, the server first uses OpenCV to preprocess the images, and then uses TensorFlow to extract text from the acquired data. The main input is image data received from the terminal, and the output is extracted text information, such as names and identification numbers.

[0666] Step 4:

[0667] The server compares the extracted identification information with the verification history in the database. Specifically, it uses a database management system (e.g., PostgreSQL) to query and compare the extracted information with past records to find matching data. The input is the text information obtained in step 3, and the output is the verification result, such as a verified flag.

[0668] Step 5:

[0669] The server determines the need for re-verification based on the matching results. Specifically, it performs an evaluation based on factors such as the number of additions and past verification records through an algorithm based on business logic. The input is the matching result, and the output is a determination of whether or not re-verification is necessary.

[0670] Step 6:

[0671] The server sends the decision result back to the terminal. Specifically, it sends the decision result to the terminal via an API, using a protocol that ensures communication stability. The input is the decision result from step 5, and the output is a data packet to the terminal.

[0672] Step 7:

[0673] When the terminal receives results from the server, it displays that information on its screen and notifies the police officer. Specifically, a pop-up notification appears on the terminal, and an audio alert is played to the police officer. The input is the judgment result from the server, and the output is visual and auditory notification.

[0674] Step 8:

[0675] The user receives a result notification from the police officer and understands that their identification information has been verified without any problems. Specifically, the police officer verbally communicates the result and returns the identification information medium to the user. The input is the police officer's verbal explanation, and the output is the user's understanding and actions.

[0676] (Application Example 1)

[0677] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0678] In public facilities and event venues, where many people need to undergo ID verification in a short amount of time, waiting times occur, leading to a decrease in operational efficiency. This invention solves this problem by quickly and reliably verifying identification information and granting permission to pass through in such situations, thereby reducing waiting times and improving operational efficiency.

[0679] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0680] In this invention, the server includes means for acquiring an image of presented identification information, means for analyzing the acquired image and extracting information, and means for comparing the extracted information with past verification history. This enables rapid and accurate ID verification.

[0681] "Identification information" refers to information used to identify an individual, and mainly includes information such as names and photographs found on ID cards and driver's licenses.

[0682] "Means of acquiring images" refers to methods of capturing image data containing identification information in digital format using cameras, scanners, etc.

[0683] "Means of analyzing images and extracting information" refers to methods of identifying necessary information from acquired images using image analysis technology, specifically including character recognition and facial recognition.

[0684] "Methods for verifying past verification history" refer to methods of comparing past information stored in a database with current information to check for matches or discrepancies.

[0685] "Means for determining the need for reconfirmation" refers to a method of determining whether or not reconfirmation is necessary based on the verification results.

[0686] "Means for displaying the judgment result" refers to a display device or interface for visually showing the results of confirmation and judgment.

[0687] "Means for determining permission to pass through" refers to a method of determining whether to permit an individual to pass through a designated location based on extracted information and matching results.

[0688] One embodiment of this invention provides a system for police officers and security staff to efficiently verify identification information. The system mainly consists of three components: a server, a terminal, and a user.

[0689] The server hosts a database for managing identification information and an AI agent for image analysis. Specifically, it extracts facial images and text information from images obtained through image analysis using OpenCV and compares them with existing database information. Furthermore, it uses Python and Flask to perform fast and secure communication with the terminal. Through this process, it compares the information with past verification history, determines the need for re-verification, and decides whether to grant permission to proceed.

[0690] The terminals are portable devices such as smartphones and tablets carried by police officers and security staff. The terminals use their cameras to photograph the ID card or driver's license presented by the user and transmit the data to a server. Once the results are notified, they are displayed on the terminal to help the individual evacuate quickly.

[0691] The user's role is to present their identification information to police officers and security staff. The system allows users to complete the verification process quickly and stress-free.

[0692] A concrete example of this system is its use at a concert venue. In this scenario, attendees photograph their ID cards with their mobile devices, the information is analyzed on a server, and they can obtain quick entry permission.

[0693] An example of a prompt for a generative AI model might be: "How can I analyze an ID image taken by a user with a camera and verify if it matches the registered data in order to perform facial recognition and ID card matching?"

[0694] Thus, the present invention provides a means to streamline the verification of identification information in police and security settings and to enhance security by utilizing a database.

[0695] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0696] Step 1:

[0697] The terminal uses its camera to capture an image of the user's identification information (such as an ID card or driver's license). This image data becomes the terminal's input. The terminal uses a communication protocol to send the image data to the server in digital format.

[0698] Step 2:

[0699] The server receives image data sent from the terminal. Based on the input image data, it performs image analysis using OpenCV. Specifically, it applies a face recognition algorithm to detect face regions in the image and extracts the necessary information from them.

[0700] Step 3:

[0701] The server compares the extracted identification information with existing information in the database. During this comparison process, data calculations determine whether the input information matches past verification history. The result of the comparison becomes the server's output.

[0702] Step 4:

[0703] The server determines whether reconfirmation is necessary based on the matching results. In this case, if the matching results match, it decides to allow passage and does not require reconfirmation. To return the decision to the terminal, the server sends the result data to the terminal.

[0704] Step 5:

[0705] The terminal receives the judgment result sent back from the server and displays the information. Based on the displayed information, police officers and security staff inform the user that they are permitted to pass through. This allows the user to quickly pass through the designated area.

[0706] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0707] This invention provides a system that allows police officers to efficiently verify identification information during questioning while simultaneously recognizing the user's emotions. This system consists of a server, a terminal, and an emotion engine.

[0708] The server is equipped with a database and an AI agent, which performs image analysis of identification information and compares it with past history. The server analyzes the images of identification information received from the terminal and extracts the necessary information. This information is managed in the database, and the need for re-verification is determined based on the verification history. In addition, recognized emotion data is also stored in the database and used for future matching.

[0709] The terminal, designed for use by police officers, captures identification information provided by the user and transmits it to a server. The terminal also features an emotion engine that analyzes the user's facial expressions and voice in real time to recognize emotions. Based on the emotional data obtained by the emotion engine, it adjusts the verification procedures and processes, and displays instructions to the police officer as needed.

[0710] As an example, consider a scenario where a terminal takes a picture of a driver's license presented by a user and sends it to a server. The server performs image analysis, extracts identification information, and compares it with the verification history. Simultaneously, an emotion engine analyzes the user's facial expressions, and if tension or anxiety is detected, it displays a warning to the police officer. In this way, flexible responses tailored to individual situations become possible, leading to improved work efficiency for police officers and greater consideration for the user.

[0711] The following describes the processing flow.

[0712] Step 1:

[0713] The user presents their identification information to the police officer and prompts them to begin the operation.

[0714] Step 2:

[0715] The device captures an image of the presented identification information using its camera and temporarily stores that image.

[0716] Step 3:

[0717] The image data captured by the device is sent to the server via secure communication.

[0718] Step 4:

[0719] The server uses an AI agent to analyze the received image data and extract identification information.

[0720] Step 5:

[0721] The server compares the extracted identification information with past verification history in the database and searches for matching data.

[0722] Step 6:

[0723] The server determines whether further verification is necessary based on the matching results and sends the result to the terminal.

[0724] Step 7:

[0725] The device uses an emotion engine to analyze the user's facial expressions and voice in real time and acquire emotion data.

[0726] Step 8:

[0727] Based on the user's emotions recognized by the device, the system will adjust necessary procedures and alert police officers, optimizing the verification process.

[0728] Step 9:

[0729] The server records the judgment results and recognized emotion data in a database, which can be used for future questioning by police officers.

[0730] Step 10:

[0731] The terminal displays a message on the screen indicating that the procedure is complete, notifying the police officer that the verification process is finished.

[0732] Step 11:

[0733] The user is notified that the verification is complete and is quickly released from the situation.

[0734] (Example 2)

[0735] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0736] Modern police questioning requires not only verifying the presented identification information but also appropriately understanding the emotions of the person being questioned. However, in conventional systems, these processes are separated, making it difficult for police officers to respond flexibly to individual situations, resulting in challenges in operational efficiency and user consideration. In particular, there is a lack of technology to sensitively detect tension and anxiety and to autonomously respond based on that.

[0737] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0738] In this invention, the server includes means for analyzing the presented image of identification information and extracting information, means for comparing it with previous verification history, means for recognizing emotions from facial expressions and voice, and means for presenting warnings and procedural adjustments. This allows police officers to grasp the emotional state of a subject in real time and respond flexibly accordingly.

[0739] "Presented identification information" refers to documents, cards, or other media containing personally identifiable information provided by the user and used for verification.

[0740] "Image" refers to visual information acquired using a camera or scanner, and includes image data containing identification information.

[0741] "Analysis" refers to a series of processes that involve extracting necessary information from images and audio, and then understanding and processing it.

[0742] "Information extraction" refers to the process of extracting features from images and audio to obtain data for use in identification and analysis.

[0743] "Verification" refers to the process of comparing extracted information with existing databases and history to check for any matching information.

[0744] "Emotion recognition" refers to technology that determines a user's psychological state from their facial expressions, tone of voice, and other factors, and identifies their emotions.

[0745] "Adjustment" refers to the process of appropriately changing procedures and response methods based on recognized information.

[0746] This invention is a system for efficiently verifying presented identification information and recognizing user emotions during police questioning. The system mainly consists of a server, terminals, and an emotion engine.

[0747] The server is equipped with a database and an AI agent. It receives images containing identifying information from the user's terminal and performs image analysis. This analysis uses image recognition technology and algorithms. The server uses image processing libraries such as OpenCV and TensorFlow to extract the necessary identifying information from the images. The extracted information is compared with the database and past verification history. This process determines whether re-verification is necessary. Furthermore, recognized emotion data is also recorded in the database and used for subsequent data matching.

[0748] The terminal is a device carried by police officers and uses its camera to capture images of identification information provided by the user. These images are transmitted to a server via wireless communication (Wi-Fi or 4G / 5G). The terminal also features an emotion engine that analyzes the user's facial expressions and voice in real time. The emotion engine utilizes technologies such as Microsoft Azure Face API and Google Cloud Vision API to recognize the user's emotional state. The results of this analysis are used to adjust the actions the police officer should take and are displayed as instructions on the terminal's screen.

[0749] A concrete example is a scenario where the terminal takes a picture of the user's driver's license and sends it to a server. The server compares the information extracted from the image with a database and returns the result to the terminal. Simultaneously, an emotion engine analyzes the user's facial expressions, and if it detects emotions such as tension or anxiety, an instruction to warn the police officer is displayed on the terminal. In this way, flexible and considerate responses during questioning become possible.

[0750] An example of a prompt for a generative AI model would be, "If the model analyzes the user's facial expression and detects tension, please display concise instructions to the police officer so that they can respond flexibly to questioning." This prompt creates a system where emotion analysis and its feedback function appropriately.

[0751] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0752] Step 1:

[0753] The device captures the identification information provided by the user. The device's camera acquires an image of the identification information, and this image becomes the input data. Specifically, the device's camera app is launched, and the identification information (e.g., driver's license) is clearly photographed using the autofocus function. The output of this process is digital image data containing the identification information.

[0754] Step 2:

[0755] The device sends the acquired image to the server. The input is the image data obtained in step 1, which is sent to the server using wireless communication (Wi-Fi or 4G / 5G). Specifically, the device uploads the image data to the server's specified endpoint using a secure protocol (e.g., HTTPS). The output of this process is the image data received by the server.

[0756] Step 3:

[0757] The server analyzes the received image and extracts identification information. The input is the image data received in step 2. The AI ​​agent installed on the server uses image recognition technology (e.g., OpenCV, TensorFlow) to extract text and facial information from the image and process the data. The output of this process is a dataset of the extracted identification information.

[0758] Step 4:

[0759] The server compares the extracted identification information with the verification history. The input is the dataset of identification information obtained in step 3. The server executes a database query to compare the identification information with past history. Specifically, the server accesses the database using Structured Query Language (SQL). The output of this process is the comparison result and a flag indicating whether re-verification is needed.

[0760] Step 5:

[0761] The device analyzes the user's facial expressions and voice in real time to recognize emotions. The input is real-time data acquired by the device's camera and microphone. The device uses an emotion engine, leveraging Microsoft Azure Face API and Google Cloud Vision API, among others, to analyze facial expressions and voice tone. The output of this process is data of the recognized emotions.

[0762] Step 6:

[0763] The terminal provides feedback to the police officer based on the emotion analysis results. The inputs are the emotion data obtained in step 5 and the matching results in step 4. The terminal displays a feedback message on the screen, providing the police officer with appropriate procedures and warnings. Specifically, the terminal's user interface displays instructions via a pop-up message to users who express tension, instructing them to respond carefully. The output of this process is the instructions the police officer receives through the terminal.

[0764] (Application Example 2)

[0765] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0766] In recent years, accurately and quickly verifying the identification information of visitors and passersby has become crucial in security services. Furthermore, understanding the emotional state of visitors is necessary to provide more appropriate responses. However, current systems lack efficient methods for simultaneously verifying identification information and recognizing emotions. This invention aims to solve these problems and provide an auxiliary system that enables security guards to respond quickly and appropriately.

[0767] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0768] In this invention, the server includes means for acquiring an image of presented identification information, means for analyzing the acquired image and extracting information, and means for recognizing emotions using the identification information and associated facial expression data. This makes it possible to determine the visitor's emotions simultaneously with verifying the identification information and to quickly provide instructions to security guards.

[0769] "Identifying information" refers to data provided to identify an individual, and typically includes information found in official documents or identification cards.

[0770] "Analyzing an image" refers to the process of processing acquired image data and extracting necessary information, typically using computer vision technology.

[0771] "Facial expression data" refers to data used to analyze a person's facial features and estimate their emotional state.

[0772] "Recognizing emotions" is the process of judging a person's emotional state from their facial expressions, voice, etc., and outputting it as a signal.

[0773] "Displaying action guidelines" means showing instructions on the device screen regarding what actions should be taken next, based on the information and analysis results obtained.

[0774] To realize this application, it is necessary to build a system in which the server, terminal, and user work together in cooperation.

[0775] The server is responsible for high-speed processing and secure database management. The server receives identification information and facial expression data transmitted from the terminal, analyzes the images using OpenCV, and performs emotion recognition using TensorFlow. Based on the analysis results, it searches the database for past history and matches any discrepancies. Through this process, the server stores newly acquired information, ensuring it is always up-to-date.

[0776] The device functions as smart glasses worn by security guards, capturing visitor identification information and facial expressions through its camera. The data captured by the camera is immediately sent to a server for analysis. This data receives real-time feedback from the server, and necessary instructions are displayed on the device's screen. For example, a green light is displayed if identity is confirmed, and a warning message is displayed if anxiety is detected in the emotions.

[0777] Users are visitors and beneficiaries of this system. By providing identification information, users can quickly analyze that information and take appropriate action.

[0778] A concrete example is security checks at large event venues. In this environment, there are many visitors, and efficient and accurate responses are required. The system helps to alleviate the burden on staff and improve security by supplementing these efforts.

[0779] The following prompt statements can be used for generative AI models.

[0780] Prompt example: "Analyze the visitor's emotional state—whether they appear smiling and comfortable, or tense and anxious—and advise the security guard accordingly."

[0781] In this way, the present invention provides a system that integrates rapid and accurate identification and sentiment analysis.

[0782] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0783] Step 1:

[0784] The device captures the user's identification information and facial expressions using its camera, acquiring the resulting image data. The input is image data, and the output is a digital image file ready for transmission to the server. The specific action performed here is to send the acquired image to the server via the network.

[0785] Step 2:

[0786] The server begins analyzing the received image data. The input is image data sent from the terminal, and the output is the results of face recognition and identification information extraction. The server uses OpenCV to analyze this data, identify faces in the image, and extract them as identification information.

[0787] Step 3:

[0788] The server compares the extracted identification information with past verification history in the database. The input is the extracted identification information, and the output is the matching result. At this stage, it quickly searches for a matching history and records the result.

[0789] Step 4:

[0790] The server recognizes emotions based on the received facial expression data. The input is facial expression data, and the output is recognized emotion information. Using TensorFlow, an emotion analysis model analyzes facial expressions in real time and determines the user's emotional state.

[0791] Step 5:

[0792] The server sends instructions to the terminal based on the matching results and sentiment recognition results. The input is the matching and sentiment analysis results, and the output is a feedback message to the terminal. Based on the generative AI model, the server creates prompt sentences and displays context-appropriate instructions on the terminal's display.

[0793] Step 6:

[0794] The terminal displays instructions received from the server as visual feedback to the security guard. Input is feedback messages from the server, and output is the instruction display on the screen. Specifically, colored lights or text messages are displayed on the terminal depending on the situation.

[0795] Therefore, users can undergo security checks quickly and appropriately.

[0796] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0797] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0798] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0799] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0800] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0801] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0802] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0803] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0804] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0805] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0806] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0807] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0808] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0809] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0810] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0811] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0812] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0813] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0814] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0815] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0816] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0817] The following is further disclosed regarding the embodiments described above.

[0818] (Claim 1)

[0819] A means for acquiring an image of the presented identification information,

[0820] A means for analyzing the acquired image and extracting information,

[0821] A means for comparing the extracted information with past verification history,

[0822] A means for determining the need for reconfirmation based on the aforementioned verification results,

[0823] Means for displaying the aforementioned judgment result,

[0824] A system that includes this.

[0825] (Claim 2)

[0826] The system according to claim 1, further comprising means for managing a database that stores the aforementioned matching results and judgment results.

[0827] (Claim 3)

[0828] The system according to claim 1, further comprising means for securely transmitting the acquired image.

[0829] "Example 1"

[0830] (Claim 1)

[0831] Means for acquiring an image of the presented identification information medium,

[0832] An analysis device for analyzing the acquired images and extracting identification information,

[0833] A means for comparing the extracted identification information with the verification history data,

[0834] A determination device for determining the need for reconfirmation based on the aforementioned verification results,

[0835] A means for displaying the aforementioned judgment result and notifying the presenter,

[0836] A system that includes this.

[0837] (Claim 2)

[0838] The system according to claim 1, further comprising a database for storing and managing the aforementioned matching results and judgment results.

[0839] (Claim 3)

[0840] The system according to claim 1, further comprising communication means for securely transmitting the acquired image to an information processing device.

[0841] "Application Example 1"

[0842] (Claim 1)

[0843] A means for acquiring an image of the presented identification information,

[0844] A means for analyzing the acquired image and extracting information,

[0845] A means for comparing the extracted information with past verification history,

[0846] A means for determining the need for reconfirmation based on the aforementioned verification results,

[0847] Means for displaying the aforementioned judgment result,

[0848] A means for determining permission to pass using the extracted information and matching results,

[0849] A system that includes this.

[0850] (Claim 2)

[0851] The system according to claim 1, further comprising means for managing a database that stores the aforementioned matching results and judgment results.

[0852] (Claim 3)

[0853] The system according to claim 1, further comprising means for securely transmitting the acquired image.

[0854] "Example 2 of combining an emotion engine"

[0855] (Claim 1)

[0856] Means for obtaining an image of the presented identification information,

[0857] A means for analyzing the acquired image and extracting information,

[0858] A means for comparing the extracted information with previous verification history,

[0859] A means for evaluating the need for reconfirmation based on the aforementioned verification results,

[0860] A means of recognizing the user's emotions from their facial expressions and voice,

[0861] Based on the results of the aforementioned emotion recognition, a means of presenting warnings or procedural adjustments is provided.

[0862] Means for displaying the aforementioned evaluation results and emotion recognition results,

[0863] A system that includes this.

[0864] (Claim 2)

[0865] The system according to claim 1, further comprising means for managing a data storage device that stores the matching results and emotion recognition results information.

[0866] (Claim 3)

[0867] The system according to claim 1, further comprising means for securely communicating the acquired image.

[0868] "Application example 2 when combining with an emotional engine"

[0869] (Claim 1)

[0870] A means for acquiring an image of the presented identification information,

[0871] A means for analyzing the acquired image and extracting information,

[0872] A means for comparing the extracted information with past verification history,

[0873] A means for determining the need for reconfirmation based on the aforementioned verification results,

[0874] Means for displaying the aforementioned judgment result,

[0875] A means for recognizing emotions using the aforementioned identification information and associated facial expression data,

[0876] A means for displaying behavioral guidelines based on the aforementioned emotion recognition results,

[0877] A system that includes this.

[0878] (Claim 2)

[0879] The system according to claim 1, further comprising means for managing a database that stores the aforementioned matching results and judgment results.

[0880] (Claim 3)

[0881] The system according to claim 1, comprising means for securely transmitting the acquired image and emotion data. [Explanation of symbols]

[0882] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring an image of the presented identification information, A means for analyzing the acquired image and extracting information, A means for comparing the extracted information with past verification history, A means for determining the need for reconfirmation based on the aforementioned verification results, Means for displaying the aforementioned judgment result, A system that includes this.

2. The system according to claim 1, further comprising means for managing a database that stores the aforementioned matching results and judgment results.

3. The system according to claim 1, further comprising means for securely transmitting the acquired image.

Images