System

Abstract

A system is provided.SOLUTION: A system comprising: means for obtaining video from a luggage inspection device; means for sending the video to a production AI and requesting analysis; means for receiving analysis results from the production AI and determining a status of the luggage; and means for displaying the status of the luggage on a display.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Current baggage inspection at airports requires human visual inspection, making it difficult to secure human resources. Furthermore, during peak times, passengers have to wait, reducing convenience for users. To solve this problem, there is a need to build an automated baggage inspection system. [Means for solving the problem]

[0005] The present invention solves the above problems by the following means. It includes a means for acquiring video data from a baggage inspection device, a means for sending the video data to a generation AI and requesting analysis, and a means for receiving the analysis results from the generation AI and determining the status of the baggage. It also includes a means for displaying the baggage status on a display device. Furthermore, by providing a system that also includes a means for preprocessing the video data into a format that is easy to analyze and a means for issuing an alert when the baggage status is "requires confirmation," it provides a system that achieves full automation and efficiency of baggage inspection, contributing to reducing human resources and improving passenger service.

[0006] "Baggage inspection equipment" means a mechanical device used to inspect the contents of baggage at an airport or other facility.

[0007] "Video data" refers to image information of baggage acquired by a baggage inspection device.

[0008] "Generative AI" refers to artificial intelligence technology for analyzing acquired video data.

[0009] "Analysis" is the process in which the generative AI diagnoses the contents and structure of baggage based on video data and evaluates its safety.

[0010] "Baggage status" is information indicating the condition of the baggage based on the analysis results of the generating AI, and includes evaluations such as "passed" or "requires confirmation."

[0011] A "display device" is a device that visually presents the status of baggage to users and staff.

[0012] "Preprocessing" is the process of processing and organizing video data so that the generative AI can analyze it efficiently.

[0013] An "alert" is a warning signal or message that is issued when the baggage status is determined to be "in need of review." [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention provides a system that includes a baggage inspection device, a generating AI, a server, and a display device to fully automate baggage inspection at airports. This system realizes a series of automated processes, from scanning baggage to analyzing and displaying the results.

[0036] System Overview

[0037] 1. Baggage scanning process

[0038] The terminal acquires video data from the baggage inspection device. This is done by taking images of the baggage using cameras and X-ray devices installed inside the device when the baggage is placed in the inspection device and scanning begins.

[0039] 2. Video data transfer and preprocessing

[0040] The device sends the captured video data to the server, which then performs preprocessing as needed before sending the received video data to the generation AI. Preprocessing is a process that adjusts the image resolution or trims unnecessary parts.

[0041] 3. Analysis by generative AI

[0042] The server sends the preprocessed video data to the generation AI for analysis. The generation AI analyzes the contents and shape of the baggage based on the received video data and evaluates its safety. As a result of the analysis, it determines whether the baggage's status is "passed" or "requires inspection."

[0043] 4. Receiving analysis results and determining status

[0044] The server receives the analysis results from the AI ​​generator and determines the baggage status based on the results. If the status is "passed," the baggage is processed without any problems. On the other hand, if it is determined to require further investigation, an alert is sent to staff.

[0045] 5. Displaying the results

[0046] The terminal displays the baggage status received from the server on the display device. If the baggage has been "passed through," the display will show "passed through" and the user can proceed. If the baggage has been "checked," the display will show "checked," and the user will have to ask airport staff for confirmation.

[0047] Specific examples

[0048] Example 1: If there are no problems with your luggage

[0049] User Action:

[0050] The user places their baggage on the baggage inspection device and presses the scan button.

[0051] Terminal handling:

[0052] The terminal acquires the video data and transmits it to the server.

[0053] Server Action:

[0054] The server sends the video data to the generation AI and receives the analysis results.

[0055] The analysis result is determined to be "pass" and the result is sent to the terminal.

[0056] Terminal display:

[0057] The terminal will display "Passed" and the user can proceed.

[0058] Example 2: There is a problem with your package

[0059] User Action:

[0060] The user places their baggage on the baggage inspection device and presses the scan button.

[0061] Terminal handling:

[0062] The terminal acquires the video data and transmits it to the server.

[0063] Server Action:

[0064] The server sends the video data to the generation AI and receives the analysis results.

[0065] The analysis result is determined to be "required for confirmation," and the result is sent to the terminal.

[0066] Terminal display:

[0067] The terminal will display "Confirmation Required" and the user will need to ask airport staff for confirmation.

[0068] As described above, the present invention can fully automate the baggage inspection process at airports, thereby reducing the need for human resources and improving user convenience.

[0069] The processing flow will be explained below.

[0070] Step 1: The user places their baggage on the baggage inspection device and presses the scan button, which starts scanning the baggage.

[0071] Step 2: The terminal acquires image data from the baggage inspection equipment. Specifically, the on-board camera and X-ray device capture image data of the baggage.

[0072] Step 3: The device sends the acquired video data to the server. At this time, the device confirms that communication with the server has been established and then starts data transfer.

[0073] Step 4: The server receives the video data from the device and preprocesses it into a format that is easy for the AI ​​to analyze. This preprocessing includes adjusting the image size and removing noise.

[0074] Step 5: The server sends the pre-processed video data to the generation AI for analysis. The generation AI receives the data and begins the process of analyzing the contents and shape of the baggage.

[0075] Step 6: The generated AI analyzes the video data and determines the baggage status ("passed" or "requires inspection"). The analysis results are sent back to the server.

[0076] Step 7: The server receives the analysis results from the generation AI and makes a final decision on the baggage status based on the received analysis results.

[0077] Step 8: The server sends the baggage status to the terminal, along with a detailed message about the status (e.g., the reason if it is "required").

[0078] Step 9: The terminal receives the baggage status from the server and displays "Passed" or "Confirmation Required" based on the received status.

[0079] Step 10: The user checks the results displayed on the terminal. If the status is "Passed," proceed as is. If it is "Confirmation Required," ask airport staff for confirmation.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] Conventional baggage inspection requires manual inspection, which is time-consuming and prone to human error. In addition, baggage analysis requires highly accurate judgment, making it difficult to quickly and accurately determine the safety of baggage.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes means for acquiring video data from the baggage inspection device, means for transmitting the video data to the server and performing preprocessing, means for transmitting the preprocessed video data to a generation AI and requesting analysis, means for receiving the analysis results from the generation AI and determining the status of the baggage, and means for displaying the status of the baggage on a display device. This automates the inspection process, prevents human errors, and enables fast and accurate baggage screening.

[0085] A "baggage inspection device" is a device that uses X-rays and cameras to obtain image data of the inside of baggage when it is placed there.

[0086] "Video data" is image information showing the internal structure of baggage acquired by a baggage inspection device.

[0087] The "server" is the central device of the system that receives video data, preprocesses it, requests the generation AI to analyze it, and receives the analysis results and determines the status.

[0088] "Preprocessing" refers to the process of adjusting the resolution and trimming the video data to convert it into a format that is easier to analyze.

[0089] "Generative AI" is an artificial intelligence model that analyzes received video data, evaluates the contents and shape of baggage, and determines its safety.

[0090] "Baggage status" refers to the status of the baggage, which is determined to be "passed" or "requires inspection" based on the safety of the baggage analyzed by the generating AI.

[0091] A "display device" is a device that displays the status of baggage to the user and airport staff.

[0092] An "alert" is a notification to alert airport staff when the baggage status is "requires attention."

[0093] This invention aims to improve the efficiency of baggage inspection at airports by proposing a system that includes a baggage inspection device, a generation AI, a server, and a display device. This system can realize a series of automated processes, from scanning baggage to analyzing and displaying the results.

[0094] Specifically, a user places their baggage on the baggage inspection device and presses the scan button. The baggage inspection device is equipped with a high-resolution camera and X-ray device, which capture image data of the baggage. The captured image data is then sent to a server via the terminal.

[0095] The server performs preprocessing on the received video data. This preprocessing includes adjusting the image resolution and trimming unnecessary parts. This preprocessing is performed using image processing libraries such as OpenCV and Pillow. The preprocessed video data is then sent to the generation AI.

[0096] The generating AI uses a model called "CargoScanAI" to analyze the contents and shape of baggage based on preprocessed video data and assess its safety. The analysis determines whether it contains explosives or illegal items. An example of a specific prompt is, "Analyze the video data below and check the contents of the baggage. Please return a decision of 'pass' or 'requires inspection' based on your safety standards."

[0097] The analysis results are sent back to the server, which determines the baggage's status. The baggage's status is evaluated as either "passed" or "requires review." If it is "passed," the baggage is processed without issue and the user can proceed. If it is "requires review," an alert is generated and airport staff are notified.

[0098] Finally, the terminal displays the baggage status received from the server on the display device. The display device displays information according to the status. For example, if the baggage has been "passed through," it will display "passed through" and the user can proceed. If the baggage has been "checked," it will display "checked," and the user will need to ask airport staff for confirmation.

[0099] In this way, the present invention automates the baggage inspection process, eliminating human error while also enabling fast and accurate baggage screening.

[0100] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0101] Step 1:

[0102] The terminal acquires image data from the baggage inspection device. Specifically, when a user places their baggage on the inspection device and presses the scan button, the high-resolution camera and X-ray device built into the device acquire image data of the baggage. The input data is an image of the baggage, and image data of this is generated as the output.

[0103] Step 2:

[0104] The device sends the captured video data to the server. Specifically, the video data is transferred to the server via Wi-Fi or a wired connection. The input is the video data of the baggage, and the output is the data transfer to the server.

[0105] Step 3:

[0106] The server preprocesses the received video data. Specifically, it adjusts the resolution and trims unnecessary parts. For this, it uses image processing libraries such as OpenCV and Pillow. The input is raw video data, and the output is preprocessed video data.

[0107] Step 4:

[0108] The server sends the preprocessed video data to the generation AI and requests it to analyze it. An example of a specific prompt is, "Analyze the following video data and check the contents of the baggage. Please return a decision of 'pass' or 'requires confirmation' based on safety standards." The input is the preprocessed video data, and the output is the generation of an analysis request.

[0109] Step 5:

[0110] The generating AI analyzes the received video data and evaluates the contents and shape of the baggage. As a result of the analysis, the baggage's status is determined to be "passed" or "requires inspection." Specifically, this analysis is performed by the AI ​​model "CargoScanAI." The input is pre-processed video data, and the output is the analysis result.

[0111] Step 6:

[0112] The server receives the analysis results from the generation AI and determines the baggage status based on the results. It sets the status as "passed" or "requires confirmation." The input is the analysis result, and the output is the determined status.

[0113] Step 7:

[0114] If the server determines that the baggage status is "required confirmation," it generates an alert and notifies airport staff. Specifically, the alert message is displayed on a dedicated monitor or terminal. The input is the "required confirmation" status, and the output is the generation of an alert.

[0115] Step 8:

[0116] The terminal displays the baggage status received from the server on a display device. A specific operation is to display a message such as "Passed" or "Confirmation Required" on the display device. The input is the baggage status, and the output is the status displayed on the display device.

[0117] This detailed processing step allows the baggage inspection process to be fast and efficient.

[0118] (Application example 1)

[0119] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0120] Advanced technology is required to quickly and accurately inspect baggage during security checks at airports and large event venues. However, current baggage inspection systems rely on human resources and have issues with both efficiency and accuracy. The present invention aims to solve these issues by fully automating the baggage inspection process.

[0121] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0122] In this invention, the server includes a means for acquiring video data from a baggage inspection device, a means for sending the video data to a generating AI and requesting analysis, and a means for receiving the analysis results from the generating AI and determining the status of the baggage. This enables the automation and efficiency of baggage inspection. The server further includes a means for scanning the video data of the baggage in real time using a smart device, a means for transferring the real-time scanned data to a cloud server, a means for analyzing the data using the generating AI in the cloud server, and a means for displaying the analysis results of the generating AI on a display device of the smart device. This enables fast and accurate advanced baggage inspection in a variety of security situations.

[0123] A "baggage inspection device" is a scanning device installed at airports, event venues, etc. to check the contents and security of baggage.

[0124] "Video data" refers to image information and video information of baggage acquired by the baggage inspection device.

[0125] "Generative AI" is an artificial intelligence technology that analyzes video data of baggage and evaluates its contents and safety.

[0126] A "smart device" is an electronic device that has a built-in camera and display and is capable of acquiring and displaying data, and includes, for example, smart glasses and smartphones.

[0127] A "cloud server" is a remote server that processes and analyzes data via the Internet.

[0128] "Preprocessing" refers to data adjustment operations to convert video data into a format that is easy to analyze.

[0129] "Status" refers to the result of the generating AI's assessment of the baggage's safety, and typically includes a classification such as "passed" or "requires review."

[0130] An "alert" is a warning signal or notification that is issued when baggage status is "needs attention."

[0131] "Real-time scanning" is the process of capturing video data of baggage in real time using a smart device.

[0132] A "display device" is a device that visualizes and displays baggage status and alerts.

[0133] As an embodiment of the present invention, a baggage inspection system using a smart device will be described.

[0134] Server Processing

[0135] The server receives video data acquired from the baggage inspection equipment. This video data is scanned in real time by a camera built into the smart device. Once the video data is acquired, the server performs preprocessing such as adjusting the image resolution and trimming unnecessary parts. This preprocessing converts the data into a format that is easy for the generative AI model to analyze.

[0136] Analysis by generative AI

[0137] The server sends the preprocessed video data to the generative AI and asks it to evaluate the contents of the baggage and its safety. The generative AI model analyzes this data and determines its status as safe or unsafe. Once the analysis results are available, the server receives them.

[0138] Anomaly detection and alerts

[0139] The analysis results determine whether the baggage should be allowed to pass or requires further investigation, and if it does, the server immediately sends an alert to the smart device. This alert indicates that there is a potential risk to the baggage, requiring a prompt response.

[0140] Displaying the results

[0141] The analysis results from the server are reflected on the smart device's display. When the user places their baggage on the baggage inspection device, the smart device's display will show either "Pass" or "Inspection Required."

[0142] Hardware and software used

[0143] Smart Device: Electronic devices with built-in cameras and displays, such as smart glasses and smartphones.

[0144] Cloud server: A high-performance server that processes data and analyzes generative AI models.

[0145] Generative AI model: Artificial intelligence for analyzing video data and assessing safety.

[0146] Prompt Sentence Examples

[0147] The following are examples of prompts that can be passed to the generation AI:

[0148] "Analyze the video data to determine whether dangerous items are present in the baggage and assess its safety."

[0149] Preprocessed video data is sent along with this prompt, and the generative AI model analyzes it.

[0150] Specific examples

[0151] For example, if this system is implemented at the entrance to a large event, the smart glasses will capture video data in real time as attendees scan their belongings and send it to a server. The server will preprocess the data and request analysis from a generative AI model. If the analysis result is "pass," entry is permitted; if it is "verification required," an alert will be issued immediately, requiring appropriate action. This will enable smooth and efficient security checks at large events.

[0152] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0153] Step 1:

[0154] The terminal captures real-time video data of baggage using the camera of a smart device (e.g., smart glasses). The input is the actual visual information of the baggage, which is captured as video data. The output is the captured video data of the baggage.

[0155] Step 2:

[0156] The device sends the captured video data to a cloud server via Wi-Fi. The input is the video data captured on the device, and the output is the video data sent to the cloud server. Data conversion and compression may occur during this process.

[0157] Step 3:

[0158] The server preprocesses the received video data. Specifically, it adjusts the image resolution and trims unnecessary parts. The input is the video data sent from the terminal, and the output is the preprocessed video data.

[0159] Step 4:

[0160] The server sends the preprocessed video data to the generation AI and requests it to analyze it. The input is the preprocessed video data, and the output is the data sent to the generation AI and the analysis results. At this time, the prompt statement "Analyze the video data to determine whether the baggage contains dangerous items and evaluate its safety" is used.

[0161] Step 5:

[0162] The generative AI analyzes the received video data and evaluates the safety of the baggage. The input is the preprocessed video data and a prompt, and the output is the baggage status ("passed" or "requires inspection"). The generative AI uses a deep learning model to analyze the images and determine safety.

[0163] Step 6:

[0164] The server processes the analysis results received from the generation AI and determines the baggage status. The input is the analysis results from the generation AI, and the output is the final baggage status. This status is either "passed" or "requires confirmation."

[0165] Step 7:

[0166] The server sends the confirmed baggage status to the terminal. The input is the final baggage status, and the output is a notification to the terminal. If the baggage status is "confirmation required," alert information is also included.

[0167] Step 8:

[0168] The terminal receives the baggage status from the server and displays it on the display device. The input is the baggage status sent from the server, and the output is the notification displayed on the display. The user checks the notification and takes action as necessary.

[0169] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0170] This invention provides a system that fully automates airport baggage inspection and improves service quality by recognizing user emotions. The system includes a baggage inspection device, a generative AI, a server, a display device, and an emotion engine. This system automates a series of processes, from scanning baggage to analyzing the results, displaying the results, and responding to user emotions.

[0171] System Overview

[0172] 1. Baggage scanning process

[0173] The user places their baggage on the baggage inspection device and presses the scan button. The terminal acquires video data from the baggage inspection device, which then uses the onboard camera and X-ray device to capture image data of the baggage.

[0174] 2. Video data transfer and preprocessing

[0175] The device sends the captured video data to the server, which then performs preprocessing as needed before sending the received video data to the generation AI. Preprocessing is a process that adjusts the image resolution or trims unnecessary parts.

[0176] 3. Analysis by generative AI

[0177] The server sends the preprocessed video data to the generation AI for analysis. The generation AI analyzes the contents and shape of the baggage based on the received video data and evaluates its safety. As a result of the analysis, it determines whether the baggage's status is "passed" or "requires inspection."

[0178] 4. Receiving analysis results and determining status

[0179] The server receives the analysis results from the AI ​​generator and determines the baggage status based on the results. If the status is "passed," the baggage is processed without any problems. On the other hand, if it is determined to require further investigation, an alert is sent to staff.

[0180] 5. User Emotion Recognition by Emotion Engine

[0181] The device captures the user's face with a camera and sends it to the emotion engine. The emotion engine analyzes the user's facial expressions and recognizes the user's emotions (e.g., relief, surprise, anxiety, etc.). The recognized emotions are then sent to the server.

[0182] 6. Displaying results and addressing emotions

[0183] The terminal displays the results on a display device based on the baggage status received from the server and the user's emotional state. If the status is "passed," a message corresponding to the emotion, such as "relief," is displayed. If the status is "requires attention," an alert corresponding to the emotion is displayed, and staff will respond as necessary.

[0184] Specific examples

[0185] Example 1: If there are no problems with your luggage

[0186] User Action:

[0187] The user places their baggage on the baggage inspection device and presses the scan button.

[0188] The user's face is captured by a camera.

[0189] Terminal handling:

[0190] The terminal acquires the video data and transmits it to the server.

[0191] The terminal transmits the user's facial image to the emotion engine.

[0192] Server Action:

[0193] The server sends the video data to the generation AI and receives the analysis results.

[0194] The server receives the recognized emotion data from the emotion engine.

[0195] If the analysis result is determined to be "pass," a message based on the emotion is generated and sent to the device.

[0196] Terminal display:

[0197] The device will display "Passed" and a reassuring message.

[0198] The user can proceed.

[0199] Example 2: There is a problem with your package

[0200] User Action:

[0201] The user places their baggage on the baggage inspection device and presses the scan button.

[0202] The user's face is captured by a camera.

[0203] Terminal handling:

[0204] The terminal acquires the video data and transmits it to the server.

[0205] The terminal transmits the user's facial image to the emotion engine.

[0206] Server Action:

[0207] The server sends the video data to the generation AI and receives the analysis results.

[0208] The server receives the recognized emotion data from the emotion engine.

[0209] If the analysis results indicate that "attention is required," an emotion-based alert is generated and sent to the device.

[0210] Terminal display:

[0211] The device will display a message saying "Check required" and will try to ease any fears.

[0212] The user will ask airport staff for confirmation.

[0213] In this manner, the present invention can fully automate the baggage inspection process at airports and improve the quality of service by responding to user emotions.

[0214] The processing flow will be explained below.

[0215] Step 1: The user places their baggage on the baggage inspection device and presses the scan button, which starts scanning the baggage.

[0216] Step 2: The terminal acquires image data from the baggage inspection device. Image data of the baggage is captured by a camera or X-ray device and acquired as data.

[0217] Step 3: The device sends the acquired video data to the server. At this time, the device confirms that communication with the server has been established before transferring the data.

[0218] Step 4: The server preprocesses the video data received from the device. This involves adjusting the image resolution and trimming unnecessary parts, converting it into a format that is easy for the generating AI to analyze.

[0219] Step 5: The server sends the preprocessed video data to the generation AI for analysis. The generation AI analyzes the contents and shape of the baggage based on the received video data and determines its safety.

[0220] Step 6: The generating AI analyzes the data, determines the baggage status ('passed' or 'requires review'), and sends the result back to the server.

[0221] Step 7: The server receives the analysis results from the generation AI and makes a final decision on the baggage status based on the analysis results.

[0222] Step 8: The server receives the user's facial image from the device's camera and sends it to the emotion engine, which analyzes the facial expression and recognizes the user's emotional state (e.g., relief, anxiety, surprise, etc.).

[0223] Step 9: The emotion engine sends the recognized user emotion data back to the server, which uses this data along with the analysis results.

[0224] Step 10: The server generates an appropriate message or alert based on the baggage status and the user's emotions. If the baggage has passed, the server generates a message that provides reassurance, and if the baggage needs attention, the server generates a message that includes instructions to elicit reassurance.

[0225] Step 11: The server sends the generated message or alert to the terminal, which receives it and displays it on its display device.

[0226] Step 12: The terminal displays the baggage status and a message or alert depending on the user's emotion. If the baggage is "passed through," it displays "Proceed with confidence." If the baggage is "checked," it displays a message such as "Your baggage needs to be checked. Please contact an attendant."

[0227] Step 13: The user checks the displayed result. If it is "Pass", proceed as is. If it is "Confirmation Required", ask airport staff for confirmation.

[0228] Example 2

[0229] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0230] Conventional airport baggage inspection systems have the problem that they are unable to fully alleviate users' anxiety and dissatisfaction with inspection results because they are unable to accurately analyze the contents of baggage and respond to user emotions. To solve this problem, a system is needed that can accurately analyze baggage contents and display inspection results quickly and appropriately. Furthermore, it is necessary to improve the quality of service by recognizing users' emotions and responding accordingly.

[0231] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0232] In this invention, the server includes means for acquiring video data from a baggage inspection device, means for sending the video data to a generation AI and requesting analysis, means for receiving the analysis results from the generation AI and determining the status of the baggage, means for capturing the user's face and recognizing their emotions, and means for displaying the baggage status and the user's emotions on a display device, thereby enabling accurate analysis of the contents of the baggage, prompt display of inspection results, and response appropriate to the user's emotions.

[0233] A "baggage inspection device" is a device used at airports and other locations to scan baggage using X-rays or cameras to check its contents.

[0234] "Video data" refers to data that includes visual information about baggage, such as camera images or X-ray images acquired by baggage inspection equipment.

[0235] "Generative AI" is a technology that uses artificial intelligence to analyze video data and evaluate the contents, shape, and safety of baggage.

[0236] The "analysis results" are the information output by the generating AI after analyzing the video data, and indicate the status of the baggage.

[0237] "Baggage status" refers to whether the baggage can be passed through and whether any additional inspection is required, based on the analysis results of the generated AI.

[0238] "User" refers to a person who uses the baggage inspection system and performs the baggage scanning operation.

[0239] "Capturing a face" refers to obtaining an image of a user's face using a camera.

[0240] "Emotion recognition" means analyzing the user's facial expressions and identifying their emotional state, such as relief, surprise, or anxiety, from those expressions.

[0241] A "display device" is a device for visually displaying information such as baggage status and the user's emotional state.

[0242] This invention provides a system that fully automates airport baggage inspection and improves service quality by recognizing user emotions. The system includes a baggage inspection device, a generative AI, a server, a display device, and an emotion engine. This system automates a series of processes, from scanning baggage to analyzing the results, displaying the results, and responding to user emotions.

[0243] The operation of the system is as follows.

[0244] First, the user places their baggage on the baggage inspection device and presses the scan button. The terminal acquires video data from the baggage inspection device, which is then sent to the server. The server pre-processes the video data, which includes adjusting the image resolution and trimming unnecessary parts. Once pre-processing is complete, the server sends the video data to a generation AI, which analyzes the contents and shape of the baggage to assess its safety. As a result of the analysis, the baggage's status is determined as either "passed" or "requires inspection."

[0245] The results of this analysis are sent back to the server, which determines the baggage's status. If the status is "passed," the baggage is automatically processed and the user can pass through. On the other hand, if the status is "requires attention," an alert is generated and staff will respond.

[0246] At the same time, the device captures the user's face and sends the video data to the emotion engine. The emotion engine analyzes the user's facial expressions and recognizes the user's emotional state (e.g., relief, surprise, anxiety, etc.). The recognition results are sent to the server, which then integrates the baggage status and the user's emotional state and sends them to the device. The device then displays the integrated results on a display device, with a message appropriate for the user.

[0247] For example, when a user places his / her baggage on an inspection device and presses the scan button, the following process is carried out.

[0248] 1. The terminal acquires video data of the baggage and sends it to the server.

[0249] 2. The server preprocesses the video data and sends it to the generation AI.

[0250] 3. The generated AI analyzes the video data and may determine the baggage's status as "passed" or "requires inspection."

[0251] 4. At the same time, the device captures the user's face and sends it to the emotion engine, which then recognizes the user's emotions, such as "relief" or "anxiety."

[0252] 5. The server integrates the baggage status and emotion data and sends it to the terminal.

[0253] 6. The terminal displays the integrated results on the display device and displays an appropriate message to the user.

[0254] For example, consider the following prompt:

[0255] "Please begin baggage inspection scan."

[0256] "Package analysis results: Pass or check required?"

[0257] "Capture the user's face with a camera and recognize their emotions."

[0258] As described above, this system fully automates the baggage inspection process and makes it possible to provide services that adapt to the user's emotions.

[0259] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0260] Step 1:

[0261] The user places their baggage on the baggage inspection device and presses the scan button. This causes the baggage inspection device to acquire image data of the baggage (camera images and X-ray images). The input is the baggage, and the output is image data of the baggage. Specifically, the baggage inspection device starts scanning in response to the user's operation.

[0262] Step 2:

[0263] The terminal transmits the video data acquired from the baggage inspection device to the server. The input is the video data of the baggage, and the output is the data transmitted to the server. Specifically, the terminal transmits the video data to the server using network communication.

[0264] Step 3:

[0265] The server preprocesses the received video data. This preprocessing includes adjusting the image resolution and trimming unnecessary parts. The input is the video data sent to the server, and the output is the preprocessed video data. Specifically, the server processes the data using an image processing algorithm.

[0266] Step 4:

[0267] The server sends the pre-processed video data to the generation AI. The input is the pre-processed video data, and the output is the data sent to the generation AI. Specifically, the server communicates with the generation AI and sends the data.

[0268] Step 5:

[0269] The generating AI analyzes the contents and shape of the baggage based on the received video data and evaluates its safety. As a result of the analysis, the baggage's status is determined to be "passed" or "requires inspection." The input is the video data sent to the generating AI, and the output is the baggage's status. Specifically, the generating AI analyzes the data using a machine learning algorithm.

[0270] Step 6:

[0271] The server receives the analysis results from the generation AI and determines the baggage status based on the results. If the status is "passed," the baggage is processed without any problems. If the status is "requires attention," an alert is raised and staff are notified. The input is the analysis results from the generation AI, and the output is the baggage status and an alert if necessary. Specifically, the server processes the analysis results and determines the status.

[0272] Step 7:

[0273] The device captures the user's face with a camera and sends the video data to the emotion engine. The input is the user's face video data, and the output is the data sent to the emotion engine. In concrete terms, the device captures the face with a camera and sends the data.

[0274] Step 8:

[0275] The emotion engine analyzes the user's facial expressions and recognizes the user's emotions (relief, surprise, anxiety, etc.). The recognized emotional state is sent to the server. The input is the facial image data sent to the emotion engine, and the output is the analyzed emotional data. In concrete terms, the emotion engine uses a facial expression analysis algorithm.

[0276] Step 9:

[0277] The server integrates the baggage status and emotion data and sends it to the terminal. The input is the baggage status and emotion data, and the output is the integrated result data. Specifically, the server performs the integration calculation to generate the result.

[0278] Step 10:

[0279] The terminal displays the integrated result on the display device. For example, if the status is "passed" and the user's emotion is "relieved," the terminal will display that fact and the user can proceed. If the status is "confirmation required," the terminal will display an alert and the user will ask the staff for confirmation. The input is the integrated result data, and the output is the result displayed on the display device. As a specific operation, the terminal will display an appropriate message on the display.

[0280] The above is the processing flow of this system.

[0281] (Application example 2)

[0282] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0283] Conventional baggage inspection systems automate the display of inspection results and the issuance of alerts, but they have the problem of being unable to respond in a way that takes the user's emotions into consideration. This can cause users to feel anxious or uncomfortable, and there is a need to improve the quality of service. Meanwhile, smart home security systems are also required to accurately grasp the security situation and respond appropriately according to the user's emotions. To address these issues, systems that can reduce user anxiety and ensure safety are needed not only in baggage inspection systems but also in smart home security.

[0284] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0285] In this invention, the server includes means for acquiring video data from the baggage inspection device, means for sending the video data to the generation AI and requesting analysis, means for receiving the analysis results from the generation AI and determining the status of the baggage, means for capturing the user's face using a camera and sending the results to the emotion engine, means for analyzing the user's emotions and sending the results to the server, and means for displaying an appropriate message or alert on a display device based on the emotions. This makes it possible to respond according to the user's emotions and provide a more human-like, high-quality service.

[0286] "Baggage screening device" means a device used at airports and other security checkpoints to scan baggage and verify its contents.

[0287] "Video data" refers to digital image and video data acquired from baggage inspection equipment and cameras.

[0288] "Generative AI" is an artificial intelligence technology that analyzes acquired video data and independently determines its content and characteristics.

[0289] The "analysis results" are information about the contents and condition of baggage generated after the generating AI analyzes the video data.

[0290] "Status" is information indicating the safety of baggage and the status of inspection results, and includes indicators such as "passed" and "requires confirmation."

[0291] A "display device" is a device for visually displaying baggage inspection results and messages based on the user's emotions.

[0292] An "emotion engine" is a software system that analyzes a user's face and recognizes emotions (such as relief, surprise, or anxiety).

[0293] "Capture" is the act of acquiring image or video data using a device such as a camera.

[0294] A "server" is a computer system that processes and manages data for the entire system.

[0295] "Preprocessing" refers to processes such as adjusting resolution and trimming unnecessary parts, which are carried out to convert video data into a format that is easier to analyze.

[0296] A "message" refers to information or notifications provided to a user, the content of which is adjusted based on emotions.

[0297] An "alert" is a notification that warns users or staff when the baggage status is "requires attention" or when other abnormalities occur.

[0298] The smart home security system according to the present invention is implemented by the following procedure.

[0299] System Overview

[0300] The system consists of the following major hardware and software components:

[0301] Security camera (for acquiring video data)

[0302] Emotion engine (for recognizing user emotions)

[0303] Generative AI (for video data analysis)

[0304] Server (data processing and management)

[0305] Display device (for displaying results)

[0306] 1. Acquisition and transmission of video data

[0307] First, the device (smart home system) acquires video data in real time using a security camera. When a user enters the camera's range, their face is captured and sent to the emotion engine for emotion recognition. In parallel, the entire video data is sent to the server.

[0308] 2. Emotion recognition and sending results

[0309] The emotion engine analyzes the facial image and recognizes the user's emotions (relief, surprise, anxiety, etc.). The recognition results are sent to the server, which processes the data.

[0310] 3. Video data preprocessing and analysis

[0311] The server performs preprocessing such as adjusting the resolution and trimming unnecessary parts of the video data received from the security camera before sending it to the generation AI. The preprocessed data is sent to the generation AI for analysis. The generation AI evaluates security risks (suspicious movements, abnormal behavior, etc.) based on the video data and returns the analysis results to the server.

[0312] 4. Processing and displaying results

[0313] The server determines the security status based on the analysis results received from the generation AI. If the status is "safe," the result and a message based on the user's emotion are displayed on the display device. If the status is "requires attention," an alert is issued and a message based on the emotion is displayed, prompting the user to take appropriate action if necessary.

[0314] Specific examples

[0315] For example, the flow when a security camera detects movement late at night is as follows: The camera footage is sent to the server, and if the generation AI judges it to be "safe," the server displays the message "The environment is safe. Please be at ease," even if the emotion engine recognizes "anxiety." If the generation AI judges there to be a "potential threat," the server will display the message "A potential threat has been detected. Please respond with caution," even if the emotion engine recognizes "anxiety."

[0316] Prompt Sentence Examples

[0317] An example of a prompt sent to the generative AI model is as follows:

[0318] "Please analyze the baggage scan footage to determine if it contains any suspicious items. The video data is as follows."

[0319] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0320] Step 1:

[0321] The device captures video data in real time using a security camera. When a user enters the camera's range, their face is automatically captured. The captured video data and facial image are then stored in the device's memory.

[0322] Input: Real-time footage from security cameras

[0323] Output: Video data and facial images are saved on the device.

[0324] Specific operation: The security camera detects movement, captures video data, and stores it in the device's memory.

[0325] Step 2:

[0326] The device sends the stored facial image data to the emotion engine, which analyzes the user's face and recognizes emotions (relief, surprise, anxiety, etc.). The emotion recognition results are returned to the device, which receives and stores the results.

[0327] Input: Facial image data

[0328] Output: Emotion recognition result

[0329] Specific operation: Facial image data is sent to the emotion engine and the results are saved on the device.

[0330] Step 3:

[0331] The device sends the video data acquired from the security camera to the server, which receives the video data and performs preprocessing. Specifically, it adjusts the resolution, trims unnecessary parts, and converts the data into a format that is easy for the generating AI to analyze.

[0332] Input: Video data

[0333] Output: Pre-processed video data

[0334] Specific operation: Receives video data, adjusts resolution and trims unnecessary parts.

[0335] Step 4:

[0336] The server sends the pre-processed video data to the Generator AI for analysis. The Generator AI analyzes the data and evaluates the contents and condition of the baggage (or other object). The results of this analysis are returned to the server.

[0337] Input: Preprocessed video data

[0338] Output: Analysis results

[0339] Specific operation: The generative AI analyzes the video data and sends the results to the server.

[0340] Step 5:

[0341] The server receives the analysis results from the generative AI and determines the baggage status (safe, requires further inspection, etc.), as well as emotion recognition results from the emotion engine, and combines these data to generate appropriate messages and alerts.

[0342] Input: Analysis results and emotion recognition results

[0343] Output: Baggage status and message or alert to user

[0344] Specific operation: Determine the baggage status based on the analysis results and emotion recognition results, and generate a message or alert.

[0345] Step 6:

[0346] The terminal displays a message or alert on the display device based on the baggage status and emotion received from the server. For example, if the baggage is safe and the user is feeling anxious, the message "The environment is safe. Please rest assured" is displayed.

[0347] Input: Baggage status and messages or alerts

[0348] Output: A message or alert that appears on a display device.

[0349] Specific behavior: Display a message or alert on the display device based on the baggage status and emotion.

[0350] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0351] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0352] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0353] [Second embodiment]

[0354] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0355] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0356] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0357] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0358] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0359] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0360] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0361] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0362] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0363] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0364] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0365] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0366] This invention provides a system that includes a baggage inspection device, a generating AI, a server, and a display device to fully automate baggage inspection at airports. This system realizes a series of automated processes, from scanning baggage to analyzing and displaying the results.

[0367] System Overview

[0368] 1. Baggage scanning process

[0369] The terminal acquires video data from the baggage inspection device. This is done by taking images of the baggage using cameras and X-ray devices installed inside the device when the baggage is placed in the inspection device and scanning begins.

[0370] 2. Video data transfer and preprocessing

[0371] The device sends the captured video data to the server, which then performs preprocessing as needed before sending the received video data to the generation AI. Preprocessing is a process that adjusts the image resolution or trims unnecessary parts.

[0372] 3. Analysis by generative AI

[0373] The server sends the preprocessed video data to the generation AI for analysis. The generation AI analyzes the contents and shape of the baggage based on the received video data and evaluates its safety. As a result of the analysis, it determines whether the baggage's status is "passed" or "requires inspection."

[0374] 4. Receiving analysis results and determining status

[0375] The server receives the analysis results from the AI ​​generator and determines the baggage status based on the results. If the status is "passed," the baggage is processed without any problems. On the other hand, if it is determined to require further investigation, an alert is sent to staff.

[0376] 5. Displaying the results

[0377] The terminal displays the baggage status received from the server on the display device. If the baggage has been "passed through," the display will show "passed through" and the user can proceed. If the baggage has been "checked," the display will show "checked," and the user will have to ask airport staff for confirmation.

[0378] Specific examples

[0379] Example 1: If there are no problems with your luggage

[0380] User Action:

[0381] The user places their baggage on the baggage inspection device and presses the scan button.

[0382] Terminal handling:

[0383] The terminal acquires the video data and transmits it to the server.

[0384] Server Action:

[0385] The server sends the video data to the generation AI and receives the analysis results.

[0386] The analysis result is determined to be "pass" and the result is sent to the terminal.

[0387] Terminal display:

[0388] The terminal will display "Passed" and the user can proceed.

[0389] Example 2: There is a problem with your package

[0390] User Action:

[0391] The user places their baggage on the baggage inspection device and presses the scan button.

[0392] Terminal handling:

[0393] The terminal acquires the video data and transmits it to the server.

[0394] Server Action:

[0395] The server sends the video data to the generation AI and receives the analysis results.

[0396] The analysis result is determined to be "required for confirmation," and the result is sent to the terminal.

[0397] Terminal display:

[0398] The terminal will display "Confirmation Required" and the user will need to ask airport staff for confirmation.

[0399] As described above, the present invention can fully automate the baggage inspection process at airports, thereby reducing the need for human resources and improving user convenience.

[0400] The processing flow will be explained below.

[0401] Step 1: The user places their baggage on the baggage inspection device and presses the scan button, which starts scanning the baggage.

[0402] Step 2: The terminal acquires image data from the baggage inspection equipment. Specifically, the on-board camera and X-ray device capture image data of the baggage.

[0403] Step 3: The device sends the acquired video data to the server. At this time, the device confirms that communication with the server has been established and then starts data transfer.

[0404] Step 4: The server receives the video data from the device and preprocesses it into a format that is easy for the AI ​​to analyze. This preprocessing includes adjusting the image size and removing noise.

[0405] Step 5: The server sends the pre-processed video data to the generation AI for analysis. The generation AI receives the data and begins the process of analyzing the contents and shape of the baggage.

[0406] Step 6: The generated AI analyzes the video data and determines the baggage status ("passed" or "requires inspection"). The analysis results are sent back to the server.

[0407] Step 7: The server receives the analysis results from the generation AI and makes a final decision on the baggage status based on the received analysis results.

[0408] Step 8: The server sends the baggage status to the terminal, along with a detailed message about the status (e.g., the reason if it is "required").

[0409] Step 9: The terminal receives the baggage status from the server and displays "Passed" or "Confirmation Required" based on the received status.

[0410] Step 10: The user checks the results displayed on the terminal. If the status is "Passed," proceed as is. If it is "Confirmation Required," ask airport staff for confirmation.

[0411] Example 1

[0412] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0413] Conventional baggage inspection requires manual inspection, which is time-consuming and prone to human error. In addition, baggage analysis requires highly accurate judgment, making it difficult to quickly and accurately determine the safety of baggage.

[0414] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0415] In this invention, the server includes means for acquiring video data from the baggage inspection device, means for transmitting the video data to the server and performing preprocessing, means for transmitting the preprocessed video data to a generation AI and requesting analysis, means for receiving the analysis results from the generation AI and determining the status of the baggage, and means for displaying the status of the baggage on a display device. This automates the inspection process, prevents human errors, and enables fast and accurate baggage screening.

[0416] A "baggage inspection device" is a device that uses X-rays and cameras to obtain image data of the inside of baggage when it is placed there.

[0417] "Video data" is image information showing the internal structure of baggage acquired by a baggage inspection device.

[0418] The "server" is the central device of the system that receives video data, preprocesses it, requests the generation AI to analyze it, and receives the analysis results and determines the status.

[0419] "Preprocessing" refers to the process of adjusting the resolution and trimming the video data to convert it into a format that is easier to analyze.

[0420] "Generative AI" is an artificial intelligence model that analyzes received video data, evaluates the contents and shape of baggage, and determines its safety.

[0421] "Baggage status" refers to the status of the baggage, which is determined to be "passed" or "requires inspection" based on the safety of the baggage analyzed by the generating AI.

[0422] A "display device" is a device that displays the status of baggage to the user and airport staff.

[0423] An "alert" is a notification to alert airport staff when the baggage status is "requires attention."

[0424] This invention aims to improve the efficiency of baggage inspection at airports by proposing a system that includes a baggage inspection device, a generation AI, a server, and a display device. This system can realize a series of automated processes, from scanning baggage to analyzing and displaying the results.

[0425] Specifically, a user places their baggage on the baggage inspection device and presses the scan button. The baggage inspection device is equipped with a high-resolution camera and X-ray device, which capture image data of the baggage. The captured image data is then sent to a server via the terminal.

[0426] The server performs preprocessing on the received video data. This preprocessing includes adjusting the image resolution and trimming unnecessary parts. This preprocessing is performed using image processing libraries such as OpenCV and Pillow. The preprocessed video data is then sent to the generation AI.

[0427] The generating AI uses a model called "CargoScanAI" to analyze the contents and shape of baggage based on preprocessed video data and assess its safety. The analysis determines whether it contains explosives or illegal items. An example of a specific prompt is, "Analyze the video data below and check the contents of the baggage. Please return a decision of 'pass' or 'requires inspection' based on your safety standards."

[0428] The analysis results are sent back to the server, which determines the baggage's status. The baggage's status is evaluated as either "passed" or "requires review." If it is "passed," the baggage is processed without issue and the user can proceed. If it is "requires review," an alert is generated and airport staff are notified.

[0429] Finally, the terminal displays the baggage status received from the server on the display device. The display device displays information according to the status. For example, if the baggage has been "passed through," it will display "passed through" and the user can proceed. If the baggage has been "checked," it will display "checked," and the user will need to ask airport staff for confirmation.

[0430] In this way, the present invention automates the baggage inspection process, eliminating human error while also enabling fast and accurate baggage screening.

[0431] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0432] Step 1:

[0433] The terminal acquires image data from the baggage inspection device. Specifically, when a user places their baggage on the inspection device and presses the scan button, the high-resolution camera and X-ray device built into the device acquire image data of the baggage. The input data is an image of the baggage, and image data of this is generated as the output.

[0434] Step 2:

[0435] The device sends the captured video data to the server. Specifically, the video data is transferred to the server via Wi-Fi or a wired connection. The input is the video data of the baggage, and the output is the data transfer to the server.

[0436] Step 3:

[0437] The server preprocesses the received video data. Specifically, it adjusts the resolution and trims unnecessary parts. For this, it uses image processing libraries such as OpenCV and Pillow. The input is raw video data, and the output is preprocessed video data.

[0438] Step 4:

[0439] The server sends the preprocessed video data to the generation AI and requests it to analyze it. An example of a specific prompt is, "Analyze the following video data and check the contents of the baggage. Please return a decision of 'pass' or 'requires confirmation' based on safety standards." The input is the preprocessed video data, and the output is the generation of an analysis request.

[0440] Step 5:

[0441] The generating AI analyzes the received video data and evaluates the contents and shape of the baggage. As a result of the analysis, the baggage's status is determined to be "passed" or "requires inspection." Specifically, this analysis is performed by the AI ​​model "CargoScanAI." The input is pre-processed video data, and the output is the analysis result.

[0442] Step 6:

[0443] The server receives the analysis results from the generation AI and determines the baggage status based on the results. It sets the status as "passed" or "requires confirmation." The input is the analysis result, and the output is the determined status.

[0444] Step 7:

[0445] If the server determines that the baggage status is "required confirmation," it generates an alert and notifies airport staff. Specifically, the alert message is displayed on a dedicated monitor or terminal. The input is the "required confirmation" status, and the output is the generation of an alert.

[0446] Step 8:

[0447] The terminal displays the baggage status received from the server on a display device. A specific operation is to display a message such as "Passed" or "Confirmation Required" on the display device. The input is the baggage status, and the output is the status displayed on the display device.

[0448] This detailed processing step allows the baggage inspection process to be fast and efficient.

[0449] (Application example 1)

[0450] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0451] Advanced technology is required to quickly and accurately inspect baggage during security checks at airports and large event venues. However, current baggage inspection systems rely on human resources and have issues with both efficiency and accuracy. The present invention aims to solve these issues by fully automating the baggage inspection process.

[0452] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0453] In this invention, the server includes a means for acquiring video data from a baggage inspection device, a means for sending the video data to a generating AI and requesting analysis, and a means for receiving the analysis results from the generating AI and determining the status of the baggage. This enables the automation and efficiency of baggage inspection. The server further includes a means for scanning the video data of the baggage in real time using a smart device, a means for transferring the real-time scanned data to a cloud server, a means for analyzing the data using the generating AI in the cloud server, and a means for displaying the analysis results of the generating AI on a display device of the smart device. This enables fast and accurate advanced baggage inspection in a variety of security situations.

[0454] A "baggage inspection device" is a scanning device installed at airports, event venues, etc. to check the contents and security of baggage.

[0455] "Video data" refers to image information and video information of baggage acquired by the baggage inspection device.

[0456] "Generative AI" is an artificial intelligence technology that analyzes video data of baggage and evaluates its contents and safety.

[0457] A "smart device" is an electronic device that has a built-in camera and display and is capable of acquiring and displaying data, and includes, for example, smart glasses and smartphones.

[0458] A "cloud server" is a remote server that processes and analyzes data via the Internet.

[0459] "Preprocessing" refers to data adjustment operations to convert video data into a format that is easy to analyze.

[0460] "Status" refers to the result of the generating AI's assessment of the baggage's safety, and typically includes a classification such as "passed" or "requires review."

[0461] An "alert" is a warning signal or notification that is issued when baggage status is "needs attention."

[0462] "Real-time scanning" is the process of capturing video data of baggage in real time using a smart device.

[0463] A "display device" is a device that visualizes and displays baggage status and alerts.

[0464] As an embodiment of the present invention, a baggage inspection system using a smart device will be described.

[0465] Server Processing

[0466] The server receives video data acquired from the baggage inspection equipment. This video data is scanned in real time by a camera built into the smart device. Once the video data is acquired, the server performs preprocessing such as adjusting the image resolution and trimming unnecessary parts. This preprocessing converts the data into a format that is easy for the generative AI model to analyze.

[0467] Analysis by generative AI

[0468] The server sends the preprocessed video data to the generative AI and asks it to evaluate the contents of the baggage and its safety. The generative AI model analyzes this data and determines its status as safe or unsafe. Once the analysis results are available, the server receives them.

[0469] Anomaly detection and alerts

[0470] The analysis results determine whether the baggage should be allowed to pass or requires further investigation, and if it does, the server immediately sends an alert to the smart device. This alert indicates that there is a potential risk to the baggage, requiring a prompt response.

[0471] Displaying the results

[0472] The analysis results from the server are reflected on the smart device's display. When the user places their baggage on the baggage inspection device, the smart device's display will show either "Pass" or "Inspection Required."

[0473] Hardware and software used

[0474] Smart Device: Electronic devices with built-in cameras and displays, such as smart glasses and smartphones.

[0475] Cloud server: A high-performance server that processes data and analyzes generative AI models.

[0476] Generative AI model: Artificial intelligence for analyzing video data and assessing safety.

[0477] Prompt Sentence Examples

[0478] The following are examples of prompts that can be passed to the generation AI:

[0479] "Analyze the video data to determine whether dangerous items are present in the baggage and assess its safety."

[0480] Preprocessed video data is sent along with this prompt, and the generative AI model analyzes it.

[0481] Specific examples

[0482] For example, if this system is implemented at the entrance to a large event, the smart glasses will capture video data in real time as attendees scan their belongings and send it to a server. The server will preprocess the data and request analysis from a generative AI model. If the analysis result is "pass," entry is permitted; if it is "verification required," an alert will be issued immediately, requiring appropriate action. This will enable smooth and efficient security checks at large events.

[0483] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0484] Step 1:

[0485] The terminal captures real-time video data of baggage using the camera of a smart device (e.g., smart glasses). The input is the actual visual information of the baggage, which is captured as video data. The output is the captured video data of the baggage.

[0486] Step 2:

[0487] The device sends the captured video data to a cloud server via Wi-Fi. The input is the video data captured on the device, and the output is the video data sent to the cloud server. Data conversion and compression may occur during this process.

[0488] Step 3:

[0489] The server preprocesses the received video data. Specifically, it adjusts the image resolution and trims unnecessary parts. The input is the video data sent from the terminal, and the output is the preprocessed video data.

[0490] Step 4:

[0491] The server sends the preprocessed video data to the generation AI and requests it to analyze it. The input is the preprocessed video data, and the output is the data sent to the generation AI and the analysis results. At this time, the prompt statement "Analyze the video data to determine whether the baggage contains dangerous items and evaluate its safety" is used.

[0492] Step 5:

[0493] The generative AI analyzes the received video data and evaluates the safety of the baggage. The input is the preprocessed video data and a prompt, and the output is the baggage status ("passed" or "requires inspection"). The generative AI uses a deep learning model to analyze the images and determine safety.

[0494] Step 6:

[0495] The server processes the analysis results received from the generation AI and determines the baggage status. The input is the analysis results from the generation AI, and the output is the final baggage status. This status is either "passed" or "requires confirmation."

[0496] Step 7:

[0497] The server sends the confirmed baggage status to the terminal. The input is the final baggage status, and the output is a notification to the terminal. If the baggage status is "confirmation required," alert information is also included.

[0498] Step 8:

[0499] The terminal receives the baggage status from the server and displays it on the display device. The input is the baggage status sent from the server, and the output is the notification displayed on the display. The user checks the notification and takes action as necessary.

[0500] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0501] This invention provides a system that fully automates airport baggage inspection and improves service quality by recognizing user emotions. The system includes a baggage inspection device, a generative AI, a server, a display device, and an emotion engine. This system automates a series of processes, from scanning baggage to analyzing the results, displaying the results, and responding to user emotions.

[0502] System Overview

[0503] 1. Baggage scanning process

[0504] The user places their baggage on the baggage inspection device and presses the scan button. The terminal acquires video data from the baggage inspection device, which then uses the onboard camera and X-ray device to capture image data of the baggage.

[0505] 2. Video data transfer and preprocessing

[0506] The device sends the captured video data to the server, which then performs preprocessing as needed before sending the received video data to the generation AI. Preprocessing is a process that adjusts the image resolution or trims unnecessary parts.

[0507] 3. Analysis by generative AI

[0508] The server sends the preprocessed video data to the generation AI for analysis. The generation AI analyzes the contents and shape of the baggage based on the received video data and evaluates its safety. As a result of the analysis, it determines whether the baggage's status is "passed" or "requires inspection."

[0509] 4. Receiving analysis results and determining status

[0510] The server receives the analysis results from the AI ​​generator and determines the baggage status based on the results. If the status is "passed," the baggage is processed without any problems. On the other hand, if it is determined to require further investigation, an alert is sent to staff.

[0511] 5. User Emotion Recognition by Emotion Engine

[0512] The device captures the user's face with a camera and sends it to the emotion engine. The emotion engine analyzes the user's facial expressions and recognizes the user's emotions (e.g., relief, surprise, anxiety, etc.). The recognized emotions are then sent to the server.

[0513] 6. Displaying results and addressing emotions

[0514] The terminal displays the results on a display device based on the baggage status received from the server and the user's emotional state. If the status is "passed," a message corresponding to the emotion, such as "relief," is displayed. If the status is "requires attention," an alert corresponding to the emotion is displayed, and staff will respond as necessary.

[0515] Specific examples

[0516] Example 1: If there are no problems with your luggage

[0517] User Action:

[0518] The user places their baggage on the baggage inspection device and presses the scan button.

[0519] The user's face is captured by a camera.

[0520] Terminal handling:

[0521] The terminal acquires the video data and transmits it to the server.

[0522] The terminal transmits the user's facial image to the emotion engine.

[0523] Server Action:

[0524] The server sends the video data to the generation AI and receives the analysis results.

[0525] The server receives the recognized emotion data from the emotion engine.

[0526] If the analysis result is determined to be "pass," a message based on the emotion is generated and sent to the device.

[0527] Terminal display:

[0528] The device will display "Passed" and a reassuring message.

[0529] The user can proceed.

[0530] Example 2: There is a problem with your package

[0531] User Action:

[0532] The user places their baggage on the baggage inspection device and presses the scan button.

[0533] The user's face is captured by a camera.

[0534] Terminal handling:

[0535] The terminal acquires the video data and transmits it to the server.

[0536] The terminal transmits the user's facial image to the emotion engine.

[0537] Server Action:

[0538] The server sends the video data to the generation AI and receives the analysis results.

[0539] The server receives the recognized emotion data from the emotion engine.

[0540] If the analysis results indicate that "attention is required," an emotion-based alert is generated and sent to the device.

[0541] Terminal display:

[0542] The device will display a message saying "Check required" and will try to ease any fears.

[0543] The user will ask airport staff for confirmation.

[0544] In this manner, the present invention can fully automate the baggage inspection process at airports and improve the quality of service by responding to user emotions.

[0545] The processing flow will be explained below.

[0546] Step 1: The user places their baggage on the baggage inspection device and presses the scan button, which starts scanning the baggage.

[0547] Step 2: The terminal acquires image data from the baggage inspection device. Image data of the baggage is captured by a camera or X-ray device and acquired as data.

[0548] Step 3: The device sends the acquired video data to the server. At this time, the device confirms that communication with the server has been established before transferring the data.

[0549] Step 4: The server preprocesses the video data received from the device. This involves adjusting the image resolution and trimming unnecessary parts, converting it into a format that is easy for the generating AI to analyze.

[0550] Step 5: The server sends the preprocessed video data to the generation AI for analysis. The generation AI analyzes the contents and shape of the baggage based on the received video data and determines its safety.

[0551] Step 6: The generating AI analyzes the data, determines the baggage status ('passed' or 'requires review'), and sends the result back to the server.

[0552] Step 7: The server receives the analysis results from the generation AI and makes a final decision on the baggage status based on the analysis results.

[0553] Step 8: The server receives the user's facial image from the device's camera and sends it to the emotion engine, which analyzes the facial expression and recognizes the user's emotional state (e.g., relief, anxiety, surprise, etc.).

[0554] Step 9: The emotion engine sends the recognized user emotion data back to the server, which uses this data along with the analysis results.

[0555] Step 10: The server generates an appropriate message or alert based on the baggage status and the user's emotions. If the baggage has passed, the server generates a message that provides reassurance, and if the baggage needs attention, the server generates a message that includes instructions to elicit reassurance.

[0556] Step 11: The server sends the generated message or alert to the terminal, which receives it and displays it on its display device.

[0557] Step 12: The terminal displays the baggage status and a message or alert depending on the user's emotion. If the baggage is "passed through," it displays "Proceed with confidence." If the baggage is "checked," it displays a message such as "Your baggage needs to be checked. Please contact an attendant."

[0558] Step 13: The user checks the displayed result. If it is "Pass", proceed as is. If it is "Confirmation Required", ask airport staff for confirmation.

[0559] Example 2

[0560] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0561] Conventional airport baggage inspection systems have the problem that they are unable to fully alleviate users' anxiety and dissatisfaction with inspection results because they are unable to accurately analyze the contents of baggage and respond to user emotions. To solve this problem, a system is needed that can accurately analyze baggage contents and display inspection results quickly and appropriately. Furthermore, it is necessary to improve the quality of service by recognizing users' emotions and responding accordingly.

[0562] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0563] In this invention, the server includes means for acquiring video data from a baggage inspection device, means for sending the video data to a generation AI and requesting analysis, means for receiving the analysis results from the generation AI and determining the status of the baggage, means for capturing the user's face and recognizing their emotions, and means for displaying the baggage status and the user's emotions on a display device, thereby enabling accurate analysis of the contents of the baggage, prompt display of inspection results, and response appropriate to the user's emotions.

[0564] A "baggage inspection device" is a device used at airports and other locations to scan baggage using X-rays or cameras to check its contents.

[0565] "Video data" refers to data that includes visual information about baggage, such as camera images or X-ray images acquired by baggage inspection equipment.

[0566] "Generative AI" is a technology that uses artificial intelligence to analyze video data and evaluate the contents, shape, and safety of baggage.

[0567] The "analysis results" are the information output by the generating AI after analyzing the video data, and indicate the status of the baggage.

[0568] "Baggage status" refers to whether the baggage can be passed through and whether any additional inspection is required, based on the analysis results of the generated AI.

[0569] "User" refers to a person who uses the baggage inspection system and performs the baggage scanning operation.

[0570] "Capturing a face" refers to obtaining an image of a user's face using a camera.

[0571] "Emotion recognition" means analyzing the user's facial expressions and identifying their emotional state, such as relief, surprise, or anxiety, from those expressions.

[0572] A "display device" is a device for visually displaying information such as baggage status and the user's emotional state.

[0573] This invention provides a system that fully automates airport baggage inspection and improves service quality by recognizing user emotions. The system includes a baggage inspection device, a generative AI, a server, a display device, and an emotion engine. This system automates a series of processes, from scanning baggage to analyzing the results, displaying the results, and responding to user emotions.

[0574] The operation of the system is as follows.

[0575] First, the user places their baggage on the baggage inspection device and presses the scan button. The terminal acquires video data from the baggage inspection device, which is then sent to the server. The server pre-processes the video data, which includes adjusting the image resolution and trimming unnecessary parts. Once pre-processing is complete, the server sends the video data to a generation AI, which analyzes the contents and shape of the baggage to assess its safety. As a result of the analysis, the baggage's status is determined as either "passed" or "requires inspection."

[0576] The results of this analysis are sent back to the server, which determines the baggage's status. If the status is "passed," the baggage is automatically processed and the user can pass through. On the other hand, if the status is "requires attention," an alert is generated and staff will respond.

[0577] At the same time, the device captures the user's face and sends the video data to the emotion engine. The emotion engine analyzes the user's facial expressions and recognizes the user's emotional state (e.g., relief, surprise, anxiety, etc.). The recognition results are sent to the server, which then integrates the baggage status and the user's emotional state and sends them to the device. The device then displays the integrated results on a display device, with a message appropriate for the user.

[0578] For example, when a user places his / her baggage on an inspection device and presses the scan button, the following process is carried out.

[0579] 1. The terminal acquires video data of the baggage and sends it to the server.

[0580] 2. The server preprocesses the video data and sends it to the generation AI.

[0581] 3. The generated AI analyzes the video data and may determine the baggage's status as "passed" or "requires inspection."

[0582] 4. At the same time, the device captures the user's face and sends it to the emotion engine, which then recognizes the user's emotions, such as "relief" or "anxiety."

[0583] 5. The server integrates the baggage status and emotion data and sends it to the terminal.

[0584] 6. The terminal displays the integrated results on the display device and displays an appropriate message to the user.

[0585] For example, consider the following prompt:

[0586] "Please begin baggage inspection scan."

[0587] "Package analysis results: Pass or check required?"

[0588] "Capture the user's face with a camera and recognize their emotions."

[0589] As described above, this system fully automates the baggage inspection process and makes it possible to provide services that adapt to the user's emotions.

[0590] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0591] Step 1:

[0592] The user places their baggage on the baggage inspection device and presses the scan button. This causes the baggage inspection device to acquire image data of the baggage (camera images and X-ray images). The input is the baggage, and the output is image data of the baggage. Specifically, the baggage inspection device starts scanning in response to the user's operation.

[0593] Step 2:

[0594] The terminal transmits the video data acquired from the baggage inspection device to the server. The input is the video data of the baggage, and the output is the data transmitted to the server. Specifically, the terminal transmits the video data to the server using network communication.

[0595] Step 3:

[0596] The server preprocesses the received video data. This preprocessing includes adjusting the image resolution and trimming unnecessary parts. The input is the video data sent to the server, and the output is the preprocessed video data. Specifically, the server processes the data using an image processing algorithm.

[0597] Step 4:

[0598] The server sends the pre-processed video data to the generation AI. The input is the pre-processed video data, and the output is the data sent to the generation AI. Specifically, the server communicates with the generation AI and sends the data.

[0599] Step 5:

[0600] The generating AI analyzes the contents and shape of the baggage based on the received video data and evaluates its safety. As a result of the analysis, the baggage's status is determined to be "passed" or "requires inspection." The input is the video data sent to the generating AI, and the output is the baggage's status. Specifically, the generating AI analyzes the data using a machine learning algorithm.

[0601] Step 6:

[0602] The server receives the analysis results from the generation AI and determines the baggage status based on the results. If the status is "passed," the baggage is processed without any problems. If the status is "requires attention," an alert is raised and staff are notified. The input is the analysis results from the generation AI, and the output is the baggage status and an alert if necessary. Specifically, the server processes the analysis results and determines the status.

[0603] Step 7:

[0604] The device captures the user's face with a camera and sends the video data to the emotion engine. The input is the user's face video data, and the output is the data sent to the emotion engine. In concrete terms, the device captures the face with a camera and sends the data.

[0605] Step 8:

[0606] The emotion engine analyzes the user's facial expressions and recognizes the user's emotions (relief, surprise, anxiety, etc.). The recognized emotional state is sent to the server. The input is the facial image data sent to the emotion engine, and the output is the analyzed emotional data. In concrete terms, the emotion engine uses a facial expression analysis algorithm.

[0607] Step 9:

[0608] The server integrates the baggage status and emotion data and sends it to the terminal. The input is the baggage status and emotion data, and the output is the integrated result data. Specifically, the server performs the integration calculation to generate the result.

[0609] Step 10:

[0610] The terminal displays the integrated result on the display device. For example, if the status is "passed" and the user's emotion is "relieved," the terminal will display that fact and the user can proceed. If the status is "confirmation required," the terminal will display an alert and the user will ask the staff for confirmation. The input is the integrated result data, and the output is the result displayed on the display device. As a specific operation, the terminal will display an appropriate message on the display.

[0611] The above is the processing flow of this system.

[0612] (Application example 2)

[0613] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0614] Conventional baggage inspection systems automate the display of inspection results and the issuance of alerts, but they have the problem of being unable to respond in a way that takes the user's emotions into consideration. This can cause users to feel anxious or uncomfortable, and there is a need to improve the quality of service. Meanwhile, smart home security systems are also required to accurately grasp the security situation and respond appropriately according to the user's emotions. To address these issues, systems that can reduce user anxiety and ensure safety are needed not only in baggage inspection systems but also in smart home security.

[0615] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0616] In this invention, the server includes means for acquiring video data from the baggage inspection device, means for sending the video data to the generation AI and requesting analysis, means for receiving the analysis results from the generation AI and determining the status of the baggage, means for capturing the user's face using a camera and sending the results to the emotion engine, means for analyzing the user's emotions and sending the results to the server, and means for displaying an appropriate message or alert on a display device based on the emotions. This makes it possible to respond according to the user's emotions and provide a more human-like, high-quality service.

[0617] "Baggage screening device" means a device used at airports and other security checkpoints to scan baggage and verify its contents.

[0618] "Video data" refers to digital image and video data acquired from baggage inspection equipment and cameras.

[0619] "Generative AI" is an artificial intelligence technology that analyzes acquired video data and independently determines its content and characteristics.

[0620] The "analysis results" are information about the contents and condition of baggage generated after the generating AI analyzes the video data.

[0621] "Status" is information indicating the safety of baggage and the status of inspection results, and includes indicators such as "passed" and "requires confirmation."

[0622] A "display device" is a device for visually displaying baggage inspection results and messages based on the user's emotions.

[0623] An "emotion engine" is a software system that analyzes a user's face and recognizes emotions (such as relief, surprise, or anxiety).

[0624] "Capture" is the act of acquiring image or video data using a device such as a camera.

[0625] A "server" is a computer system that processes and manages data for the entire system.

[0626] "Preprocessing" refers to processes such as adjusting resolution and trimming unnecessary parts, which are carried out to convert video data into a format that is easier to analyze.

[0627] A "message" refers to information or notifications provided to a user, the content of which is adjusted based on emotions.

[0628] An "alert" is a notification that warns users or staff when the baggage status is "requires attention" or when other abnormalities occur.

[0629] The smart home security system according to the present invention is implemented by the following procedure.

[0630] System Overview

[0631] The system consists of the following major hardware and software components:

[0632] Security camera (for acquiring video data)

[0633] Emotion engine (for recognizing user emotions)

[0634] Generative AI (for video data analysis)

[0635] Server (data processing and management)

[0636] Display device (for displaying results)

[0637] 1. Acquisition and transmission of video data

[0638] First, the device (smart home system) acquires video data in real time using a security camera. When a user enters the camera's range, their face is captured and sent to the emotion engine for emotion recognition. In parallel, the entire video data is sent to the server.

[0639] 2. Emotion recognition and sending results

[0640] The emotion engine analyzes the facial image and recognizes the user's emotions (relief, surprise, anxiety, etc.). The recognition results are sent to the server, which processes the data.

[0641] 3. Video data preprocessing and analysis

[0642] The server performs preprocessing such as adjusting the resolution and trimming unnecessary parts of the video data received from the security camera before sending it to the generation AI. The preprocessed data is sent to the generation AI for analysis. The generation AI evaluates security risks (suspicious movements, abnormal behavior, etc.) based on the video data and returns the analysis results to the server.

[0643] 4. Processing and displaying results

[0644] The server determines the security status based on the analysis results received from the generation AI. If the status is "safe," the result and a message based on the user's emotion are displayed on the display device. If the status is "requires attention," an alert is issued and a message based on the emotion is displayed, prompting the user to take appropriate action if necessary.

[0645] Specific examples

[0646] For example, the flow when a security camera detects movement late at night is as follows: The camera footage is sent to the server, and if the generation AI judges it to be "safe," the server displays the message "The environment is safe. Please be at ease," even if the emotion engine recognizes "anxiety." If the generation AI judges there to be a "potential threat," the server will display the message "A potential threat has been detected. Please respond with caution," even if the emotion engine recognizes "anxiety."

[0647] Prompt Sentence Examples

[0648] An example of a prompt sent to the generative AI model is as follows:

[0649] "Please analyze the baggage scan footage to determine if it contains any suspicious items. The video data is as follows."

[0650] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0651] Step 1:

[0652] The device captures video data in real time using a security camera. When a user enters the camera's range, their face is automatically captured. The captured video data and facial image are then stored in the device's memory.

[0653] Input: Real-time footage from security cameras

[0654] Output: Video data and facial images are saved on the device.

[0655] Specific operation: The security camera detects movement, captures video data, and stores it in the device's memory.

[0656] Step 2:

[0657] The device sends the stored facial image data to the emotion engine, which analyzes the user's face and recognizes emotions (relief, surprise, anxiety, etc.). The emotion recognition results are returned to the device, which receives and stores the results.

[0658] Input: Facial image data

[0659] Output: Emotion recognition result

[0660] Specific operation: Facial image data is sent to the emotion engine and the results are saved on the device.

[0661] Step 3:

[0662] The device sends the video data acquired from the security camera to the server, which receives the video data and performs preprocessing. Specifically, it adjusts the resolution, trims unnecessary parts, and converts the data into a format that is easy for the generating AI to analyze.

[0663] Input: Video data

[0664] Output: Pre-processed video data

[0665] Specific operation: Receives video data, adjusts resolution and trims unnecessary parts.

[0666] Step 4:

[0667] The server sends the pre-processed video data to the Generator AI for analysis. The Generator AI analyzes the data and evaluates the contents and condition of the baggage (or other object). The results of this analysis are returned to the server.

[0668] Input: Preprocessed video data

[0669] Output: Analysis results

[0670] Specific operation: The generative AI analyzes the video data and sends the results to the server.

[0671] Step 5:

[0672] The server receives the analysis results from the generative AI and determines the baggage status (safe, requires further inspection, etc.), as well as emotion recognition results from the emotion engine, and combines these data to generate appropriate messages and alerts.

[0673] Input: Analysis results and emotion recognition results

[0674] Output: Baggage status and message or alert to user

[0675] Specific operation: Determine the baggage status based on the analysis results and emotion recognition results, and generate a message or alert.

[0676] Step 6:

[0677] The terminal displays a message or alert on the display device based on the baggage status and emotion received from the server. For example, if the baggage is safe and the user is feeling anxious, the message "The environment is safe. Please rest assured" is displayed.

[0678] Input: Baggage status and messages or alerts

[0679] Output: A message or alert that appears on a display device.

[0680] Specific behavior: Display a message or alert on the display device based on the baggage status and emotion.

[0681] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0682] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0683] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0684] [Third embodiment]

[0685] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0686] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0687] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0688] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0689] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0690] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0691] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0692] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0693] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0694] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0695] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0696] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0697] This invention provides a system that includes a baggage inspection device, a generating AI, a server, and a display device to fully automate baggage inspection at airports. This system realizes a series of automated processes, from scanning baggage to analyzing and displaying the results.

[0698] System Overview

[0699] 1. Baggage scanning process

[0700] The terminal acquires video data from the baggage inspection device. This is done by taking images of the baggage using cameras and X-ray devices installed inside the device when the baggage is placed in the inspection device and scanning begins.

[0701] 2. Video data transfer and preprocessing

[0702] The device sends the captured video data to the server, which then performs preprocessing as needed before sending the received video data to the generation AI. Preprocessing is a process that adjusts the image resolution or trims unnecessary parts.

[0703] 3. Analysis by generative AI

[0704] The server sends the preprocessed video data to the generation AI for analysis. The generation AI analyzes the contents and shape of the baggage based on the received video data and evaluates its safety. As a result of the analysis, it determines whether the baggage's status is "passed" or "requires inspection."

[0705] 4. Receiving analysis results and determining status

[0706] The server receives the analysis results from the AI ​​generator and determines the baggage status based on the results. If the status is "passed," the baggage is processed without any problems. On the other hand, if it is determined to require further investigation, an alert is sent to staff.

[0707] 5. Displaying the results

[0708] The terminal displays the baggage status received from the server on the display device. If the baggage has been "passed through," the display will show "passed through" and the user can proceed. If the baggage has been "checked," the display will show "checked," and the user will have to ask airport staff for confirmation.

[0709] Specific examples

[0710] Example 1: If there are no problems with your luggage

[0711] User Action:

[0712] The user places their baggage on the baggage inspection device and presses the scan button.

[0713] Terminal handling:

[0714] The terminal acquires the video data and transmits it to the server.

[0715] Server Action:

[0716] The server sends the video data to the generation AI and receives the analysis results.

[0717] The analysis result is determined to be "pass" and the result is sent to the terminal.

[0718] Terminal display:

[0719] The terminal will display "Passed" and the user can proceed.

[0720] Example 2: There is a problem with your package

[0721] User Action:

[0722] The user places their baggage on the baggage inspection device and presses the scan button.

[0723] Terminal handling:

[0724] The terminal acquires the video data and transmits it to the server.

[0725] Server Action:

[0726] The server sends the video data to the generation AI and receives the analysis results.

[0727] The analysis result is determined to be "required for confirmation," and the result is sent to the terminal.

[0728] Terminal display:

[0729] The terminal will display "Confirmation Required" and the user will need to ask airport staff for confirmation.

[0730] As described above, the present invention can fully automate the baggage inspection process at airports, thereby reducing the need for human resources and improving user convenience.

[0731] The processing flow will be explained below.

[0732] Step 1: The user places their baggage on the baggage inspection device and presses the scan button, which starts scanning the baggage.

[0733] Step 2: The terminal acquires image data from the baggage inspection equipment. Specifically, the on-board camera and X-ray device capture image data of the baggage.

[0734] Step 3: The device sends the acquired video data to the server. At this time, the device confirms that communication with the server has been established and then starts data transfer.

[0735] Step 4: The server receives the video data from the device and preprocesses it into a format that is easy for the AI ​​to analyze. This preprocessing includes adjusting the image size and removing noise.

[0736] Step 5: The server sends the pre-processed video data to the generation AI for analysis. The generation AI receives the data and begins the process of analyzing the contents and shape of the baggage.

[0737] Step 6: The generated AI analyzes the video data and determines the baggage status ("passed" or "requires inspection"). The analysis results are sent back to the server.

[0738] Step 7: The server receives the analysis results from the generation AI and makes a final decision on the baggage status based on the received analysis results.

[0739] Step 8: The server sends the baggage status to the terminal, along with a detailed message about the status (e.g., the reason if it is "required").

[0740] Step 9: The terminal receives the baggage status from the server and displays "Passed" or "Confirmation Required" based on the received status.

[0741] Step 10: The user checks the results displayed on the terminal. If the status is "Passed," proceed as is. If it is "Confirmation Required," ask airport staff for confirmation.

[0742] Example 1

[0743] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0744] Conventional baggage inspection requires manual inspection, which is time-consuming and prone to human error. In addition, baggage analysis requires highly accurate judgment, making it difficult to quickly and accurately determine the safety of baggage.

[0745] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0746] In this invention, the server includes means for acquiring video data from the baggage inspection device, means for transmitting the video data to the server and performing preprocessing, means for transmitting the preprocessed video data to a generation AI and requesting analysis, means for receiving the analysis results from the generation AI and determining the status of the baggage, and means for displaying the status of the baggage on a display device. This automates the inspection process, prevents human errors, and enables fast and accurate baggage screening.

[0747] A "baggage inspection device" is a device that uses X-rays and cameras to obtain image data of the inside of baggage when it is placed there.

[0748] "Video data" is image information showing the internal structure of baggage acquired by a baggage inspection device.

[0749] The "server" is the central device of the system that receives video data, preprocesses it, requests the generation AI to analyze it, and receives the analysis results and determines the status.

[0750] "Preprocessing" refers to the process of adjusting the resolution and trimming the video data to convert it into a format that is easier to analyze.

[0751] "Generative AI" is an artificial intelligence model that analyzes received video data, evaluates the contents and shape of baggage, and determines its safety.

[0752] "Baggage status" refers to the status of the baggage, which is determined to be "passed" or "requires inspection" based on the safety of the baggage analyzed by the generating AI.

[0753] A "display device" is a device that displays the status of baggage to the user and airport staff.

[0754] An "alert" is a notification to alert airport staff when the baggage status is "requires attention."

[0755] This invention aims to improve the efficiency of baggage inspection at airports by proposing a system that includes a baggage inspection device, a generation AI, a server, and a display device. This system can realize a series of automated processes, from scanning baggage to analyzing and displaying the results.

[0756] Specifically, a user places their baggage on the baggage inspection device and presses the scan button. The baggage inspection device is equipped with a high-resolution camera and X-ray device, which capture image data of the baggage. The captured image data is then sent to a server via the terminal.

[0757] The server performs preprocessing on the received video data. This preprocessing includes adjusting the image resolution and trimming unnecessary parts. This preprocessing is performed using image processing libraries such as OpenCV and Pillow. The preprocessed video data is then sent to the generation AI.

[0758] The generating AI uses a model called "CargoScanAI" to analyze the contents and shape of baggage based on preprocessed video data and assess its safety. The analysis determines whether it contains explosives or illegal items. An example of a specific prompt is, "Analyze the video data below and check the contents of the baggage. Please return a decision of 'pass' or 'requires inspection' based on your safety standards."

[0759] The analysis results are sent back to the server, which determines the baggage's status. The baggage's status is evaluated as either "passed" or "requires review." If it is "passed," the baggage is processed without issue and the user can proceed. If it is "requires review," an alert is generated and airport staff are notified.

[0760] Finally, the terminal displays the baggage status received from the server on the display device. The display device displays information according to the status. For example, if the baggage has been "passed through," it will display "passed through" and the user can proceed. If the baggage has been "checked," it will display "checked," and the user will need to ask airport staff for confirmation.

[0761] In this way, the present invention automates the baggage inspection process, eliminating human error while also enabling fast and accurate baggage screening.

[0762] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0763] Step 1:

[0764] The terminal acquires image data from the baggage inspection device. Specifically, when a user places their baggage on the inspection device and presses the scan button, the high-resolution camera and X-ray device built into the device acquire image data of the baggage. The input data is an image of the baggage, and image data of this is generated as the output.

[0765] Step 2:

[0766] The device sends the captured video data to the server. Specifically, the video data is transferred to the server via Wi-Fi or a wired connection. The input is the video data of the baggage, and the output is the data transfer to the server.

[0767] Step 3:

[0768] The server preprocesses the received video data. Specifically, it adjusts the resolution and trims unnecessary parts. For this, it uses image processing libraries such as OpenCV and Pillow. The input is raw video data, and the output is preprocessed video data.

[0769] Step 4:

[0770] The server sends the preprocessed video data to the generation AI and requests it to analyze it. An example of a specific prompt is, "Analyze the following video data and check the contents of the baggage. Please return a decision of 'pass' or 'requires confirmation' based on safety standards." The input is the preprocessed video data, and the output is the generation of an analysis request.

[0771] Step 5:

[0772] The generating AI analyzes the received video data and evaluates the contents and shape of the baggage. As a result of the analysis, the baggage's status is determined to be "passed" or "requires inspection." Specifically, this analysis is performed by the AI ​​model "CargoScanAI." The input is pre-processed video data, and the output is the analysis result.

[0773] Step 6:

[0774] The server receives the analysis results from the generation AI and determines the baggage status based on the results. It sets the status as "passed" or "requires confirmation." The input is the analysis result, and the output is the determined status.

[0775] Step 7:

[0776] If the server determines that the baggage status is "required confirmation," it generates an alert and notifies airport staff. Specifically, the alert message is displayed on a dedicated monitor or terminal. The input is the "required confirmation" status, and the output is the generation of an alert.

[0777] Step 8:

[0778] The terminal displays the baggage status received from the server on a display device. A specific operation is to display a message such as "Passed" or "Confirmation Required" on the display device. The input is the baggage status, and the output is the status displayed on the display device.

[0779] This detailed processing step allows the baggage inspection process to be fast and efficient.

[0780] (Application example 1)

[0781] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0782] Advanced technology is required to quickly and accurately inspect baggage during security checks at airports and large event venues. However, current baggage inspection systems rely on human resources and have issues with both efficiency and accuracy. The present invention aims to solve these issues by fully automating the baggage inspection process.

[0783] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0784] In this invention, the server includes a means for acquiring video data from a baggage inspection device, a means for sending the video data to a generating AI and requesting analysis, and a means for receiving the analysis results from the generating AI and determining the status of the baggage. This enables the automation and efficiency of baggage inspection. The server further includes a means for scanning the video data of the baggage in real time using a smart device, a means for transferring the real-time scanned data to a cloud server, a means for analyzing the data using the generating AI in the cloud server, and a means for displaying the analysis results of the generating AI on a display device of the smart device. This enables fast and accurate advanced baggage inspection in a variety of security situations.

[0785] A "baggage inspection device" is a scanning device installed at airports, event venues, etc. to check the contents and security of baggage.

[0786] "Video data" refers to image information and video information of baggage acquired by the baggage inspection device.

[0787] "Generative AI" is an artificial intelligence technology that analyzes video data of baggage and evaluates its contents and safety.

[0788] A "smart device" is an electronic device that has a built-in camera and display and is capable of acquiring and displaying data, and includes, for example, smart glasses and smartphones.

[0789] A "cloud server" is a remote server that processes and analyzes data via the Internet.

[0790] "Preprocessing" refers to data adjustment operations to convert video data into a format that is easy to analyze.

[0791] "Status" refers to the result of the generating AI's assessment of the baggage's safety, and typically includes a classification such as "passed" or "requires review."

[0792] An "alert" is a warning signal or notification that is issued when baggage status is "needs attention."

[0793] "Real-time scanning" is the process of capturing video data of baggage in real time using a smart device.

[0794] A "display device" is a device that visualizes and displays baggage status and alerts.

[0795] As an embodiment of the present invention, a baggage inspection system using a smart device will be described.

[0796] Server Processing

[0797] The server receives video data acquired from the baggage inspection equipment. This video data is scanned in real time by a camera built into the smart device. Once the video data is acquired, the server performs preprocessing such as adjusting the image resolution and trimming unnecessary parts. This preprocessing converts the data into a format that is easy for the generative AI model to analyze.

[0798] Analysis by generative AI

[0799] The server sends the preprocessed video data to the generative AI and asks it to evaluate the contents of the baggage and its safety. The generative AI model analyzes this data and determines its status as safe or unsafe. Once the analysis results are available, the server receives them.

[0800] Anomaly detection and alerts

[0801] The analysis results determine whether the baggage should be allowed to pass or requires further investigation, and if it does, the server immediately sends an alert to the smart device. This alert indicates that there is a potential risk to the baggage, requiring a prompt response.

[0802] Displaying the results

[0803] The analysis results from the server are reflected on the smart device's display. When the user places their baggage on the baggage inspection device, the smart device's display will show either "Pass" or "Inspection Required."

[0804] Hardware and software used

[0805] Smart Device: Electronic devices with built-in cameras and displays, such as smart glasses and smartphones.

[0806] Cloud server: A high-performance server that processes data and analyzes generative AI models.

[0807] Generative AI model: Artificial intelligence for analyzing video data and assessing safety.

[0808] Prompt Sentence Examples

[0809] The following are examples of prompts that can be passed to the generation AI:

[0810] "Analyze the video data to determine whether dangerous items are present in the baggage and assess its safety."

[0811] Preprocessed video data is sent along with this prompt, and the generative AI model analyzes it.

[0812] Specific examples

[0813] For example, if this system is implemented at the entrance to a large event, the smart glasses will capture video data in real time as attendees scan their belongings and send it to a server. The server will preprocess the data and request analysis from a generative AI model. If the analysis result is "pass," entry is permitted; if it is "verification required," an alert will be issued immediately, requiring appropriate action. This will enable smooth and efficient security checks at large events.

[0814] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0815] Step 1:

[0816] The terminal captures real-time video data of baggage using the camera of a smart device (e.g., smart glasses). The input is the actual visual information of the baggage, which is captured as video data. The output is the captured video data of the baggage.

[0817] Step 2:

[0818] The device sends the captured video data to a cloud server via Wi-Fi. The input is the video data captured on the device, and the output is the video data sent to the cloud server. Data conversion and compression may occur during this process.

[0819] Step 3:

[0820] The server preprocesses the received video data. Specifically, it adjusts the image resolution and trims unnecessary parts. The input is the video data sent from the terminal, and the output is the preprocessed video data.

[0821] Step 4:

[0822] The server sends the preprocessed video data to the generation AI and requests it to analyze it. The input is the preprocessed video data, and the output is the data sent to the generation AI and the analysis results. At this time, the prompt statement "Analyze the video data to determine whether the baggage contains dangerous items and evaluate its safety" is used.

[0823] Step 5:

[0824] The generative AI analyzes the received video data and evaluates the safety of the baggage. The input is the preprocessed video data and a prompt, and the output is the baggage status ("passed" or "requires inspection"). The generative AI uses a deep learning model to analyze the images and determine safety.

[0825] Step 6:

[0826] The server processes the analysis results received from the generation AI and determines the baggage status. The input is the analysis results from the generation AI, and the output is the final baggage status. This status is either "passed" or "requires confirmation."

[0827] Step 7:

[0828] The server sends the confirmed baggage status to the terminal. The input is the final baggage status, and the output is a notification to the terminal. If the baggage status is "confirmation required," alert information is also included.

[0829] Step 8:

[0830] The terminal receives the baggage status from the server and displays it on the display device. The input is the baggage status sent from the server, and the output is the notification displayed on the display. The user checks the notification and takes action as necessary.

[0831] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0832] This invention provides a system that fully automates airport baggage inspection and improves service quality by recognizing user emotions. The system includes a baggage inspection device, a generative AI, a server, a display device, and an emotion engine. This system automates a series of processes, from scanning baggage to analyzing the results, displaying the results, and responding to user emotions.

[0833] System Overview

[0834] 1. Baggage scanning process

[0835] The user places their baggage on the baggage inspection device and presses the scan button. The terminal acquires video data from the baggage inspection device, which then uses the onboard camera and X-ray device to capture image data of the baggage.

[0836] 2. Video data transfer and preprocessing

[0837] The device sends the captured video data to the server, which then performs preprocessing as needed before sending the received video data to the generation AI. Preprocessing is a process that adjusts the image resolution or trims unnecessary parts.

[0838] 3. Analysis by generative AI

[0839] The server sends the preprocessed video data to the generation AI for analysis. The generation AI analyzes the contents and shape of the baggage based on the received video data and evaluates its safety. As a result of the analysis, it determines whether the baggage's status is "passed" or "requires inspection."

[0840] 4. Receiving analysis results and determining status

[0841] The server receives the analysis results from the AI ​​generator and determines the baggage status based on the results. If the status is "passed," the baggage is processed without any problems. On the other hand, if it is determined to require further investigation, an alert is sent to staff.

[0842] 5. User Emotion Recognition by Emotion Engine

[0843] The device captures the user's face with a camera and sends it to the emotion engine. The emotion engine analyzes the user's facial expressions and recognizes the user's emotions (e.g., relief, surprise, anxiety, etc.). The recognized emotions are then sent to the server.

[0844] 6. Displaying results and addressing emotions

[0845] The terminal displays the results on a display device based on the baggage status received from the server and the user's emotional state. If the status is "passed," a message corresponding to the emotion, such as "relief," is displayed. If the status is "requires attention," an alert corresponding to the emotion is displayed, and staff will respond as necessary.

[0846] Specific examples

[0847] Example 1: If there are no problems with your luggage

[0848] User Action:

[0849] The user places their baggage on the baggage inspection device and presses the scan button.

[0850] The user's face is captured by a camera.

[0851] Terminal handling:

[0852] The terminal acquires the video data and transmits it to the server.

[0853] The terminal transmits the user's facial image to the emotion engine.

[0854] Server Action:

[0855] The server sends the video data to the generation AI and receives the analysis results.

[0856] The server receives the recognized emotion data from the emotion engine.

[0857] If the analysis result is determined to be "pass," a message based on the emotion is generated and sent to the device.

[0858] Terminal display:

[0859] The device will display "Passed" and a reassuring message.

[0860] The user can proceed.

[0861] Example 2: There is a problem with your package

[0862] User Action:

[0863] The user places their baggage on the baggage inspection device and presses the scan button.

[0864] The user's face is captured by a camera.

[0865] Terminal handling:

[0866] The terminal acquires the video data and transmits it to the server.

[0867] The terminal transmits the user's facial image to the emotion engine.

[0868] Server Action:

[0869] The server sends the video data to the generation AI and receives the analysis results.

[0870] The server receives the recognized emotion data from the emotion engine.

[0871] If the analysis results indicate that "attention is required," an emotion-based alert is generated and sent to the device.

[0872] Terminal display:

[0873] The device will display a message saying "Check required" and will try to ease any fears.

[0874] The user will ask airport staff for confirmation.

[0875] In this manner, the present invention can fully automate the baggage inspection process at airports and improve the quality of service by responding to user emotions.

[0876] The processing flow will be explained below.

[0877] Step 1: The user places their baggage on the baggage inspection device and presses the scan button, which starts scanning the baggage.

[0878] Step 2: The terminal acquires image data from the baggage inspection device. Image data of the baggage is captured by a camera or X-ray device and acquired as data.

[0879] Step 3: The device sends the acquired video data to the server. At this time, the device confirms that communication with the server has been established before transferring the data.

[0880] Step 4: The server preprocesses the video data received from the device. This involves adjusting the image resolution and trimming unnecessary parts, converting it into a format that is easy for the generating AI to analyze.

[0881] Step 5: The server sends the preprocessed video data to the generation AI for analysis. The generation AI analyzes the contents and shape of the baggage based on the received video data and determines its safety.

[0882] Step 6: The generating AI analyzes the data, determines the baggage status ('passed' or 'requires review'), and sends the result back to the server.

[0883] Step 7: The server receives the analysis results from the generation AI and makes a final decision on the baggage status based on the analysis results.

[0884] Step 8: The server receives the user's facial image from the device's camera and sends it to the emotion engine, which analyzes the facial expression and recognizes the user's emotional state (e.g., relief, anxiety, surprise, etc.).

[0885] Step 9: The emotion engine sends the recognized user emotion data back to the server, which uses this data along with the analysis results.

[0886] Step 10: The server generates an appropriate message or alert based on the baggage status and the user's emotions. If the baggage has passed, the server generates a message that provides reassurance, and if the baggage needs attention, the server generates a message that includes instructions to elicit reassurance.

[0887] Step 11: The server sends the generated message or alert to the terminal, which receives it and displays it on its display device.

[0888] Step 12: The terminal displays the baggage status and a message or alert depending on the user's emotion. If the baggage is "passed through," it displays "Proceed with confidence." If the baggage is "checked," it displays a message such as "Your baggage needs to be checked. Please contact an attendant."

[0889] Step 13: The user checks the displayed result. If it is "Pass", proceed as is. If it is "Confirmation Required", ask airport staff for confirmation.

[0890] Example 2

[0891] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0892] Conventional airport baggage inspection systems have the problem that they are unable to fully alleviate users' anxiety and dissatisfaction with inspection results because they are unable to accurately analyze the contents of baggage and respond to user emotions. To solve this problem, a system is needed that can accurately analyze baggage contents and display inspection results quickly and appropriately. Furthermore, it is necessary to improve the quality of service by recognizing users' emotions and responding accordingly.

[0893] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0894] In this invention, the server includes means for acquiring video data from a baggage inspection device, means for sending the video data to a generation AI and requesting analysis, means for receiving the analysis results from the generation AI and determining the status of the baggage, means for capturing the user's face and recognizing their emotions, and means for displaying the baggage status and the user's emotions on a display device, thereby enabling accurate analysis of the contents of the baggage, prompt display of inspection results, and response appropriate to the user's emotions.

[0895] A "baggage inspection device" is a device used at airports and other locations to scan baggage using X-rays or cameras to check its contents.

[0896] "Video data" refers to data that includes visual information about baggage, such as camera images or X-ray images acquired by baggage inspection equipment.

[0897] "Generative AI" is a technology that uses artificial intelligence to analyze video data and evaluate the contents, shape, and safety of baggage.

[0898] The "analysis results" are the information output by the generating AI after analyzing the video data, and indicate the status of the baggage.

[0899] "Baggage status" refers to whether the baggage can be passed through and whether any additional inspection is required, based on the analysis results of the generated AI.

[0900] "User" refers to a person who uses the baggage inspection system and performs the baggage scanning operation.

[0901] "Capturing a face" refers to obtaining an image of a user's face using a camera.

[0902] "Emotion recognition" means analyzing the user's facial expressions and identifying their emotional state, such as relief, surprise, or anxiety, from those expressions.

[0903] A "display device" is a device for visually displaying information such as baggage status and the user's emotional state.

[0904] This invention provides a system that fully automates airport baggage inspection and improves service quality by recognizing user emotions. The system includes a baggage inspection device, a generative AI, a server, a display device, and an emotion engine. This system automates a series of processes, from scanning baggage to analyzing the results, displaying the results, and responding to user emotions.

[0905] The operation of the system is as follows.

[0906] First, the user places their baggage on the baggage inspection device and presses the scan button. The terminal acquires video data from the baggage inspection device, which is then sent to the server. The server pre-processes the video data, which includes adjusting the image resolution and trimming unnecessary parts. Once pre-processing is complete, the server sends the video data to a generation AI, which analyzes the contents and shape of the baggage to assess its safety. As a result of the analysis, the baggage's status is determined as either "passed" or "requires inspection."

[0907] The results of this analysis are sent back to the server, which determines the baggage's status. If the status is "passed," the baggage is automatically processed and the user can pass through. On the other hand, if the status is "requires attention," an alert is generated and staff will respond.

[0908] At the same time, the device captures the user's face and sends the video data to the emotion engine. The emotion engine analyzes the user's facial expressions and recognizes the user's emotional state (e.g., relief, surprise, anxiety, etc.). The recognition results are sent to the server, which then integrates the baggage status and the user's emotional state and sends them to the device. The device then displays the integrated results on a display device, with a message appropriate for the user.

[0909] For example, when a user places his / her baggage on an inspection device and presses the scan button, the following process is carried out.

[0910] 1. The terminal acquires video data of the baggage and sends it to the server.

[0911] 2. The server preprocesses the video data and sends it to the generation AI.

[0912] 3. The generated AI analyzes the video data and may determine the baggage's status as "passed" or "requires inspection."

[0913] 4. At the same time, the device captures the user's face and sends it to the emotion engine, which then recognizes the user's emotions, such as "relief" or "anxiety."

[0914] 5. The server integrates the baggage status and emotion data and sends it to the terminal.

[0915] 6. The terminal displays the integrated results on the display device and displays an appropriate message to the user.

[0916] For example, consider the following prompt:

[0917] "Please begin baggage inspection scan."

[0918] "Package analysis results: Pass or check required?"

[0919] "Capture the user's face with a camera and recognize their emotions."

[0920] As described above, this system fully automates the baggage inspection process and makes it possible to provide services that adapt to the user's emotions.

[0921] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0922] Step 1:

[0923] The user places their baggage on the baggage inspection device and presses the scan button. This causes the baggage inspection device to acquire image data of the baggage (camera images and X-ray images). The input is the baggage, and the output is image data of the baggage. Specifically, the baggage inspection device starts scanning in response to the user's operation.

[0924] Step 2:

[0925] The terminal transmits the video data acquired from the baggage inspection device to the server. The input is the video data of the baggage, and the output is the data transmitted to the server. Specifically, the terminal transmits the video data to the server using network communication.

[0926] Step 3:

[0927] The server preprocesses the received video data. This preprocessing includes adjusting the image resolution and trimming unnecessary parts. The input is the video data sent to the server, and the output is the preprocessed video data. Specifically, the server processes the data using an image processing algorithm.

[0928] Step 4:

[0929] The server sends the pre-processed video data to the generation AI. The input is the pre-processed video data, and the output is the data sent to the generation AI. Specifically, the server communicates with the generation AI and sends the data.

[0930] Step 5:

[0931] The generating AI analyzes the contents and shape of the baggage based on the received video data and evaluates its safety. As a result of the analysis, the baggage's status is determined to be "passed" or "requires inspection." The input is the video data sent to the generating AI, and the output is the baggage's status. Specifically, the generating AI analyzes the data using a machine learning algorithm.

[0932] Step 6:

[0933] The server receives the analysis results from the generation AI and determines the baggage status based on the results. If the status is "passed," the baggage is processed without any problems. If the status is "requires attention," an alert is raised and staff are notified. The input is the analysis results from the generation AI, and the output is the baggage status and an alert if necessary. Specifically, the server processes the analysis results and determines the status.

[0934] Step 7:

[0935] The device captures the user's face with a camera and sends the video data to the emotion engine. The input is the user's face video data, and the output is the data sent to the emotion engine. In concrete terms, the device captures the face with a camera and sends the data.

[0936] Step 8:

[0937] The emotion engine analyzes the user's facial expressions and recognizes the user's emotions (relief, surprise, anxiety, etc.). The recognized emotional state is sent to the server. The input is the facial image data sent to the emotion engine, and the output is the analyzed emotional data. In concrete terms, the emotion engine uses a facial expression analysis algorithm.

[0938] Step 9:

[0939] The server integrates the baggage status and emotion data and sends it to the terminal. The input is the baggage status and emotion data, and the output is the integrated result data. Specifically, the server performs the integration calculation to generate the result.

[0940] Step 10:

[0941] The terminal displays the integrated result on the display device. For example, if the status is "passed" and the user's emotion is "relieved," the terminal will display that fact and the user can proceed. If the status is "confirmation required," the terminal will display an alert and the user will ask the staff for confirmation. The input is the integrated result data, and the output is the result displayed on the display device. As a specific operation, the terminal will display an appropriate message on the display.

[0942] The above is the processing flow of this system.

[0943] (Application example 2)

[0944] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0945] Conventional baggage inspection systems automate the display of inspection results and the issuance of alerts, but they have the problem of being unable to respond in a way that takes the user's emotions into consideration. This can cause users to feel anxious or uncomfortable, and there is a need to improve the quality of service. Meanwhile, smart home security systems are also required to accurately grasp the security situation and respond appropriately according to the user's emotions. To address these issues, systems that can reduce user anxiety and ensure safety are needed not only in baggage inspection systems but also in smart home security.

[0946] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0947] In this invention, the server includes means for acquiring video data from the baggage inspection device, means for sending the video data to the generation AI and requesting analysis, means for receiving the analysis results from the generation AI and determining the status of the baggage, means for capturing the user's face using a camera and sending the results to the emotion engine, means for analyzing the user's emotions and sending the results to the server, and means for displaying an appropriate message or alert on a display device based on the emotions. This makes it possible to respond according to the user's emotions and provide a more human-like, high-quality service.

[0948] "Baggage screening device" means a device used at airports and other security checkpoints to scan baggage and verify its contents.

[0949] "Video data" refers to digital image and video data acquired from baggage inspection equipment and cameras.

[0950] "Generative AI" is an artificial intelligence technology that analyzes acquired video data and independently determines its content and characteristics.

[0951] The "analysis results" are information about the contents and condition of baggage generated after the generating AI analyzes the video data.

[0952] "Status" is information indicating the safety of baggage and the status of inspection results, and includes indicators such as "passed" and "requires confirmation."

[0953] A "display device" is a device for visually displaying baggage inspection results and messages based on the user's emotions.

[0954] An "emotion engine" is a software system that analyzes a user's face and recognizes emotions (such as relief, surprise, or anxiety).

[0955] "Capture" is the act of acquiring image or video data using a device such as a camera.

[0956] A "server" is a computer system that processes and manages data for the entire system.

[0957] "Preprocessing" refers to processes such as adjusting resolution and trimming unnecessary parts, which are carried out to convert video data into a format that is easier to analyze.

[0958] A "message" refers to information or notifications provided to a user, the content of which is adjusted based on emotions.

[0959] An "alert" is a notification that warns users or staff when the baggage status is "requires attention" or when other abnormalities occur.

[0960] The smart home security system according to the present invention is implemented by the following procedure.

[0961] System Overview

[0962] The system consists of the following major hardware and software components:

[0963] Security camera (for acquiring video data)

[0964] Emotion engine (for recognizing user emotions)

[0965] Generative AI (for video data analysis)

[0966] Server (data processing and management)

[0967] Display device (for displaying results)

[0968] 1. Acquisition and transmission of video data

[0969] First, the device (smart home system) acquires video data in real time using a security camera. When a user enters the camera's range, their face is captured and sent to the emotion engine for emotion recognition. In parallel, the entire video data is sent to the server.

[0970] 2. Emotion recognition and sending results

[0971] The emotion engine analyzes the facial image and recognizes the user's emotions (relief, surprise, anxiety, etc.). The recognition results are sent to the server, which processes the data.

[0972] 3. Video data preprocessing and analysis

[0973] The server performs preprocessing such as adjusting the resolution and trimming unnecessary parts of the video data received from the security camera before sending it to the generation AI. The preprocessed data is sent to the generation AI for analysis. The generation AI evaluates security risks (suspicious movements, abnormal behavior, etc.) based on the video data and returns the analysis results to the server.

[0974] 4. Processing and displaying results

[0975] The server determines the security status based on the analysis results received from the generation AI. If the status is "safe," the result and a message based on the user's emotion are displayed on the display device. If the status is "requires attention," an alert is issued and a message based on the emotion is displayed, prompting the user to take appropriate action if necessary.

[0976] Specific examples

[0977] For example, the flow when a security camera detects movement late at night is as follows: The camera footage is sent to the server, and if the generation AI judges it to be "safe," the server displays the message "The environment is safe. Please be at ease," even if the emotion engine recognizes "anxiety." If the generation AI judges there to be a "potential threat," the server will display the message "A potential threat has been detected. Please respond with caution," even if the emotion engine recognizes "anxiety."

[0978] Prompt Sentence Examples

[0979] An example of a prompt sent to the generative AI model is as follows:

[0980] "Please analyze the baggage scan footage to determine if it contains any suspicious items. The video data is as follows."

[0981] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0982] Step 1:

[0983] The device captures video data in real time using a security camera. When a user enters the camera's range, their face is automatically captured. The captured video data and facial image are then stored in the device's memory.

[0984] Input: Real-time footage from security cameras

[0985] Output: Video data and facial images are saved on the device.

[0986] Specific operation: The security camera detects movement, captures video data, and stores it in the device's memory.

[0987] Step 2:

[0988] The device sends the stored facial image data to the emotion engine, which analyzes the user's face and recognizes emotions (relief, surprise, anxiety, etc.). The emotion recognition results are returned to the device, which receives and stores the results.

[0989] Input: Facial image data

[0990] Output: Emotion recognition result

[0991] Specific operation: Facial image data is sent to the emotion engine and the results are saved on the device.

[0992] Step 3:

[0993] The device sends the video data acquired from the security camera to the server, which receives the video data and performs preprocessing. Specifically, it adjusts the resolution, trims unnecessary parts, and converts the data into a format that is easy for the generating AI to analyze.

[0994] Input: Video data

[0995] Output: Pre-processed video data

[0996] Specific operation: Receives video data, adjusts resolution and trims unnecessary parts.

[0997] Step 4:

[0998] The server sends the pre-processed video data to the Generator AI for analysis. The Generator AI analyzes the data and evaluates the contents and condition of the baggage (or other object). The results of this analysis are returned to the server.

[0999] Input: Preprocessed video data

[1000] Output: Analysis results

[1001] Specific operation: The generative AI analyzes the video data and sends the results to the server.

[1002] Step 5:

[1003] The server receives the analysis results from the generative AI and determines the baggage status (safe, requires further inspection, etc.), as well as emotion recognition results from the emotion engine, and combines these data to generate appropriate messages and alerts.

[1004] Input: Analysis results and emotion recognition results

[1005] Output: Baggage status and message or alert to user

[1006] Specific operation: Determine the baggage status based on the analysis results and emotion recognition results, and generate a message or alert.

[1007] Step 6:

[1008] The terminal displays a message or alert on the display device based on the baggage status and emotion received from the server. For example, if the baggage is safe and the user is feeling anxious, the message "The environment is safe. Please rest assured" is displayed.

[1009] Input: Baggage status and messages or alerts

[1010] Output: A message or alert that appears on a display device.

[1011] Specific behavior: Display a message or alert on the display device based on the baggage status and emotion.

[1012] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1013] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1014] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1015] [Fourth embodiment]

[1016] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1017] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1019] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1020] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1021] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1023] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1024] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1025] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1026] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1027] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1028] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1029] This invention provides a system that includes a baggage inspection device, a generating AI, a server, and a display device to fully automate baggage inspection at airports. This system realizes a series of automated processes, from scanning baggage to analyzing and displaying the results.

[1030] System Overview

[1031] 1. Baggage scanning process

[1032] The terminal acquires video data from the baggage inspection device. This is done by taking images of the baggage using cameras and X-ray devices installed inside the device when the baggage is placed in the inspection device and scanning begins.

[1033] 2. Video data transfer and preprocessing

[1034] The device sends the captured video data to the server, which then performs preprocessing as needed before sending the received video data to the generation AI. Preprocessing is a process that adjusts the image resolution or trims unnecessary parts.

[1035] 3. Analysis by generative AI

[1036] The server sends the preprocessed video data to the generation AI for analysis. The generation AI analyzes the contents and shape of the baggage based on the received video data and evaluates its safety. As a result of the analysis, it determines whether the baggage's status is "passed" or "requires inspection."

[1037] 4. Receiving analysis results and determining status

[1038] The server receives the analysis results from the AI ​​generator and determines the baggage status based on the results. If the status is "passed," the baggage is processed without any problems. On the other hand, if it is determined to require further investigation, an alert is sent to staff.

[1039] 5. Displaying the results

[1040] The terminal displays the baggage status received from the server on the display device. If the baggage has been "passed through," the display will show "passed through" and the user can proceed. If the baggage has been "checked," the display will show "checked," and the user will have to ask airport staff for confirmation.

[1041] Specific examples

[1042] Example 1: If there are no problems with your luggage

[1043] User Action:

[1044] The user places their baggage on the baggage inspection device and presses the scan button.

[1045] Terminal handling:

[1046] The terminal acquires the video data and transmits it to the server.

[1047] Server Action:

[1048] The server sends the video data to the generation AI and receives the analysis results.

[1049] The analysis result is determined to be "pass" and the result is sent to the terminal.

[1050] Terminal display:

[1051] The terminal will display "Passed" and the user can proceed.

[1052] Example 2: There is a problem with your package

[1053] User Action:

[1054] The user places their baggage on the baggage inspection device and presses the scan button.

[1055] Terminal handling:

[1056] The terminal acquires the video data and transmits it to the server.

[1057] Server Action:

[1058] The server sends the video data to the generation AI and receives the analysis results.

[1059] The analysis result is determined to be "required for confirmation," and the result is sent to the terminal.

[1060] Terminal display:

[1061] The terminal will display "Confirmation Required" and the user will need to ask airport staff for confirmation.

[1062] As described above, the present invention can fully automate the baggage inspection process at airports, thereby reducing the need for human resources and improving user convenience.

[1063] The processing flow will be explained below.

[1064] Step 1: The user places their baggage on the baggage inspection device and presses the scan button, which starts scanning the baggage.

[1065] Step 2: The terminal acquires image data from the baggage inspection equipment. Specifically, the on-board camera and X-ray device capture image data of the baggage.

[1066] Step 3: The device sends the acquired video data to the server. At this time, the device confirms that communication with the server has been established and then starts data transfer.

[1067] Step 4: The server receives the video data from the device and preprocesses it into a format that is easy for the AI ​​to analyze. This preprocessing includes adjusting the image size and removing noise.

[1068] Step 5: The server sends the pre-processed video data to the generation AI for analysis. The generation AI receives the data and begins the process of analyzing the contents and shape of the baggage.

[1069] Step 6: The generated AI analyzes the video data and determines the baggage status ("passed" or "requires inspection"). The analysis results are sent back to the server.

[1070] Step 7: The server receives the analysis results from the generation AI and makes a final decision on the baggage status based on the received analysis results.

[1071] Step 8: The server sends the baggage status to the terminal, along with a detailed message about the status (e.g., the reason if it is "required").

[1072] Step 9: The terminal receives the baggage status from the server and displays "Passed" or "Confirmation Required" based on the received status.

[1073] Step 10: The user checks the results displayed on the terminal. If the status is "Passed," proceed as is. If it is "Confirmation Required," ask airport staff for confirmation.

[1074] Example 1

[1075] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1076] Conventional baggage inspection requires manual inspection, which is time-consuming and prone to human error. In addition, baggage analysis requires highly accurate judgment, making it difficult to quickly and accurately determine the safety of baggage.

[1077] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1078] In this invention, the server includes means for acquiring video data from the baggage inspection device, means for transmitting the video data to the server and performing preprocessing, means for transmitting the preprocessed video data to a generation AI and requesting analysis, means for receiving the analysis results from the generation AI and determining the status of the baggage, and means for displaying the status of the baggage on a display device. This automates the inspection process, prevents human errors, and enables fast and accurate baggage screening.

[1079] A "baggage inspection device" is a device that uses X-rays and cameras to obtain image data of the inside of baggage when it is placed there.

[1080] "Video data" is image information showing the internal structure of baggage acquired by a baggage inspection device.

[1081] The "server" is the central device of the system that receives video data, preprocesses it, requests the generation AI to analyze it, and receives the analysis results and determines the status.

[1082] "Preprocessing" refers to the process of adjusting the resolution and trimming the video data to convert it into a format that is easier to analyze.

[1083] "Generative AI" is an artificial intelligence model that analyzes received video data, evaluates the contents and shape of baggage, and determines its safety.

[1084] "Baggage status" refers to the status of the baggage, which is determined to be "passed" or "requires inspection" based on the safety of the baggage analyzed by the generating AI.

[1085] A "display device" is a device that displays the status of baggage to the user and airport staff.

[1086] An "alert" is a notification to alert airport staff when the baggage status is "requires attention."

[1087] This invention aims to improve the efficiency of baggage inspection at airports by proposing a system that includes a baggage inspection device, a generation AI, a server, and a display device. This system can realize a series of automated processes, from scanning baggage to analyzing and displaying the results.

[1088] Specifically, a user places their baggage on the baggage inspection device and presses the scan button. The baggage inspection device is equipped with a high-resolution camera and X-ray device, which capture image data of the baggage. The captured image data is then sent to a server via the terminal.

[1089] The server performs preprocessing on the received video data. This preprocessing includes adjusting the image resolution and trimming unnecessary parts. This preprocessing is performed using image processing libraries such as OpenCV and Pillow. The preprocessed video data is then sent to the generation AI.

[1090] The generating AI uses a model called "CargoScanAI" to analyze the contents and shape of baggage based on preprocessed video data and assess its safety. The analysis determines whether it contains explosives or illegal items. An example of a specific prompt is, "Analyze the video data below and check the contents of the baggage. Please return a decision of 'pass' or 'requires inspection' based on your safety standards."

[1091] The analysis results are sent back to the server, which determines the baggage's status. The baggage's status is evaluated as either "passed" or "requires review." If it is "passed," the baggage is processed without issue and the user can proceed. If it is "requires review," an alert is generated and airport staff are notified.

[1092] Finally, the terminal displays the baggage status received from the server on the display device. The display device displays information according to the status. For example, if the baggage has been "passed through," it will display "passed through" and the user can proceed. If the baggage has been "checked," it will display "checked," and the user will need to ask airport staff for confirmation.

[1093] In this way, the present invention automates the baggage inspection process, eliminating human error while also enabling fast and accurate baggage screening.

[1094] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1095] Step 1:

[1096] The terminal acquires image data from the baggage inspection device. Specifically, when a user places their baggage on the inspection device and presses the scan button, the high-resolution camera and X-ray device built into the device acquire image data of the baggage. The input data is an image of the baggage, and image data of this is generated as the output.

[1097] Step 2:

[1098] The device sends the captured video data to the server. Specifically, the video data is transferred to the server via Wi-Fi or a wired connection. The input is the video data of the baggage, and the output is the data transfer to the server.

[1099] Step 3:

[1100] The server preprocesses the received video data. Specifically, it adjusts the resolution and trims unnecessary parts. For this, it uses image processing libraries such as OpenCV and Pillow. The input is raw video data, and the output is preprocessed video data.

[1101] Step 4:

[1102] The server sends the preprocessed video data to the generation AI and requests it to analyze it. An example of a specific prompt is, "Analyze the following video data and check the contents of the baggage. Please return a decision of 'pass' or 'requires confirmation' based on safety standards." The input is the preprocessed video data, and the output is the generation of an analysis request.

[1103] Step 5:

[1104] The generating AI analyzes the received video data and evaluates the contents and shape of the baggage. As a result of the analysis, the baggage's status is determined to be "passed" or "requires inspection." Specifically, this analysis is performed by the AI ​​model "CargoScanAI." The input is pre-processed video data, and the output is the analysis result.

[1105] Step 6:

[1106] The server receives the analysis results from the generation AI and determines the baggage status based on the results. It sets the status as "passed" or "requires confirmation." The input is the analysis result, and the output is the determined status.

[1107] Step 7:

[1108] If the server determines that the baggage status is "required confirmation," it generates an alert and notifies airport staff. Specifically, the alert message is displayed on a dedicated monitor or terminal. The input is the "required confirmation" status, and the output is the generation of an alert.

[1109] Step 8:

[1110] The terminal displays the baggage status received from the server on a display device. A specific operation is to display a message such as "Passed" or "Confirmation Required" on the display device. The input is the baggage status, and the output is the status displayed on the display device.

[1111] This detailed processing step allows the baggage inspection process to be fast and efficient.

[1112] (Application example 1)

[1113] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1114] Advanced technology is required to quickly and accurately inspect baggage during security checks at airports and large event venues. However, current baggage inspection systems rely on human resources and have issues with both efficiency and accuracy. The present invention aims to solve these issues by fully automating the baggage inspection process.

[1115] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1116] In this invention, the server includes a means for acquiring video data from a baggage inspection device, a means for sending the video data to a generating AI and requesting analysis, and a means for receiving the analysis results from the generating AI and determining the status of the baggage. This enables the automation and efficiency of baggage inspection. The server further includes a means for scanning the video data of the baggage in real time using a smart device, a means for transferring the real-time scanned data to a cloud server, a means for analyzing the data using the generating AI in the cloud server, and a means for displaying the analysis results of the generating AI on a display device of the smart device. This enables fast and accurate advanced baggage inspection in a variety of security situations.

[1117] A "baggage inspection device" is a scanning device installed at airports, event venues, etc. to check the contents and security of baggage.

[1118] "Video data" refers to image information and video information of baggage acquired by the baggage inspection device.

[1119] "Generative AI" is an artificial intelligence technology that analyzes video data of baggage and evaluates its contents and safety.

[1120] A "smart device" is an electronic device that has a built-in camera and display and is capable of acquiring and displaying data, and includes, for example, smart glasses and smartphones.

[1121] A "cloud server" is a remote server that processes and analyzes data via the Internet.

[1122] "Preprocessing" refers to data adjustment operations to convert video data into a format that is easy to analyze.

[1123] "Status" refers to the result of the generating AI's assessment of the baggage's safety, and typically includes a classification such as "passed" or "requires review."

[1124] An "alert" is a warning signal or notification that is issued when baggage status is "needs attention."

[1125] "Real-time scanning" is the process of capturing video data of baggage in real time using a smart device.

[1126] A "display device" is a device that visualizes and displays baggage status and alerts.

[1127] As an embodiment of the present invention, a baggage inspection system using a smart device will be described.

[1128] Server Processing

[1129] The server receives video data acquired from the baggage inspection equipment. This video data is scanned in real time by a camera built into the smart device. Once the video data is acquired, the server performs preprocessing such as adjusting the image resolution and trimming unnecessary parts. This preprocessing converts the data into a format that is easy for the generative AI model to analyze.

[1130] Analysis by generative AI

[1131] The server sends the preprocessed video data to the generative AI and asks it to evaluate the contents of the baggage and its safety. The generative AI model analyzes this data and determines its status as safe or unsafe. Once the analysis results are available, the server receives them.

[1132] Anomaly detection and alerts

[1133] The analysis results determine whether the baggage should be allowed to pass or requires further investigation, and if it does, the server immediately sends an alert to the smart device. This alert indicates that there is a potential risk to the baggage, requiring a prompt response.

[1134] Displaying the results

[1135] The analysis results from the server are reflected on the smart device's display. When the user places their baggage on the baggage inspection device, the smart device's display will show either "Pass" or "Inspection Required."

[1136] Hardware and software used

[1137] Smart Device: Electronic devices with built-in cameras and displays, such as smart glasses and smartphones.

[1138] Cloud server: A high-performance server that processes data and analyzes generative AI models.

[1139] Generative AI model: Artificial intelligence for analyzing video data and assessing safety.

[1140] Prompt Sentence Examples

[1141] The following are examples of prompts that can be passed to the generation AI:

[1142] "Analyze the video data to determine whether dangerous items are present in the baggage and assess its safety."

[1143] Preprocessed video data is sent along with this prompt, and the generative AI model analyzes it.

[1144] Specific examples

[1145] For example, if this system is implemented at the entrance to a large event, the smart glasses will capture video data in real time as attendees scan their belongings and send it to a server. The server will preprocess the data and request analysis from a generative AI model. If the analysis result is "pass," entry is permitted; if it is "verification required," an alert will be issued immediately, requiring appropriate action. This will enable smooth and efficient security checks at large events.

[1146] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1147] Step 1:

[1148] The terminal captures real-time video data of baggage using the camera of a smart device (e.g., smart glasses). The input is the actual visual information of the baggage, which is captured as video data. The output is the captured video data of the baggage.

[1149] Step 2:

[1150] The device sends the captured video data to a cloud server via Wi-Fi. The input is the video data captured on the device, and the output is the video data sent to the cloud server. Data conversion and compression may occur during this process.

[1151] Step 3:

[1152] The server preprocesses the received video data. Specifically, it adjusts the image resolution and trims unnecessary parts. The input is the video data sent from the terminal, and the output is the preprocessed video data.

[1153] Step 4:

[1154] The server sends the preprocessed video data to the generation AI and requests it to analyze it. The input is the preprocessed video data, and the output is the data sent to the generation AI and the analysis results. At this time, the prompt statement "Analyze the video data to determine whether the baggage contains dangerous items and evaluate its safety" is used.

[1155] Step 5:

[1156] The generative AI analyzes the received video data and evaluates the safety of the baggage. The input is the preprocessed video data and a prompt, and the output is the baggage status ("passed" or "requires inspection"). The generative AI uses a deep learning model to analyze the images and determine safety.

[1157] Step 6:

[1158] The server processes the analysis results received from the generation AI and determines the baggage status. The input is the analysis results from the generation AI, and the output is the final baggage status. This status is either "passed" or "requires confirmation."

[1159] Step 7:

[1160] The server sends the confirmed baggage status to the terminal. The input is the final baggage status, and the output is a notification to the terminal. If the baggage status is "confirmation required," alert information is also included.

[1161] Step 8:

[1162] The terminal receives the baggage status from the server and displays it on the display device. The input is the baggage status sent from the server, and the output is the notification displayed on the display. The user checks the notification and takes action as necessary.

[1163] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1164] This invention provides a system that fully automates airport baggage inspection and improves service quality by recognizing user emotions. The system includes a baggage inspection device, a generative AI, a server, a display device, and an emotion engine. This system automates a series of processes, from scanning baggage to analyzing the results, displaying the results, and responding to user emotions.

[1165] System Overview

[1166] 1. Baggage scanning process

[1167] The user places their baggage on the baggage inspection device and presses the scan button. The terminal acquires video data from the baggage inspection device, which then uses the onboard camera and X-ray device to capture image data of the baggage.

[1168] 2. Video data transfer and preprocessing

[1169] The device sends the captured video data to the server, which then performs preprocessing as needed before sending the received video data to the generation AI. Preprocessing is a process that adjusts the image resolution or trims unnecessary parts.

[1170] 3. Analysis by generative AI

[1171] The server sends the preprocessed video data to the generation AI for analysis. The generation AI analyzes the contents and shape of the baggage based on the received video data and evaluates its safety. As a result of the analysis, it determines whether the baggage's status is "passed" or "requires inspection."

[1172] 4. Receiving analysis results and determining status

[1173] The server receives the analysis results from the AI ​​generator and determines the baggage status based on the results. If the status is "passed," the baggage is processed without any problems. On the other hand, if it is determined to require further investigation, an alert is sent to staff.

[1174] 5. User Emotion Recognition by Emotion Engine

[1175] The device captures the user's face with a camera and sends it to the emotion engine. The emotion engine analyzes the user's facial expressions and recognizes the user's emotions (e.g., relief, surprise, anxiety, etc.). The recognized emotions are then sent to the server.

[1176] 6. Displaying results and addressing emotions

[1177] The terminal displays the results on a display device based on the baggage status received from the server and the user's emotional state. If the status is "passed," a message corresponding to the emotion, such as "relief," is displayed. If the status is "requires attention," an alert corresponding to the emotion is displayed, and staff will respond as necessary.

[1178] Specific examples

[1179] Example 1: If there are no problems with your luggage

[1180] User Action:

[1181] The user places their baggage on the baggage inspection device and presses the scan button.

[1182] The user's face is captured by a camera.

[1183] Terminal handling:

[1184] The terminal acquires the video data and transmits it to the server.

[1185] The terminal transmits the user's facial image to the emotion engine.

[1186] Server Action:

[1187] The server sends the video data to the generation AI and receives the analysis results.

[1188] The server receives the recognized emotion data from the emotion engine.

[1189] If the analysis result is determined to be "pass," a message based on the emotion is generated and sent to the device.

[1190] Terminal display:

[1191] The device will display "Passed" and a reassuring message.

[1192] The user can proceed.

[1193] Example 2: There is a problem with your package

[1194] User Action:

[1195] The user places their baggage on the baggage inspection device and presses the scan button.

[1196] The user's face is captured by a camera.

[1197] Terminal handling:

[1198] The terminal acquires the video data and transmits it to the server.

[1199] The terminal transmits the user's facial image to the emotion engine.

[1200] Server Action:

[1201] The server sends the video data to the generation AI and receives the analysis results.

[1202] The server receives the recognized emotion data from the emotion engine.

[1203] If the analysis results indicate that "attention is required," an emotion-based alert is generated and sent to the device.

[1204] Terminal display:

[1205] The device will display a message saying "Check required" and will try to ease any fears.

[1206] The user will ask airport staff for confirmation.

[1207] In this manner, the present invention can fully automate the baggage inspection process at airports and improve the quality of service by responding to user emotions.

[1208] The processing flow will be explained below.

[1209] Step 1: The user places their baggage on the baggage inspection device and presses the scan button, which starts scanning the baggage.

[1210] Step 2: The terminal acquires image data from the baggage inspection device. Image data of the baggage is captured by a camera or X-ray device and acquired as data.

[1211] Step 3: The device sends the acquired video data to the server. At this time, the device confirms that communication with the server has been established before transferring the data.

[1212] Step 4: The server preprocesses the video data received from the device. This involves adjusting the image resolution and trimming unnecessary parts, converting it into a format that is easy for the generating AI to analyze.

[1213] Step 5: The server sends the preprocessed video data to the generation AI for analysis. The generation AI analyzes the contents and shape of the baggage based on the received video data and determines its safety.

[1214] Step 6: The generating AI analyzes the data, determines the baggage status ('passed' or 'requires review'), and sends the result back to the server.

[1215] Step 7: The server receives the analysis results from the generation AI and makes a final decision on the baggage status based on the analysis results.

[1216] Step 8: The server receives the user's facial image from the device's camera and sends it to the emotion engine, which analyzes the facial expression and recognizes the user's emotional state (e.g., relief, anxiety, surprise, etc.).

[1217] Step 9: The emotion engine sends the recognized user emotion data back to the server, which uses this data along with the analysis results.

[1218] Step 10: The server generates an appropriate message or alert based on the baggage status and the user's emotions. If the baggage has passed, the server generates a message that provides reassurance, and if the baggage needs attention, the server generates a message that includes instructions to elicit reassurance.

[1219] Step 11: The server sends the generated message or alert to the terminal, which receives it and displays it on its display device.

[1220] Step 12: The terminal displays the baggage status and a message or alert depending on the user's emotion. If the baggage is "passed through," it displays "Proceed with confidence." If the baggage is "checked," it displays a message such as "Your baggage needs to be checked. Please contact an attendant."

[1221] Step 13: The user checks the displayed result. If it is "Pass", proceed as is. If it is "Confirmation Required", ask airport staff for confirmation.

[1222] Example 2

[1223] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1224] Conventional airport baggage inspection systems have the problem that they are unable to fully alleviate users' anxiety and dissatisfaction with inspection results because they are unable to accurately analyze the contents of baggage and respond to user emotions. To solve this problem, a system is needed that can accurately analyze baggage contents and display inspection results quickly and appropriately. Furthermore, it is necessary to improve the quality of service by recognizing users' emotions and responding accordingly.

[1225] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1226] In this invention, the server includes means for acquiring video data from a baggage inspection device, means for sending the video data to a generation AI and requesting analysis, means for receiving the analysis results from the generation AI and determining the status of the baggage, means for capturing the user's face and recognizing their emotions, and means for displaying the baggage status and the user's emotions on a display device, thereby enabling accurate analysis of the contents of the baggage, prompt display of inspection results, and response appropriate to the user's emotions.

[1227] A "baggage inspection device" is a device used at airports and other locations to scan baggage using X-rays or cameras to check its contents.

[1228] "Video data" refers to data that includes visual information about baggage, such as camera images or X-ray images acquired by baggage inspection equipment.

[1229] "Generative AI" is a technology that uses artificial intelligence to analyze video data and evaluate the contents, shape, and safety of baggage.

[1230] The "analysis results" are the information output by the generating AI after analyzing the video data, and indicate the status of the baggage.

[1231] "Baggage status" refers to whether the baggage can be passed through and whether any additional inspection is required, based on the analysis results of the generated AI.

[1232] "User" refers to a person who uses the baggage inspection system and performs the baggage scanning operation.

[1233] "Capturing a face" refers to obtaining an image of a user's face using a camera.

[1234] "Emotion recognition" means analyzing the user's facial expressions and identifying their emotional state, such as relief, surprise, or anxiety, from those expressions.

[1235] A "display device" is a device for visually displaying information such as baggage status and the user's emotional state.

[1236] This invention provides a system that fully automates airport baggage inspection and improves service quality by recognizing user emotions. The system includes a baggage inspection device, a generative AI, a server, a display device, and an emotion engine. This system automates a series of processes, from scanning baggage to analyzing the results, displaying the results, and responding to user emotions.

[1237] The operation of the system is as follows.

[1238] First, the user places their baggage on the baggage inspection device and presses the scan button. The terminal acquires video data from the baggage inspection device, which is then sent to the server. The server pre-processes the video data, which includes adjusting the image resolution and trimming unnecessary parts. Once pre-processing is complete, the server sends the video data to a generation AI, which analyzes the contents and shape of the baggage to assess its safety. As a result of the analysis, the baggage's status is determined as either "passed" or "requires inspection."

[1239] The results of this analysis are sent back to the server, which determines the baggage's status. If the status is "passed," the baggage is automatically processed and the user can pass through. On the other hand, if the status is "requires attention," an alert is generated and staff will respond.

[1240] At the same time, the device captures the user's face and sends the video data to the emotion engine. The emotion engine analyzes the user's facial expressions and recognizes the user's emotional state (e.g., relief, surprise, anxiety, etc.). The recognition results are sent to the server, which then integrates the baggage status and the user's emotional state and sends them to the device. The device then displays the integrated results on a display device, with a message appropriate for the user.

[1241] For example, when a user places his / her baggage on an inspection device and presses the scan button, the following process is carried out.

[1242] 1. The terminal acquires video data of the baggage and sends it to the server.

[1243] 2. The server preprocesses the video data and sends it to the generation AI.

[1244] 3. The generated AI analyzes the video data and may determine the baggage's status as "passed" or "requires inspection."

[1245] 4. At the same time, the device captures the user's face and sends it to the emotion engine, which then recognizes the user's emotions, such as "relief" or "anxiety."

[1246] 5. The server integrates the baggage status and emotion data and sends it to the terminal.

[1247] 6. The terminal displays the integrated results on the display device and displays an appropriate message to the user.

[1248] For example, consider the following prompt:

[1249] "Please begin baggage inspection scan."

[1250] "Package analysis results: Pass or check required?"

[1251] "Capture the user's face with a camera and recognize their emotions."

[1252] As described above, this system fully automates the baggage inspection process and makes it possible to provide services that adapt to the user's emotions.

[1253] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1254] Step 1:

[1255] The user places their baggage on the baggage inspection device and presses the scan button. This causes the baggage inspection device to acquire image data of the baggage (camera images and X-ray images). The input is the baggage, and the output is image data of the baggage. Specifically, the baggage inspection device starts scanning in response to the user's operation.

[1256] Step 2:

[1257] The terminal transmits the video data acquired from the baggage inspection device to the server. The input is the video data of the baggage, and the output is the data transmitted to the server. Specifically, the terminal transmits the video data to the server using network communication.

[1258] Step 3:

[1259] The server preprocesses the received video data. This preprocessing includes adjusting the image resolution and trimming unnecessary parts. The input is the video data sent to the server, and the output is the preprocessed video data. Specifically, the server processes the data using an image processing algorithm.

[1260] Step 4:

[1261] The server sends the pre-processed video data to the generation AI. The input is the pre-processed video data, and the output is the data sent to the generation AI. Specifically, the server communicates with the generation AI and sends the data.

[1262] Step 5:

[1263] The generating AI analyzes the contents and shape of the baggage based on the received video data and evaluates its safety. As a result of the analysis, the baggage's status is determined to be "passed" or "requires inspection." The input is the video data sent to the generating AI, and the output is the baggage's status. Specifically, the generating AI analyzes the data using a machine learning algorithm.

[1264] Step 6:

[1265] The server receives the analysis results from the generation AI and determines the baggage status based on the results. If the status is "passed," the baggage is processed without any problems. If the status is "requires attention," an alert is raised and staff are notified. The input is the analysis results from the generation AI, and the output is the baggage status and an alert if necessary. Specifically, the server processes the analysis results and determines the status.

[1266] Step 7:

[1267] The device captures the user's face with a camera and sends the video data to the emotion engine. The input is the user's face video data, and the output is the data sent to the emotion engine. In concrete terms, the device captures the face with a camera and sends the data.

[1268] Step 8:

[1269] The emotion engine analyzes the user's facial expressions and recognizes the user's emotions (relief, surprise, anxiety, etc.). The recognized emotional state is sent to the server. The input is the facial image data sent to the emotion engine, and the output is the analyzed emotional data. In concrete terms, the emotion engine uses a facial expression analysis algorithm.

[1270] Step 9:

[1271] The server integrates the baggage status and emotion data and sends it to the terminal. The input is the baggage status and emotion data, and the output is the integrated result data. Specifically, the server performs the integration calculation to generate the result.

[1272] Step 10:

[1273] The terminal displays the integrated result on the display device. For example, if the status is "passed" and the user's emotion is "relieved," the terminal will display that fact and the user can proceed. If the status is "confirmation required," the terminal will display an alert and the user will ask the staff for confirmation. The input is the integrated result data, and the output is the result displayed on the display device. As a specific operation, the terminal will display an appropriate message on the display.

[1274] The above is the processing flow of this system.

[1275] (Application example 2)

[1276] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1277] Conventional baggage inspection systems automate the display of inspection results and the issuance of alerts, but they have the problem of being unable to respond in a way that takes the user's emotions into consideration. This can cause users to feel anxious or uncomfortable, and there is a need to improve the quality of service. Meanwhile, smart home security systems are also required to accurately grasp the security situation and respond appropriately according to the user's emotions. To address these issues, systems that can reduce user anxiety and ensure safety are needed not only in baggage inspection systems but also in smart home security.

[1278] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1279] In this invention, the server includes means for acquiring video data from the baggage inspection device, means for sending the video data to the generation AI and requesting analysis, means for receiving the analysis results from the generation AI and determining the status of the baggage, means for capturing the user's face using a camera and sending the results to the emotion engine, means for analyzing the user's emotions and sending the results to the server, and means for displaying an appropriate message or alert on a display device based on the emotions. This makes it possible to respond according to the user's emotions and provide a more human-like, high-quality service.

[1280] "Baggage screening device" means a device used at airports and other security checkpoints to scan baggage and verify its contents.

[1281] "Video data" refers to digital image and video data acquired from baggage inspection equipment and cameras.

[1282] "Generative AI" is an artificial intelligence technology that analyzes acquired video data and independently determines its content and characteristics.

[1283] The "analysis results" are information about the contents and condition of baggage generated after the generating AI analyzes the video data.

[1284] "Status" is information indicating the safety of baggage and the status of inspection results, and includes indicators such as "passed" and "requires confirmation."

[1285] A "display device" is a device for visually displaying baggage inspection results and messages based on the user's emotions.

[1286] An "emotion engine" is a software system that analyzes a user's face and recognizes emotions (such as relief, surprise, or anxiety).

[1287] "Capture" is the act of acquiring image or video data using a device such as a camera.

[1288] A "server" is a computer system that processes and manages data for the entire system.

[1289] "Preprocessing" refers to processes such as adjusting resolution and trimming unnecessary parts, which are carried out to convert video data into a format that is easier to analyze.

[1290] A "message" refers to information or notifications provided to a user, the content of which is adjusted based on emotions.

[1291] An "alert" is a notification that warns users or staff when the baggage status is "requires attention" or when other abnormalities occur.

[1292] The smart home security system according to the present invention is implemented by the following procedure.

[1293] System Overview

[1294] The system consists of the following major hardware and software components:

[1295] Security camera (for acquiring video data)

[1296] Emotion engine (for recognizing user emotions)

[1297] Generative AI (for video data analysis)

[1298] Server (data processing and management)

[1299] Display device (for displaying results)

[1300] 1. Acquisition and transmission of video data

[1301] First, the device (smart home system) acquires video data in real time using a security camera. When a user enters the camera's range, their face is captured and sent to the emotion engine for emotion recognition. In parallel, the entire video data is sent to the server.

[1302] 2. Emotion recognition and sending results

[1303] The emotion engine analyzes the facial image and recognizes the user's emotions (relief, surprise, anxiety, etc.). The recognition results are sent to the server, which processes the data.

[1304] 3. Video data preprocessing and analysis

[1305] The server performs preprocessing such as adjusting the resolution and trimming unnecessary parts of the video data received from the security camera before sending it to the generation AI. The preprocessed data is sent to the generation AI for analysis. The generation AI evaluates security risks (suspicious movements, abnormal behavior, etc.) based on the video data and returns the analysis results to the server.

[1306] 4. Processing and displaying results

[1307] The server determines the security status based on the analysis results received from the generation AI. If the status is "safe," the result and a message based on the user's emotion are displayed on the display device. If the status is "requires attention," an alert is issued and a message based on the emotion is displayed, prompting the user to take appropriate action if necessary.

[1308] Specific examples

[1309] For example, the flow when a security camera detects movement late at night is as follows: The camera footage is sent to the server, and if the generation AI judges it to be "safe," the server displays the message "The environment is safe. Please be at ease," even if the emotion engine recognizes "anxiety." If the generation AI judges there to be a "potential threat," the server will display the message "A potential threat has been detected. Please respond with caution," even if the emotion engine recognizes "anxiety."

[1310] Prompt Sentence Examples

[1311] An example of a prompt sent to the generative AI model is as follows:

[1312] "Please analyze the baggage scan footage to determine if it contains any suspicious items. The video data is as follows."

[1313] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1314] Step 1:

[1315] The device captures video data in real time using a security camera. When a user enters the camera's range, their face is automatically captured. The captured video data and facial image are then stored in the device's memory.

[1316] Input: Real-time footage from security cameras

[1317] Output: Video data and facial images are saved on the device.

[1318] Specific operation: The security camera detects movement, captures video data, and stores it in the device's memory.

[1319] Step 2:

[1320] The device sends the stored facial image data to the emotion engine, which analyzes the user's face and recognizes emotions (relief, surprise, anxiety, etc.). The emotion recognition results are returned to the device, which receives and stores the results.

[1321] Input: Facial image data

[1322] Output: Emotion recognition result

[1323] Specific operation: Facial image data is sent to the emotion engine and the results are saved on the device.

[1324] Step 3:

[1325] The device sends the video data acquired from the security camera to the server, which receives the video data and performs preprocessing. Specifically, it adjusts the resolution, trims unnecessary parts, and converts the data into a format that is easy for the generating AI to analyze.

[1326] Input: Video data

[1327] Output: Pre-processed video data

[1328] Specific operation: Receives video data, adjusts resolution and trims unnecessary parts.

[1329] Step 4:

[1330] The server sends the pre-processed video data to the Generator AI for analysis. The Generator AI analyzes the data and evaluates the contents and condition of the baggage (or other object). The results of this analysis are returned to the server.

[1331] Input: Preprocessed video data

[1332] Output: Analysis results

[1333] Specific operation: The generative AI analyzes the video data and sends the results to the server.

[1334] Step 5:

[1335] The server receives the analysis results from the generative AI and determines the baggage status (safe, requires further inspection, etc.), as well as emotion recognition results from the emotion engine, and combines these data to generate appropriate messages and alerts.

[1336] Input: Analysis results and emotion recognition results

[1337] Output: Baggage status and message or alert to user

[1338] Specific operation: Determine the baggage status based on the analysis results and emotion recognition results, and generate a message or alert.

[1339] Step 6:

[1340] The terminal displays a message or alert on the display device based on the baggage status and emotion received from the server. For example, if the baggage is safe and the user is feeling anxious, the message "The environment is safe. Please rest assured" is displayed.

[1341] Input: Baggage status and messages or alerts

[1342] Output: A message or alert that appears on a display device.

[1343] Specific behavior: Display a message or alert on the display device based on the baggage status and emotion.

[1344] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1345] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1346] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1347] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1348] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1349] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1350] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1351] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1352] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1353] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1354] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1355] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1356] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1357] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1358] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1359] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1360] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1361] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1362] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1363] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1364] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1365] The following is further disclosed regarding the above embodiment.

[1366] (Claim 1)

[1367] means for acquiring video data from a baggage inspection device;

[1368] A means for transmitting the video data to a generation AI and requesting analysis;

[1369] A means for receiving an analysis result from the generating AI and determining the status of baggage;

[1370] The system includes means for displaying the status of said baggage on a display device.

[1371] (Claim 2)

[1372] 10. The system of claim 1, further comprising means for pre-processing the video data into a format that is easy to analyze.

[1373] (Claim 3)

[1374] 10. The system of claim 1, further comprising means for issuing an alert if the baggage status is "needs attention."

[1375] "Example 1"

[1376] (Claim 1)

[1377] means for acquiring video data from a baggage inspection device;

[1378] means for transmitting the video data to a server and performing preprocessing;

[1379] A means for transmitting the preprocessed video data to a generation AI and requesting analysis;

[1380] A means for receiving an analysis result from the generating AI and determining the status of baggage;

[1381] The system includes means for displaying the status of said baggage on a display device.

[1382] (Claim 2)

[1383] 10. The system of claim 1, further comprising means for pre-processing the video data into a format that is easy to analyze.

[1384] (Claim 3)

[1385] 10. The system of claim 1, further comprising means for issuing an alert if the baggage status is "needs attention."

[1386] "Application Example 1"

[1387] (Claim 1)

[1388] means for acquiring video data from a baggage inspection device;

[1389] A means for transmitting the video data to a generation AI and requesting analysis;

[1390] A means for receiving an analysis result from the generating AI and determining the status of baggage;

[1391] means for displaying the status of said baggage on a display device;

[1392] A means for scanning video data of baggage in real time using a smart device;

[1393] means for transferring the real-time scan data to a cloud server;

[1394] A means for performing analysis using a generating AI on a cloud server;

[1395] The system includes a means for displaying the analysis results on a display device of a smart device.

[1396] (Claim 2)

[1397] 10. The system of claim 1, further comprising means for pre-processing the video data into a format that is easy to analyze.

[1398] (Claim 3)

[1399] 10. The system of claim 1, further comprising means for issuing an alert if the baggage status is "needs attention."

[1400] "Example 2: Combining Emotion Engines"

[1401] (Claim 1)

[1402] means for acquiring video data from a baggage inspection device;

[1403] A means for transmitting the video data to a generation AI and requesting analysis;

[1404] A means for receiving an analysis result from the generating AI and determining the status of baggage;

[1405] means for capturing a user's face and recognizing emotions;

[1406] The system includes means for displaying the status of the baggage and the user's emotions on a display device.

[1407] (Claim 2)

[1408] 10. The system of claim 1, further comprising means for pre-processing the video data into a format that is easy to analyze.

[1409] (Claim 3)

[1410] 10. The system of claim 1, further comprising means for issuing an alert if the baggage status is "needs attention."

[1411] "Application example 2 when combining emotion engines"

[1412] (Claim 1)

[1413] means for acquiring video data from a baggage inspection device;

[1414] A means for transmitting the video data to a generation AI and requesting analysis;

[1415] A means for receiving an analysis result from the generating AI and determining the status of baggage;

[1416] means for displaying the status of said baggage on a display device;

[1417] means for capturing a user's face using a camera and transmitting the captured image to an emotion engine;

[1418] means for analyzing the user's emotions and transmitting the results to a server;

[1419] Means for displaying appropriate messages or alerts on a display device based on emotions

[1420] A system including:

[1421] (Claim 2)

[1422] 10. The system of claim 1, further comprising means for pre-processing the video data into a format that is easy to analyze.

[1423] (Claim 3)

[1424] The system according to claim 1, further comprising means for issuing an alert when the baggage status is "check required" and instructing a response taking into account the user's feelings. [Explanation of symbols]

[1425] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for acquiring video data from a baggage inspection device; A means for transmitting the video data to a generation AI and requesting analysis; A means for receiving an analysis result from the generating AI and determining the status of baggage; The system includes means for displaying the status of said baggage on a display device.

2. The system of claim 1 further comprising means for pre-processing the video data into a format that is easy to analyze.

3. The system of claim 1 further comprising means for issuing an alert if the baggage status is "needs attention."

Images