System

Abstract

A system is provided.SOLUTION: A system includes a user terminal that uploads a photograph of a lost article, a server that receives the uploaded photograph and analyzes a feature of an object using an image recognition means, a means that stores the analyzed feature in a database, a server that receives feature information input by a person who has lost an article and analyzes the feature using a natural language processing means and collates the feature with object information in the database, a means that notifies a user terminal of a matching result based on the collated information, and a means that records the matching information in a blockchain and ensures transparency and reliability.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, tens of millions of lost items occur annually on public transportation, in commercial facilities, and elsewhere, and dealing with these cases requires a tremendous amount of time and effort. This problem is extremely stressful for people who have lost their items, and there is a need for a method to quickly and accurately connect the finder with the appropriate owner. Furthermore, traditional methods lack trust between the finder and the owner, exposing them to the risk of fraud and information tampering. [Means for solving the problem]

[0005] The present invention provides a user terminal that uploads photos of lost items, a server that receives the uploaded photos and analyzes the features of the object using image recognition means, a means for saving the analyzed features in a database, a server that receives feature information entered by the person who lost the item, analyzes it using natural language processing means, and compares it with object information in the database, a means for notifying the user terminal of a matching result based on the compared information, and a means for recording the matching information in a blockchain to ensure transparency and reliability.Furthermore, by providing a chat function and including a means for users to communicate with each other after matching and a means for recording information after the delivery is completed again in the blockchain, the overall reliability and efficiency are improved.

[0006] A "user terminal" is a computer device through which a user inputs information and receives notifications from a server.

[0007] A "server" is a central management system for processing information received from user terminals and managing a database.

[0008] "Image recognition means" is a technology that analyzes and extracts the characteristics of objects from uploaded photos.

[0009] A "database" is a system for storing analyzed feature information and information entered by the user.

[0010] "Natural language processing means" is a technology for analyzing information input as text and understanding human language.

[0011] "Matching result" is information generated when a dropped item matches a picked-up item.

[0012] The "chat function" is a communication function that allows users to exchange messages with each other.

[0013] "Blockchain" is a technology that records data on a distributed ledger to prevent information tampering and ensure reliability. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] A specific embodiment of the present invention is described below. The present invention is a system that quickly and accurately connects the person who found a lost item with the person who lost it, utilizing the image recognition and natural language processing capabilities of AI, and further using blockchain technology to ensure the transparency and reliability of information.

[0036] User Registration

[0037] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. This information is received by the device and sent to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete.

[0038] Registering lost items

[0039] The user takes a photo of the lost item they found, enters the necessary text information (e.g., location, date, time, characteristics of the item, etc.), and sends it from the device to the server. The server analyzes the photo data using image recognition to extract detailed information about the item (color, shape, size, brand name, etc.). This extracted information is stored in a database along with the text information. The device is then notified that registration is complete.

[0040] Search for lost items

[0041] To search for a lost item, the user inputs characteristic information as text, which the device then sends to the server. The server then uses natural language processing to analyze the input text and compare it with object information in the database. If a matching item is found, the server notifies the user of the matching result; if not, it sends a "not found" notification.

[0042] Notification of matching results

[0043] The server then sends notifications to both the person who lost the item and the person who found it based on the information matched in the database. The server also records this matching information on the blockchain, ensuring transparency and reliability of the information.

[0044] Communication and delivery

[0045] The user receives a notification and communicates with the finder using the chat function within the app. After communicating with the finder, the details of the delivery are decided. After the delivery is completed, it is confirmed within the app, and the server records the delivery completion information again on the blockchain.

[0046] Specific examples

[0047] As a concrete example, let's say someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and enters the text information, "I found it on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database.

[0048] Meanwhile, if the person who lost their wallet uses the app to search for "I lost a black leather wallet. It's about 15cm x 10cm," the server will use this information to search the database and match it with the registered information of the person who found it. The matching results are notified to both parties, who then communicate via chat and hand over the wallet at a designated location. After the handover is complete, the server records this information on the blockchain to ensure overall transparency and reliability.

[0049] In this way, the present invention is a system that allows users to easily register and search for lost items, performs quick and accurate matching, and utilizes blockchain technology to ensure overall transparency and reliability.

[0050] The processing flow will be explained below.

[0051] Handling user registration

[0052] Step 1:

[0053] User: Launches the app and accesses the user registration screen.

[0054] Step 2:

[0055] User: Enter the required information such as name, contact details, and email address, and tap the "Register" button.

[0056] Step 3:

[0057] Terminal: The entered user information is sent to the server in JSON format.

[0058] Step 4:

[0059] Server: Analyzes the received user information and stores it in a database. Generates a response indicating registration completion and sends it to the terminal.

[0060] Step 5:

[0061] Terminal: Receives notification of registration completion and displays it to the user.

[0062] Processing lost and found entries

[0063] Step 1:

[0064] User: Access the app's "Lost Item Registration" screen to register the item they found.

[0065] Step 2:

[0066] User: Take a photo of the lost item, enter the necessary text information (e.g., location, date, time, item characteristics, etc.), and tap the "Register" button.

[0067] Step 3:

[0068] Terminal: Sends photo data and text information to the server.

[0069] Step 4:

[0070] Server: The received photo data is passed to the image recognition module, which analyzes the object's detailed information (color, shape, size, brand name, etc.).

[0071] Step 5:

[0072] Server: Combines the analysis results with the text information and stores them in a database. Sends a notification of registration completion to the device.

[0073] Step 6:

[0074] Terminal: Receives notification of registration completion and displays it to the user.

[0075] Processing lost and found searches

[0076] Step 1:

[0077] User: Access the app's "Lost Item Search" screen and enter the characteristics of the lost item.

[0078] Step 2:

[0079] User: Check the information entered and tap the "Search" button.

[0080] Step 3:

[0081] Terminal: Sends the input characteristic information to the server.

[0082] Step 4:

[0083] Server: Passes the received feature information to the natural language processing module for analysis.

[0084] Step 5:

[0085] Server: Compares the analyzed information with the lost items information in the database.

[0086] Step 6:

[0087] Server: If a matching item is found, it generates a matching result and notifies the user's device. If not found, it notifies the user.

[0088] Step 7:

[0089] Terminal: Receives the matching result notification and displays it to the user.

[0090] Processing of match result notifications

[0091] Step 1:

[0092] Server: Checks the database for a match between the user's dropped item and the item they found.

[0093] Step 2:

[0094] Server: Notifies the users (those who dropped the item and those who picked it up) of the matching results.

[0095] Step 3:

[0096] Server: Records the matching information immutably on the blockchain.

[0097] Contact and delivery processing

[0098] Step 1:

[0099] Users: Receive in-app notifications, check match results, and use the chat feature to contact the finder.

[0100] Step 2:

[0101] Users (those who lost the item and those who found it): Use the chat function to communicate and decide the details of the delivery.

[0102] Step 3:

[0103] User (the person who lost the item): Receives the lost item from the finder at the designated location. The completion of the delivery is confirmed within the app.

[0104] Step 4:

[0105] User (the person who found the item): Confirm in the app that the item was returned safely.

[0106] Step 5:

[0107] Server: Receives information about the completion of the transfer and records it on the blockchain.

[0108] The above is the specific operation of each processing step. The user, device, and server work together to efficiently complete the series of processes of registering, searching, matching, and handing over lost items.

[0109] Example 1

[0110] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0111] In conventional lost property management systems, it was difficult to quickly and accurately connect the person who found the lost item with the person who dropped it, and the transparency and reliability of information were not sufficiently ensured.In addition, the means of communication between users and information management after the handover were insufficient, which reduced the reliability of the entire system.

[0112] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0113] In this invention, the server includes a terminal for users to upload photos of lost items, an information processing device that receives the uploaded photos and analyzes the features of the object using image recognition means, means for saving the analyzed features in a storage device, an information processing device that receives feature information entered by the user who lost the item, analyzes it using natural language processing means, and compares it with the object information in the storage device, means for notifying the user of a matching result based on the generated comparison information, means for recording the matching information in a fixed ledger technology to ensure information transparency and reliability, means for users to receive the notification and communicate using a chat function, and means for recording information after the transfer is completed in the fixed ledger technology again. This allows the person who found the lost item to quickly and accurately connect with the person who dropped it, ensures information transparency and reliability, allows users to efficiently communicate with each other, and enables information management after the transfer is completed.

[0114] A "user terminal" is a device for uploading photos of lost items, and has the function of transmitting the input information to a server.

[0115] An "information processing device" is a device that has the function of analyzing input information using image recognition means and natural language processing means and storing the information in a storage device called a database.

[0116] "Image recognition means" refers to technology or software that analyzes uploaded photographs and identifies the characteristics of objects contained within them.

[0117] "Natural language processing means" refers to technology or software that analyzes text information entered by a user and understands its meaning and intent.

[0118] The "chat function" is a communication function that allows users to exchange messages and information in real time.

[0119] "Fixed ledger technology" is a technology similar to blockchain technology that preserves information once recorded with transparency and reliability, without being tampered with.

[0120] A "memory device" is a database or other storage system for storing analyzed features and other data.

[0121] "Matching results" refer to the server analyzing the characteristic information provided by the user and comparing it with information in the database to identify matching lost item information.

[0122] "Information on completion of delivery" is data indicating that the delivery of the lost item, which was determined through communication between users, has been completed.

[0123] This invention is a system that quickly and accurately connects the person who found a lost item with the person who lost it, utilizing image recognition and natural language processing capabilities, as well as blockchain technology to ensure the transparency and reliability of information.

[0124] First, the user starts the app and enters the necessary information, such as their name, contact details, and email address. The device receives this information and sends it to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete. At this time, the server uses an SQL database such as MySQL (registered trademark) or PostgreSQL.

[0125] Next, the user takes a photo of the lost item and sends the necessary text information (e.g., location, date, time, and characteristics of the item) from the device to the server. The server analyzes the received photo using TENSORFLOW (registered trademark) or OpenCV to extract detailed information about the item (color, shape, size, brand name, etc.). These details are stored in a database along with the text information, and the device is notified that registration is complete.

[0126] To search for a lost item, the user inputs feature information as text, which the device then sends to the server. The server then analyzes the input text using a natural language processing model such as BERT and compares it with object information in a database. If a matching item is found, the server notifies the user's device of the matching result. If the item is not found, the server sends a "not found" notification.

[0127] The server then sends notifications to both the person who lost the item and the person who found it based on the information collated in the database.The server also records this matching information on a blockchain such as Ethereum, ensuring the transparency and reliability of the information.

[0128] The user receives a notification and uses the chat function within the app (for example, Firebase chat service) to communicate with the finder. After communicating, they decide the details of the delivery. After the delivery is completed, this is confirmed within the app, and the server records the delivery completion information again on the blockchain.

[0129] As a concrete example, let's say someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and enters the text information, "I found it on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database.

[0130] Meanwhile, if the person who lost their wallet uses the app to search for "I lost a black leather wallet. It's about 15cm x 10cm," the server will use this information to search the database and match it with the registered information of the person who found it. The matching results are notified to both parties, who then communicate via chat and hand over the wallet at a designated location. After the handover is complete, the server records this information on the blockchain, ensuring overall transparency and reliability.

[0131] In this way, users can easily register and search for lost items, and matching can be done quickly and accurately, while blockchain technology is used to ensure overall transparency and reliability.

[0132] Example prompt sentence:

[0133] "I found a black leather wallet on a bench at the station. It measures 15cm x 10cm. How can I find the owner of this wallet?"

[0134] The above is an embodiment of the present invention.

[0135] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0136] Step 1: User Registration

[0137] The user uses the device and launches the app. The user enters the required information, such as their name, contact details, and email address. The device receives this information and sends it to the server. The server analyzes the received information and stores it in a database. It then generates a response indicating registration is complete and sends it to the device.

[0138] Input: Name, contact details, and email address entered by the user.

[0139] Data processing: Sending data from the device to the server.

[0140] Data calculation: Data analysis and database storage on the server.

[0141] Output: Registration complete response.

[0142] What happens: When a user clicks the "Register" button, the information is sent to the server and confirmed to be saved in the database. A notification of successful registration is displayed on the device.

[0143] Step 2: Register your lost property

[0144] The user uses their device to take a photo of the lost item and uploads it to the app. The user then enters text information such as the location, date, and item characteristics, and sends it from the device to the server. The server then analyzes the received photo data using image recognition techniques (e.g., TensorFlow or OpenCV) to extract detailed information about the item (color, shape, size, brand name, etc.). The extracted information and text information are then stored in a database, and the user is notified that registration is complete.

[0145] Input: A photo taken by the user and text information.

[0146] Data processing: Sending photos and text information from the device to the server.

[0147] Data calculation: Analyzing photo data and extracting detailed information using image recognition methods.

[0148] Output: Notification that registration is complete.

[0149] Specific operation: When a user presses the "Register" button, the photo and text information are sent to the server, where image analysis is performed. The results are stored in a database, and a notification of registration completion is displayed on the device.

[0150] Step 3: Search for lost property

[0151] The user uses their device to input textual information about the features of the dropped item, which is then sent from the device to the server. The server then uses natural language processing (e.g., the BERT model) to analyze the input feature information and compare it with object information in the database. If a matching item is found, a matching result is generated and notified to the user's device.

[0152] Input: Feature information entered by the user.

[0153] Data processing: Sending characteristic information from the device to the server.

[0154] Data Computing: Analysis of feature information and database search using natural language processing tools.

[0155] Output: Matching result or "not found" notification.

[0156] Specific operation: When the user presses the "Search" button, the characteristic information is analyzed and compared with the information in the database. The results are displayed on the device.

[0157] Step 4: Notification of match results

[0158] The server then sends notifications to both the person who lost the item and the person who found it based on the matched information in the database, and records this matching information on the blockchain to ensure the transparency and reliability of the information.

[0159] Input: The matched information.

[0160] Data processing: Creating notifications of matching results.

[0161] Data calculation: Blockchain recording of matching information.

[0162] Output: Notification of matching results and recording to the blockchain.

[0163] Specific operation: The server checks the matching information, generates a notification and sends it to the user. The result is displayed on the device and simultaneously recorded on the blockchain.

[0164] Step 5: Communication and delivery

[0165] The user receives a notification and uses the chat function within the app (for example, Firebase chat service) to communicate with the finder. After communicating, they decide the details of the delivery. After the delivery is completed, this is confirmed within the app, and the server records the delivery completion information again on the blockchain.

[0166] Input: Completed handover information.

[0167] Data processing: Communication via chat and confirmation of completion of delivery.

[0168] Data calculation: Blockchain recording of delivery completion information.

[0169] Output: A confirmation that the transaction has been completed and recorded on the blockchain.

[0170] How it works: Users communicate and decide on the details of the transaction. Once the transaction is complete, the information is recorded on the blockchain.

[0171] The above is a detailed description of each processing step of the system.

[0172] (Application example 1)

[0173] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0174] In modern factory environments, lost parts and tools have a significant impact on production efficiency. Work is often delayed when workers are unable to find the parts they need. Managing and locating lost items requires a great deal of time and effort, creating a need for a system that can quickly and efficiently resolve this issue. Furthermore, measures to ensure transparency and reliability are insufficient, and this also requires improvement.

[0175] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0176] In this invention, the server includes a means for preventing and quickly locating parts and tools lost in a factory environment, a means for searching for parts information by voice input or text input and analyzing it with natural language processing means, and a means for notifying the owner of the lost item with an internal notification function. This makes it possible to quickly and efficiently locate lost parts and tools in the factory, ensuring transparency and reliability.

[0177] Key Word Definitions

[0178] A "user terminal" is a computing device used by a user, such as a smartphone or tablet.

[0179] A "server" is a computer system that communicates with user terminals over a network and processes and stores data.

[0180] "Image recognition means" is a technology for detecting objects from photographs and videos taken with a camera and analyzing their features.

[0181] A "database" is a system for efficiently storing, retrieving, and managing information.

[0182] "Natural language processing means" refers to technology that enables computers to understand, interpret, and generate human language.

[0183] "Blockchain" is a distributed digital ledger and technology that increases the transparency and reliability of information.

[0184] An "internal notification function" is a system that sends messages and alerts to specific users or devices.

[0185] "Factory environment" means the physical and working environment within a factory where products are manufactured.

[0186] "Voice input" is a method of inputting voice as data through a microphone.

[0187] "Text entry" is a method of entering characters and symbols as data using a keyboard or software.

[0188] MODE FOR CARRYING OUT THE INVENTION

[0189] The system for implementing this invention aims to efficiently prevent and quickly locate parts and tools lost in a factory environment. The technical elements that make up this system and their specific processing will be described below.

[0190] System Components

[0191] User terminal

[0192] User devices can be smartphones, tablets, or factory robots. These devices are used to take photos of parts and tools, and provide an interface for registering information, searching, and receiving notifications. The main software components are a mobile app and a client application for data transfer.

[0193] server

[0194] The server is the central component that communicates with user devices over the network, processes uploaded photos and text information, manages the database, notifies matching results, and records information on the blockchain. It includes the following main software and hardware components:

[0195] Image recognition: Uses image processing libraries such as TensorFlow and OpenCV to extract object features from uploaded photos.

[0196] Database: A relational database management system such as MySQL or PostgreSQL is used to store the analyzed feature information.

[0197] Natural language processing tools: NLP libraries such as BERT and spaCy are used to analyze the text information entered by the user and match it with object information in the database.

[0198] Notifications: Use notification services such as Firebase Cloud Messaging to notify users of match results and other important information.

[0199] Blockchain: Using Ethereum, Hyperledger, etc., matching information and delivery completion information are recorded on the blockchain to ensure transparency and reliability.

[0200] Usage and Operation Procedures

[0201] First, the user registers with the system using their user terminal, and the necessary information is stored in a database. Next, when the user registers information about a lost or found item, they take a photo using their terminal and send the photo along with detailed text information to the server. The server receives this data and analyzes it using image recognition and natural language processing. The analyzed data is then stored in a database and, if necessary, notifies the user of the matching results.

[0202] After the delivery is complete, the delivery details are also recorded on the blockchain again, ensuring overall transparency and reliability. This prevents loss of production efficiency due to lost parts or tools in the factory and ensures that needed items can be found quickly.

[0203] Specific examples

[0204] For example, if a "blue drill bit (50mm)" is lost in a factory, a worker launches the smartphone app and searches for "lost drill bit." They voice-type "blue drill bit, 50mm" into the app, and the AI ​​converts the voice to text, then searches the database for a matching part. The matching part information is notified, and the blue drill bit picked up by another worker is found. They communicate using the built-in chat function to complete the part transfer. The transfer completion information is then recorded on the blockchain.

[0205] Prompt sentence for generative AI model

[0206] User Input: "Blue drill bit, 50mm"

[0207] AI process: Converts voice input into text and searches a database for matching parts.

[0208] Result: Notify the customer of the matching part information and communicate with them via the internal chat function.

[0209] The above is a specific embodiment for implementing this system, which makes it possible to efficiently manage parts and tools in a factory environment and to manage information in a transparent and reliable manner.

[0210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0211] System processing steps

[0212] Processing flow

[0213] Step 1:

[0214] A user launches the app on a user device such as a smartphone or tablet and enters their own information (name, contact details, email address, etc.). This input information is sent from the user device to the server. The server receives the sent information and stores it in a database. This process registers the user's basic information within the system.

[0215] Input: Name, contact information, email address

[0216] Output: User information stored in the database

[0217] Step 2:

[0218] The user takes a photo of the part or tool they have picked up and enters the necessary text information (e.g., location, date and time, and object characteristics) into their user terminal. The user terminal then sends this information to the server. The server receives the photo data and text information, analyzes the photo data using image recognition tools, and extracts the object's characteristics (color, shape, size, etc.). The extracted information and the entered text information are stored in a database.

[0219] Input: Photo of the object, location, date and time, object characteristics

[0220] Output: Object features and text information stored in a database

[0221] Step 3:

[0222] When a user drops an object, they input feature information into their user terminal or transmit it via voice input. The user terminal then transmits the input feature information to the server. The server uses natural language processing to analyze the text and voice data and compares it with object information in a database. If the server finds matching object information, it notifies the user terminal of the details.

[0223] Input: Feature information (text or audio)

[0224] Output: Matching result notification

[0225] Step 4:

[0226] If the server finds a match, it records the information on the blockchain to ensure transparency and reliability. Information is securely recorded and cannot be tampered with.

[0227] Input: Matching results

[0228] Output: Information recorded on the blockchain

[0229] Step 5:

[0230] Users who receive a matching notification can use the app's chat function to confirm information about the items they found and lost, and to communicate details about the delivery. The server also manages chat data in real time.

[0231] Input: Chat message

[0232] Output: Send a message to the other user

[0233] Step 6:

[0234] After the item has been delivered, the user confirms the delivery in the app, and the server records this information back on the blockchain, ensuring transparency and authenticity of the transaction completion.

[0235] Input: Delivery completion information

[0236] Output: Delivery completion information recorded on the blockchain

[0237] Specific examples of processing

[0238] Step 1: User Registration

[0239] The user's device transmits the name, contact information, and email address entered in the app, which is then received by the server and stored in a MySQL database.

[0240] Step 2: Register your lost property

[0241] The user's device sends a photo of the object and text information to the server, which then uses TensorFlow to analyze the photo, extract the object's features, and store them in a database.

[0242] Step 3: Find your lost item

[0243] The user inputs the characteristics of the lost item via text or voice, and the user device sends this to the server. The server analyzes it using BERT, compares it with the database, and notifies the user device of any matches.

[0244] Step 4: Record your match information

[0245] The server records the matching results on the blockchain using Ethereum, which ensures the transparency and immutability of the information.

[0246] Step 5: Chat with us

[0247] Users who receive a match notification can contact each other using the app's chat function. The server sends and receives messages in real time using WebSocket.

[0248] Step 6: Record the completion of the handover

[0249] Once the transfer is complete, the user device sends the completion information to the server, which then records this information on the blockchain again to ensure the transparency of the transfer details.

[0250] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0251] A specific embodiment of the present invention is described below. The present invention is a system that quickly and accurately connects the person who found a lost item with the person who lost it, utilizing the image recognition and natural language processing capabilities of AI, as well as blockchain technology and an emotion engine to ensure the transparency and reliability of information.

[0252] User Registration

[0253] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. This information is received by the device and sent to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete.

[0254] Registering lost items

[0255] The user takes a photo of the lost item they found, enters the necessary text information (e.g., location, date, time, characteristics of the item, etc.), and sends it from the device to the server. The server analyzes the photo data using image recognition to extract detailed information about the item (color, shape, size, brand name, etc.). This extracted information is stored in a database along with the text information. The device is then notified that registration is complete.

[0256] Search for lost items

[0257] To search for a lost item, the user inputs characteristic information as text, which the device then sends to the server. The server then uses natural language processing to analyze the input text and compare it with object information in the database. If a matching item is found, the server notifies the user of the matching result; if not, it sends a "not found" notification.

[0258] Notification of matching results

[0259] The server then sends notifications to both the person who lost the item and the person who found it based on the information matched in the database. The server also records this matching information on the blockchain, ensuring transparency and reliability of the information.

[0260] Use of emotion engine

[0261] The server is equipped with an emotion engine that recognizes emotions from user input and actions. Matching notifications and chat responses are then reflected in the emotion engine's evaluation.

[0262] Communication and delivery

[0263] The user receives a notification and uses the app's chat function to contact the finder. After contact and confirmation of the details of the return, the emotion engine analyzes the user's emotional state and adjusts the response accordingly. Once the return is complete, the return is confirmed within the app, and the server records the return information on the blockchain.

[0264] Specific examples

[0265] As a concrete example, suppose someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and registers it by entering the text "I found this on a bench at the station." The server analyzes the photo, extracts the information "black leather wallet, approximately 15cm x 10cm," and stores it in the database. Meanwhile, if the person who lost the wallet uses the app to search "I lost my black leather wallet, approximately 15cm x 10cm," the server uses this information to search the database and matches it with the registered information of the person who found it. The matching results are notified to both parties, who communicate via chat and then hand over the wallet at a designated location. During this process, the server uses its emotion engine to analyze the user's emotional state and provide appropriate responses and notifications. After the handover is complete, the server records this information on the blockchain to ensure overall transparency and reliability.

[0266] In this way, the present invention is a system that allows users to easily register and search for lost items, performs quick and accurate matching, and ensures overall transparency and reliability by utilizing blockchain technology and an emotion engine.

[0267] The processing flow will be explained below.

[0268] Handling user registration

[0269] Step 1:

[0270] User: Launches the app and accesses the user registration screen.

[0271] Step 2:

[0272] User: Enter the required information such as name, contact details, and email address, and tap the "Register" button.

[0273] Step 3:

[0274] Terminal: The entered user information is sent to the server in JSON format.

[0275] Step 4:

[0276] Server: Analyzes the received user information and stores it in a database. Generates a response indicating registration completion and sends it to the terminal.

[0277] Step 5:

[0278] Terminal: Receives notification of registration completion and displays it to the user.

[0279] Processing lost and found entries

[0280] Step 1:

[0281] User: Access the app's "Lost Item Registration" screen to register the item they found.

[0282] Step 2:

[0283] User: Take a photo of the lost item, enter the necessary text information (e.g., location, date, time, item characteristics, etc.), and tap the "Register" button.

[0284] Step 3:

[0285] Terminal: Sends photo data and text information to the server.

[0286] Step 4:

[0287] Server: The received photo data is passed to the image recognition module, which analyzes the object's detailed information (color, shape, size, brand name, etc.).

[0288] Step 5:

[0289] Server: Combines the analysis results with the text information and stores them in a database. Sends a notification of registration completion to the device.

[0290] Step 6:

[0291] Terminal: Receives notification of registration completion and displays it to the user.

[0292] Processing lost and found searches

[0293] Step 1:

[0294] User: Access the app's "Lost Item Search" screen and enter the characteristics of the lost item.

[0295] Step 2:

[0296] User: Check the information entered and tap the "Search" button.

[0297] Step 3:

[0298] Terminal: Sends the input characteristic information to the server.

[0299] Step 4:

[0300] Server: Passes the received feature information to a natural language processing (NLP) module for analysis.

[0301] Step 5:

[0302] Server: Compares the analyzed information with the lost items information in the database.

[0303] Step 6:

[0304] Server: If a matching item is found, it generates a matching result and notifies the user device. If no matching item is found, it notifies the user device.

[0305] Step 7:

[0306] Terminal: Receives the matching result notification and displays it to the user.

[0307] Processing of match result notifications

[0308] Step 1:

[0309] Server: Checks the database for a match between the item the user dropped and the item they picked up.

[0310] Step 2:

[0311] Server: Notifies the users (those who dropped the item and those who picked it up) of the matching results.

[0312] Step 3:

[0313] Server: Records matching information on the blockchain to ensure transparency and reliability.

[0314] Use of emotion engine

[0315] Step 1:

[0316] Server: The emotion engine analyzes the text and behavioral data entered by the user and recognizes the user's emotions.

[0317] Step 2:

[0318] Server: Runs a program that adjusts notification content and chat responses based on the user's emotions.

[0319] Step 3:

[0320] Server: Sends emotion-based adjustment results to the user's device.

[0321] Step 4:

[0322] Terminal: Displaying tailored notifications and responses to the user.

[0323] Contact and delivery processing

[0324] Step 1:

[0325] Users: Receive in-app notifications, check match results, and use the chat feature to contact the finder.

[0326] Step 2:

[0327] Users (those who lost the item and those who found it): Use the chat function to communicate and decide the details of the delivery.

[0328] Step 3:

[0329] User (the person who lost the item): Receives the lost item from the finder at the designated location. The completion of the delivery is confirmed within the app.

[0330] Step 4:

[0331] User (the person who found the item): Confirm in the app that the item was returned safely.

[0332] Step 5:

[0333] Server: Receives information about the completion of the transfer and records it on the blockchain.

[0334] Specific examples

[0335] As a concrete example, let's say someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and enters the text information, "I found it on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database.

[0336] Meanwhile, if the person who lost their wallet uses the app to search for "I lost a black leather wallet. It's about 15cm x 10cm," the server uses this information to search its database and finds a match with the registered information of the person who found it. The matching results are notified to both parties, who communicate via chat and then hand over the wallet at a designated location. During this process, the server uses its emotion engine to analyze the user's emotional state and provide appropriate responses and notifications. After the handover is complete, the server records this information on the blockchain to ensure overall transparency and reliability.

[0337] In this way, the present invention is a system that allows users to easily register and search for lost items, performs quick and accurate matching, and ensures overall transparency and reliability by utilizing blockchain technology and an emotion engine.

[0338] Example 2

[0339] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0340] In recent years, there has been a demand for a means to efficiently connect the finder and the person who lost a lost item. However, conventional systems lack transparency and reliability in this matching process, making it difficult to respond in a way that appropriately reflects the user's feelings. In particular, delays and misunderstandings can occur in information sharing and notification, which can detract from the overall user experience. The present invention aims to solve these issues and provide a system that connects users quickly, accurately, and reliably.

[0341] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an information processing device that uploads an image of the lost item, an information processing device that receives the uploaded image and analyzes the object's features using image recognition means, means for saving the analyzed features in a memory area, an information processing device that receives feature information entered by the person who lost the item, analyzes it using natural language processing means, and compares it with the object information in the memory area, means for notifying the information processing device of a matching result based on the compared information, means for recording the matching information in a distributed ledger to ensure transparency and reliability, and means for analyzing the user's emotions using an emotion engine and adjusting the response content. This allows users to quickly and accurately register and search for lost items, ensures information transparency and reliability, and enables appropriate responses according to the user's emotions.

[0342] An "information processing device" is a device that inputs, processes, stores, and outputs data, and includes computers, servers, and the like.

[0343] "Image recognition means" refers to technology that analyzes image data and automatically recognizes objects and features, and specifically refers to computer vision and deep learning technology.

[0344] A "storage area" is an area for storing data and information, and includes databases and cloud storage.

[0345] "Natural language processing means" refers to technology that analyzes text data, understands its content, and extracts necessary information, and includes natural language processing libraries and models.

[0346] "Verifying" refers to the process of comparing input information with existing data to see if it matches.

[0347] "Means of notification" refers to means for conveying specific information to a user, and includes push notifications, emails, etc.

[0348] A "distributed ledger" is a technology that records and manages transaction information in a distributed manner to ensure overall transparency and reliability, and includes blockchain technology.

[0349] An "emotion engine" is a system that analyzes emotions from user input and behavior and generates appropriate responses, and includes emotion analysis libraries and models.

[0350] The present invention is a system for quickly and accurately connecting the finder and the person who lost the lost item, and it uses a combination of image recognition, natural language processing, blockchain, and an emotion engine. The system is configured as follows.

[0351] User Registration

[0352] First, the user launches the application and enters the required information, such as name, contact details, and email address. The device then sends the entered information to the server. The server then stores the received information in a database and sends a notification of registration completion to the device. This process can use an SQL database or cloud storage.

[0353] Registering lost items

[0354] The user takes a photo of the lost item they found and enters text information such as the location, date, time, and characteristics of the item. The device then sends this to the server. The server then uses image recognition to analyze the photo data and extract detailed information about the object (color, shape, size, brand name, etc.). Image recognition libraries such as OpenCV and TensorFlow are used for the analysis. The extracted information is stored in a database along with the text information. A notification that registration is complete is then sent to the device.

[0355] Search for lost items

[0356] To search for a lost item, the user inputs characteristic information as text. The device then sends this to the server. The server then uses natural language processing (e.g., a natural language processing library or generative AI model) to analyze the input text information and compare it with object information in a database. If a matching item is found, the result is notified to the user's device. If not, a "not found" notification is sent.

[0357] Notification of matching results

[0358] The server then sends a notification to both the person who lost the item and the person who found it based on the information matched in the database. The server also records the matching information on a distributed ledger to ensure the transparency and reliability of the information. This process uses blockchain technology.

[0359] Use of emotion engine

[0360] The server is equipped with an emotion engine that recognizes emotions from user input and behavior. An AI model is used to analyze user emotions and reflect them in matching notifications and chat responses. Emotion analysis libraries and models are used for emotion recognition.

[0361] Communication and delivery

[0362] The user receives a notification and uses the app's chat function to contact the finder. After contact and the details of the delivery are decided, the server analyzes the user's emotional state using an emotion engine and adjusts the response accordingly. After the delivery is completed, the user confirms this in the app, and the server records the completion of the delivery again on the distributed ledger.

[0363] Specific examples

[0364] If someone finds a wallet at a station, they open the app, take a photo of the wallet, and enter the text information, "I found this on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database. Meanwhile, if the person who lost the wallet uses the app to search for "I lost my black leather wallet, approximately 15cm x 10cm," the server uses this information to search the database and matches it with the finder's registered information. The matching results are notified to both parties, who communicate via chat and then hand over the wallet at a designated location. During this process, the server uses an emotion engine to analyze the user's emotional state and provide appropriate responses and notifications. After the handover is complete, the server records this information on the blockchain to ensure transparency and reliability.

[0365] Prompt Sentence Examples

[0366] "I found a black leather wallet at the station. It measures 15cm x 10cm."

[0367] "I dropped my black leather wallet at the station. Has anyone found it?"

[0368] In this way, the system of the present invention allows users to easily register and search for lost items, performs quick and accurate matching, and utilizes blockchain technology and an emotion engine to ensure overall transparency and reliability.

[0369] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0370] Step 1: User Registration

[0371] The user launches the app and enters the required information such as name, contact details, and email address. The device assembles this input information into packets, encrypts them using SSL / TLS, and sends them to the server. The server deserializes the received information and saves it in a database (e.g., MySQL). Once saved, the server generates a registration completion status, encrypts it, and sends it back to the device. The device notifies the user that "registration is complete." Input: Name, contact details, email address. Output: Notification of registration completion.

[0372] Step 2: Register your lost property

[0373] The user uses the app to take a photo of the lost item and enters text information such as location, date and time, and characteristics. The device then assembles this into a packet and sends it to the server. The server then analyzes the received photo data using OpenCV to extract detailed information about the object (color, shape, size, brand name, etc.). This extracted information is integrated with the text information and stored in a database. Once registration is complete, the server generates a registration completion status and notifies the device. Input: photo data, location, date and time, characteristics. Output: notification that registration is complete.

[0374] Step 3: Search for lost property

[0375] The user enters the characteristic information of the dropped item as text into the app. The device sends this to the server. The server analyzes the received text information using natural language processing means (e.g., a generative AI model) and generates a search query. The server then uses this query to search a database for matching object information. If a matching item is found, details about it are sent to the user's device; if not, a "not found" message is sent. Input: Characteristic information text. Output: Search results or a "not found" message.

[0376] Step 4: Notification of match results

[0377] The server generates a notification message to both the person who dropped the item and the person who found it based on the matching results in the database. This message is encrypted and sent to the terminal. At the same time, the server records the matching information on the blockchain and generates a hash value for the transaction. Input: Matching result. Output: Notification message, hash value recorded on the blockchain.

[0378] Step 5: Use the Emotion Engine

[0379] The emotion engine built into the server analyzes user behavior and input in real time. It uses an AI model (e.g., emotion analysis library) to assess the user's emotional state and appropriately tailor responses and notifications based on this. Input: User behavior data, input text. Output: Emotion assessment, tailored responses and notifications.

[0380] Step 6: Communication and delivery

[0381] The user receives a matching notification and uses the app's chat function to contact the finder. The server collects the chat content and, if necessary, reanalyzes it using the emotion engine to adjust the response. After communication is established, the users decide on the details of the transfer. Once the transfer is complete, the user confirms it within the app, which the server receives and records on the blockchain. Input: Chat content, transfer information. Output: Emotion analysis results, transfer completion information recorded on the blockchain.

[0382] (Application example 2)

[0383] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0384] There is a need to quickly and accurately connect lost items with their owners, but existing systems do not ensure information transparency or reliability, and are unable to properly recognize and respond to emotions, resulting in a lack of sufficient improvement in the user experience. These issues are particularly likely to become apparent when managing and handing over lost items in physical stores.

[0385] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0386] In this invention, the server includes a user terminal that uploads a photo of the lost item, means for receiving the uploaded photo and analyzing the object's features using image recognition means, means for saving the analyzed features in a database, means for receiving feature information entered by the person who lost the item, analyzing it using natural language processing means, and comparing it with object information in the database, means for notifying the user terminal of a matching result based on the compared information, means for recording the matching information in a blockchain to ensure transparency and reliability, means for analyzing the user's emotional state using an emotion engine that recognizes emotions from the user's input and actions, and means for supporting the registration and search of lost items in physical stores. This enables fast and accurate registration, search, and delivery of lost items in physical stores.

[0387] "User terminal" refers to a device used to upload photos of lost items, including smartphones and smart glasses.

[0388] The "server" is a central system that receives uploaded photos, analyzes them using image recognition and natural language processing tools, and compares them with a database.

[0389] "Image recognition means" refers to technology for analyzing the characteristics of objects in uploaded photos and extracting information about them, and includes AI models and OCR (optical character recognition).

[0390] A "database" is a digital storage system used to store and later collate analyzed feature information and user-entered information.

[0391] "Natural language processing means" is a technology for analyzing text information entered by a user and comparing that information with object information in a database.

[0392] The "matching result" is the result of matching information between the lost item and the owner, which is generated based on the collated information.

[0393] "Blockchain" is a distributed ledger technology used to ensure transparency and reliability of information, with each transaction and record encrypted and connected in a chain.

[0394] An "emotion engine" is a technology for recognizing and analyzing emotions from user input and behavior.

[0395] "Physical store" refers to a sales or service establishment operated in a physical location, including commercial facilities where many lost items may occur.

[0396] "Registration and search support" means providing support to efficiently register and search for lost items in physical stores.

[0397] A specific embodiment of the present invention is described below. The present invention is a system that quickly and accurately connects the person who found a lost item in a physical store with the person who lost it, utilizing AI's image recognition and natural language processing capabilities, as well as blockchain technology and an emotion engine to ensure the transparency and reliability of information.

[0398] User Registration

[0399] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. This information is received by the device and sent to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete.

[0400] Registering lost items

[0401] The user takes a photo of the lost item, enters the necessary text information (e.g., location, date, time, characteristics of the item, etc.), and sends it from the device to the server. The server analyzes the photo data using image recognition to extract detailed information about the item (color, shape, size, brand name, etc.). This extracted information is stored in a database along with the text information. The device is then notified that registration is complete.

[0402] Search for lost items

[0403] To search for a lost item, the user inputs characteristic information as text, which the device then sends to the server. The server then uses natural language processing to analyze the input text and compare it with object information in the database. If a matching item is found, the server notifies the user of the matching result; if not, it sends a "not found" notification.

[0404] Notification of matching results

[0405] The server then sends notifications to both the person who lost the item and the person who found it based on the information matched in the database, and records this matching information on the blockchain to ensure the transparency and reliability of the information.

[0406] Use of emotion engine

[0407] The server is equipped with an emotion engine that recognizes emotions from user input and actions. Matching notifications and chat responses are then reflected in the emotion engine's evaluation.

[0408] Communication and delivery

[0409] The user receives a notification and uses the app's chat function to contact the finder. After contact and confirmation of the details of the return, the emotion engine analyzes the user's emotional state and adjusts the response accordingly. Once the return is complete, the return is confirmed within the app, and the server records the return information on the blockchain.

[0410] Hardware and software used

[0411] Hardware:

[0412] Smartphone: taking photos using the camera function, internet connection

[0413] Smart glasses: Photo and display functions

[0414] Robot: Lost item detection, voice guidance

[0415] software:

[0416] TensorFlow: for image recognition

[0417] pytesseract:OCR (optical character recognition)

[0418] Blockchain Library: Blockchain Technology

[0419] Natural Language Toolkit (NLTK): Natural language parsing and sentiment analysis

[0420] Specific examples

[0421] For example, if a black leather jacket is found in a store lobby, a staff member would take a photo of the black leather jacket using their smartphone and input information by voice or text, such as "Black leather jacket found in the store lobby." The system would then automatically extract detailed information about the item using image recognition and store it in a database. When a customer says something like "I lost my keys" at the cash register or information counter, the staff member would simply use the app to search for it. Matching is performed through image recognition and natural language processing, and the results are notified. Transparency is ensured by recording the information on the blockchain.

[0422] Example prompt:

[0423] "Analyze the image, extract the object features from the captured image, and link them with the text information. This will provide the best matching results based on the feature information and image information entered by the user."

[0424] This will enable quick and accurate registration, search, and delivery of lost items in physical stores.

[0425] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0426] Step 1:

[0427] The user launches the app and enters the required information, such as their name, contact details, and email address.

[0428] Input: Name, contact details, email address

[0429] Operation: The terminal receives the information entered by the user.

[0430] Output: User information entered

[0431] Step 2:

[0432] The terminal transmits the received user information to the server.

[0433] Input: Received user information

[0434] Operation: The terminal processes and sends user information to the server.

[0435] Output: User information sent to the server

[0436] Step 3:

[0437] The server stores the received user information in a database and sends a response indicating that registration is complete to the terminal.

[0438] Input: User information sent to the server

[0439] What happens: The server saves the user information in its database and generates a response confirming registration.

[0440] Output: Registration complete response

[0441] Step 4:

[0442] The user takes a photo of the lost item and enters text information such as the location, date, and characteristics of the item.

[0443] Input: Photo of lost item, text information

[0444] How it works: The user takes a photo on their device and enters the required information.

[0445] Output: Photo data and text information of lost items

[0446] Step 5:

[0447] The device sends photo data and text information about the lost item to the server.

[0448] Input: Photo data and text information of lost items

[0449] How it works: The device processes and sends the photo data and text information to the server.

[0450] Output: Photo data and text information sent to the server

[0451] Step 6:

[0452] The server analyzes the photo data using image recognition tools and extracts detailed information about the object (color, shape, size, brand name, etc.).

[0453] Input: Photo data sent to the server

[0454] How it works: Analyzes photo data using image recognition tools (e.g. TensorFlow) and extracts object features.

[0455] Output: Detailed information about what was parsed

[0456] Step 7:

[0457] The server stores the extracted information together with the text information in a database.

[0458] Input: Detailed information and text of the parsed object

[0459] How it works: Saves text information and extracted object features in a database.

[0460] Output: Information stored in the database

[0461] Step 8:

[0462] To search for a lost item, the user inputs characteristic information in text, which is then sent by the terminal to the server.

[0463] Input: Characteristic information text

[0464] Operation: The user inputs characteristic information, which the device sends to the server.

[0465] Output: Feature information sent to the server

[0466] Step 9:

[0467] The server analyzes the input feature information using natural language processing means and compares it with object information in the database.

[0468] Input: Feature information sent to the server

[0469] How it works: Analyzes the feature information using natural language processing tools (e.g., NLTK) and matches it with object information in a database.

[0470] Output: Search results

[0471] Step 10:

[0472] The server notifies the user of the matching results. If a matching item is found, the server notifies the user, and if not, it sends a "not found" notification.

[0473] Input: Search results

[0474] Operation: The server sends the search results to the user's device.

[0475] Output: Search results notified to the user's device

[0476] Step 11:

[0477] The server records the matching information on the blockchain to ensure transparency and reliability.

[0478] Input: Matching information

[0479] What it does: Records matching information in a blockchain library.

[0480] Output: Information recorded on the blockchain

[0481] Step 12:

[0482] The server is equipped with an emotion engine that recognizes emotions from user input and actions.

[0483] Input: User input and behavior information

[0484] How it works: Recognize and analyze user emotions with the emotion engine.

[0485] Output: Emotional state analysis result

[0486] Step 13:

[0487] The server analyzes the user's emotional state and reflects this in matching notifications and responses in the chat function.

[0488] Input: Emotional state analysis result

[0489] What it does: Adjusts response content based on emotion engine analysis.

[0490] Output: The adjusted response

[0491] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0492] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0493] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0494] [Second embodiment]

[0495] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0496] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0497] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0498] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0499] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0500] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0501] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0502] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0503] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0504] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0505] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0506] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0507] A specific embodiment of the present invention is described below. The present invention is a system that quickly and accurately connects the person who found a lost item with the person who lost it, utilizing the image recognition and natural language processing capabilities of AI, and further using blockchain technology to ensure the transparency and reliability of information.

[0508] User Registration

[0509] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. This information is received by the device and sent to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete.

[0510] Registering lost items

[0511] The user takes a photo of the lost item they found, enters the necessary text information (e.g., location, date, time, characteristics of the item, etc.), and sends it from the device to the server. The server analyzes the photo data using image recognition to extract detailed information about the item (color, shape, size, brand name, etc.). This extracted information is stored in a database along with the text information. The device is then notified that registration is complete.

[0512] Search for lost items

[0513] To search for a lost item, the user inputs characteristic information as text, which the device then sends to the server. The server then uses natural language processing to analyze the input text and compare it with object information in the database. If a matching item is found, the server notifies the user of the matching result; if not, it sends a "not found" notification.

[0514] Notification of matching results

[0515] The server then sends notifications to both the person who lost the item and the person who found it based on the information matched in the database. The server also records this matching information on the blockchain, ensuring transparency and reliability of the information.

[0516] Communication and delivery

[0517] The user receives a notification and communicates with the finder using the chat function within the app. After communicating with the finder, the details of the delivery are decided. After the delivery is completed, it is confirmed within the app, and the server records the delivery completion information again on the blockchain.

[0518] Specific examples

[0519] As a concrete example, let's say someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and enters the text information, "I found it on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database.

[0520] Meanwhile, if the person who lost their wallet uses the app to search for "I lost a black leather wallet. It's about 15cm x 10cm," the server will use this information to search the database and match it with the registered information of the person who found it. The matching results are notified to both parties, who then communicate via chat and hand over the wallet at a designated location. After the handover is complete, the server records this information on the blockchain to ensure overall transparency and reliability.

[0521] In this way, the present invention is a system that allows users to easily register and search for lost items, performs quick and accurate matching, and utilizes blockchain technology to ensure overall transparency and reliability.

[0522] The processing flow will be explained below.

[0523] Handling user registration

[0524] Step 1:

[0525] User: Launches the app and accesses the user registration screen.

[0526] Step 2:

[0527] User: Enter the required information such as name, contact details, and email address, and tap the "Register" button.

[0528] Step 3:

[0529] Terminal: The entered user information is sent to the server in JSON format.

[0530] Step 4:

[0531] Server: Analyzes the received user information and stores it in a database. Generates a response indicating registration completion and sends it to the terminal.

[0532] Step 5:

[0533] Terminal: Receives notification of registration completion and displays it to the user.

[0534] Processing lost and found entries

[0535] Step 1:

[0536] User: Access the app's "Lost Item Registration" screen to register the item they found.

[0537] Step 2:

[0538] User: Take a photo of the lost item, enter the necessary text information (e.g., location, date, time, item characteristics, etc.), and tap the "Register" button.

[0539] Step 3:

[0540] Terminal: Sends photo data and text information to the server.

[0541] Step 4:

[0542] Server: The received photo data is passed to the image recognition module, which analyzes the object's detailed information (color, shape, size, brand name, etc.).

[0543] Step 5:

[0544] Server: Combines the analysis results with the text information and stores them in a database. Sends a notification of registration completion to the device.

[0545] Step 6:

[0546] Terminal: Receives notification of registration completion and displays it to the user.

[0547] Processing lost and found searches

[0548] Step 1:

[0549] User: Access the app's "Lost Item Search" screen and enter the characteristics of the lost item.

[0550] Step 2:

[0551] User: Check the information entered and tap the "Search" button.

[0552] Step 3:

[0553] Terminal: Sends the input characteristic information to the server.

[0554] Step 4:

[0555] Server: Passes the received feature information to the natural language processing module for analysis.

[0556] Step 5:

[0557] Server: Compares the analyzed information with the lost items information in the database.

[0558] Step 6:

[0559] Server: If a matching item is found, it generates a matching result and notifies the user's device. If not found, it notifies the user.

[0560] Step 7:

[0561] Terminal: Receives the matching result notification and displays it to the user.

[0562] Processing of match result notifications

[0563] Step 1:

[0564] Server: Checks the database for a match between the user's dropped item and the item they found.

[0565] Step 2:

[0566] Server: Notifies the users (those who dropped the item and those who picked it up) of the matching results.

[0567] Step 3:

[0568] Server: Records the matching information immutably on the blockchain.

[0569] Contact and delivery processing

[0570] Step 1:

[0571] Users: Receive in-app notifications, check match results, and use the chat feature to contact the finder.

[0572] Step 2:

[0573] Users (those who lost the item and those who found it): Use the chat function to communicate and decide the details of the delivery.

[0574] Step 3:

[0575] User (the person who lost the item): Receives the lost item from the finder at the designated location. The completion of the delivery is confirmed within the app.

[0576] Step 4:

[0577] User (the person who found the item): Confirm in the app that the item was returned safely.

[0578] Step 5:

[0579] Server: Receives information about the completion of the transfer and records it on the blockchain.

[0580] The above is the specific operation of each processing step. The user, device, and server work together to efficiently complete the series of processes of registering, searching, matching, and handing over lost items.

[0581] Example 1

[0582] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0583] In conventional lost property management systems, it was difficult to quickly and accurately connect the person who found the lost item with the person who dropped it, and the transparency and reliability of information were not sufficiently ensured.In addition, the means of communication between users and information management after the handover were insufficient, which reduced the reliability of the entire system.

[0584] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0585] In this invention, the server includes a terminal for users to upload photos of lost items, an information processing device that receives the uploaded photos and analyzes the features of the object using image recognition means, means for saving the analyzed features in a storage device, an information processing device that receives feature information entered by the user who lost the item, analyzes it using natural language processing means, and compares it with the object information in the storage device, means for notifying the user of a matching result based on the generated comparison information, means for recording the matching information in a fixed ledger technology to ensure information transparency and reliability, means for users to receive the notification and communicate using a chat function, and means for recording information after the transfer is completed in the fixed ledger technology again. This allows the person who found the lost item to quickly and accurately connect with the person who dropped it, ensures information transparency and reliability, allows users to efficiently communicate with each other, and enables information management after the transfer is completed.

[0586] A "user terminal" is a device for uploading photos of lost items, and has the function of transmitting the input information to a server.

[0587] An "information processing device" is a device that has the function of analyzing input information using image recognition means and natural language processing means and storing the information in a storage device called a database.

[0588] "Image recognition means" refers to technology or software that analyzes uploaded photographs and identifies the characteristics of objects contained within them.

[0589] "Natural language processing means" refers to technology or software that analyzes text information entered by a user and understands its meaning and intent.

[0590] The "chat function" is a communication function that allows users to exchange messages and information in real time.

[0591] "Fixed ledger technology" is a technology similar to blockchain technology that preserves information once recorded with transparency and reliability, without being tampered with.

[0592] A "memory device" is a database or other storage system for storing analyzed features and other data.

[0593] "Matching results" refer to the server analyzing the characteristic information provided by the user and comparing it with information in the database to identify matching lost item information.

[0594] "Information on completion of delivery" is data indicating that the delivery of the lost item, which was determined through communication between users, has been completed.

[0595] This invention is a system that quickly and accurately connects the person who found a lost item with the person who lost it, utilizing image recognition and natural language processing capabilities, as well as blockchain technology to ensure the transparency and reliability of information.

[0596] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. The device receives this information and sends it to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete. The server uses an SQL database such as MySQL or PostgreSQL.

[0597] Next, the user takes a photo of the lost item and sends the necessary text information (e.g., location, date, time, and characteristics of the item) from the device to the server. The server analyzes the received photo using TensorFlow and OpenCV to extract detailed information about the item (color, shape, size, brand name, etc.). These details are stored in a database along with the text information, and the device is notified that registration is complete.

[0598] To search for a lost item, the user inputs feature information as text, which the device then sends to the server. The server then analyzes the input text using a natural language processing model such as BERT and compares it with object information in a database. If a matching item is found, the server notifies the user's device of the matching result. If the item is not found, the server sends a "not found" notification.

[0599] The server then sends notifications to both the person who lost the item and the person who found it based on the information collated in the database.The server also records this matching information on a blockchain such as Ethereum, ensuring the transparency and reliability of the information.

[0600] The user receives a notification and uses the chat function within the app (for example, Firebase chat service) to communicate with the finder. After communicating, they decide the details of the delivery. After the delivery is completed, this is confirmed within the app, and the server records the delivery completion information again on the blockchain.

[0601] As a concrete example, let's say someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and enters the text information, "I found it on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database.

[0602] Meanwhile, if the person who lost their wallet uses the app to search for "I lost a black leather wallet. It's about 15cm x 10cm," the server will use this information to search the database and match it with the registered information of the person who found it. The matching results are notified to both parties, who then communicate via chat and hand over the wallet at a designated location. After the handover is complete, the server records this information on the blockchain, ensuring overall transparency and reliability.

[0603] In this way, users can easily register and search for lost items, and matching can be done quickly and accurately, while blockchain technology is used to ensure overall transparency and reliability.

[0604] Example prompt sentence:

[0605] "I found a black leather wallet on a bench at the station. It measures 15cm x 10cm. How can I find the owner of this wallet?"

[0606] The above is an embodiment of the present invention.

[0607] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0608] Step 1: User Registration

[0609] The user uses the device and launches the app. The user enters the required information, such as their name, contact details, and email address. The device receives this information and sends it to the server. The server analyzes the received information and stores it in a database. It then generates a response indicating registration is complete and sends it to the device.

[0610] Input: Name, contact details, and email address entered by the user.

[0611] Data processing: Sending data from the device to the server.

[0612] Data calculation: Data analysis and database storage on the server.

[0613] Output: Registration complete response.

[0614] What happens: When a user clicks the "Register" button, the information is sent to the server and confirmed to be saved in the database. A notification of successful registration is displayed on the device.

[0615] Step 2: Register your lost property

[0616] The user uses their device to take a photo of the lost item and uploads it to the app. The user then enters text information such as the location, date, and item characteristics, and sends it from the device to the server. The server then analyzes the received photo data using image recognition techniques (e.g., TensorFlow or OpenCV) to extract detailed information about the item (color, shape, size, brand name, etc.). The extracted information and text information are then stored in a database, and the user is notified that registration is complete.

[0617] Input: A photo taken by the user and text information.

[0618] Data processing: Sending photos and text information from the device to the server.

[0619] Data calculation: Analyzing photo data and extracting detailed information using image recognition methods.

[0620] Output: Notification that registration is complete.

[0621] Specific operation: When a user presses the "Register" button, the photo and text information are sent to the server, where image analysis is performed. The results are stored in a database, and a notification of registration completion is displayed on the device.

[0622] Step 3: Search for lost property

[0623] The user uses their device to input textual information about the features of the dropped item, which is then sent from the device to the server. The server then uses natural language processing (e.g., the BERT model) to analyze the input feature information and compare it with object information in the database. If a matching item is found, a matching result is generated and notified to the user's device.

[0624] Input: Feature information entered by the user.

[0625] Data processing: Sending characteristic information from the device to the server.

[0626] Data Computing: Analysis of feature information and database search using natural language processing tools.

[0627] Output: Matching result or "not found" notification.

[0628] Specific operation: When the user presses the "Search" button, the characteristic information is analyzed and compared with the information in the database. The results are displayed on the device.

[0629] Step 4: Notification of match results

[0630] The server then sends notifications to both the person who lost the item and the person who found it based on the matched information in the database, and records this matching information on the blockchain to ensure the transparency and reliability of the information.

[0631] Input: The matched information.

[0632] Data processing: Creating notifications of matching results.

[0633] Data calculation: Blockchain recording of matching information.

[0634] Output: Notification of matching results and recording to the blockchain.

[0635] Specific operation: The server checks the matching information, generates a notification and sends it to the user. The result is displayed on the device and simultaneously recorded on the blockchain.

[0636] Step 5: Communication and delivery

[0637] The user receives a notification and uses the chat function within the app (for example, Firebase chat service) to communicate with the finder. After communicating, they decide the details of the delivery. After the delivery is completed, this is confirmed within the app, and the server records the delivery completion information again on the blockchain.

[0638] Input: Completed handover information.

[0639] Data processing: Communication via chat and confirmation of completion of delivery.

[0640] Data calculation: Blockchain recording of delivery completion information.

[0641] Output: A confirmation that the transaction has been completed and recorded on the blockchain.

[0642] How it works: Users communicate and decide on the details of the transaction. Once the transaction is complete, the information is recorded on the blockchain.

[0643] The above is a detailed description of each processing step of the system.

[0644] (Application example 1)

[0645] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0646] In modern factory environments, lost parts and tools have a significant impact on production efficiency. Work is often delayed when workers are unable to find the parts they need. Managing and locating lost items requires a great deal of time and effort, creating a need for a system that can quickly and efficiently resolve this issue. Furthermore, measures to ensure transparency and reliability are insufficient, and this also requires improvement.

[0647] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0648] In this invention, the server includes a means for preventing and quickly locating parts and tools lost in a factory environment, a means for searching for parts information by voice input or text input and analyzing it with natural language processing means, and a means for notifying the owner of the lost item with an internal notification function. This makes it possible to quickly and efficiently locate lost parts and tools in the factory, ensuring transparency and reliability.

[0649] Key Word Definitions

[0650] A "user terminal" is a computing device used by a user, such as a smartphone or tablet.

[0651] A "server" is a computer system that communicates with user terminals over a network and processes and stores data.

[0652] "Image recognition means" is a technology for detecting objects from photographs and videos taken with a camera and analyzing their features.

[0653] A "database" is a system for efficiently storing, retrieving, and managing information.

[0654] "Natural language processing means" refers to technology that enables computers to understand, interpret, and generate human language.

[0655] "Blockchain" is a distributed digital ledger and technology that increases the transparency and reliability of information.

[0656] An "internal notification function" is a system that sends messages and alerts to specific users or devices.

[0657] "Factory environment" means the physical and working environment within a factory where products are manufactured.

[0658] "Voice input" is a method of inputting voice as data through a microphone.

[0659] "Text entry" is a method of entering characters and symbols as data using a keyboard or software.

[0660] MODE FOR CARRYING OUT THE INVENTION

[0661] The system for implementing this invention aims to efficiently prevent and quickly locate parts and tools lost in a factory environment. The technical elements that make up this system and their specific processing will be described below.

[0662] System Components

[0663] User terminal

[0664] User devices can be smartphones, tablets, or factory robots. These devices are used to take photos of parts and tools, and provide an interface for registering information, searching, and receiving notifications. The main software components are a mobile app and a client application for data transfer.

[0665] server

[0666] The server is the central component that communicates with user devices over the network, processes uploaded photos and text information, manages the database, notifies matching results, and records information on the blockchain. It includes the following main software and hardware components:

[0667] Image recognition: Uses image processing libraries such as TensorFlow and OpenCV to extract object features from uploaded photos.

[0668] Database: A relational database management system such as MySQL or PostgreSQL is used to store the analyzed feature information.

[0669] Natural language processing tools: NLP libraries such as BERT and spaCy are used to analyze the text information entered by the user and match it with object information in the database.

[0670] Notifications: Use notification services such as Firebase Cloud Messaging to notify users of match results and other important information.

[0671] Blockchain: Using Ethereum, Hyperledger, etc., matching information and delivery completion information are recorded on the blockchain to ensure transparency and reliability.

[0672] Usage and Operation Procedures

[0673] First, the user registers with the system using their user terminal, and the necessary information is stored in a database. Next, when the user registers information about a lost or found item, they take a photo using their terminal and send the photo along with detailed text information to the server. The server receives this data and analyzes it using image recognition and natural language processing. The analyzed data is then stored in a database and, if necessary, notifies the user of the matching results.

[0674] After the delivery is complete, the delivery details are also recorded on the blockchain again, ensuring overall transparency and reliability. This prevents loss of production efficiency due to lost parts or tools in the factory and ensures that needed items can be found quickly.

[0675] Specific examples

[0676] For example, if a "blue drill bit (50mm)" is lost in a factory, a worker launches the smartphone app and searches for "lost drill bit." They voice-type "blue drill bit, 50mm" into the app, and the AI ​​converts the voice to text, then searches the database for a matching part. The matching part information is notified, and the blue drill bit picked up by another worker is found. They communicate using the built-in chat function to complete the part transfer. The transfer completion information is then recorded on the blockchain.

[0677] Prompt sentence for generative AI model

[0678] User Input: "Blue drill bit, 50mm"

[0679] AI process: Converts voice input into text and searches a database for matching parts.

[0680] Result: Notify the customer of the matching part information and communicate with them via the internal chat function.

[0681] The above is a specific embodiment for implementing this system, which makes it possible to efficiently manage parts and tools in a factory environment and to manage information in a transparent and reliable manner.

[0682] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0683] System processing steps

[0684] Processing flow

[0685] Step 1:

[0686] A user launches the app on a user device such as a smartphone or tablet and enters their own information (name, contact details, email address, etc.). This input information is sent from the user device to the server. The server receives the sent information and stores it in a database. This process registers the user's basic information within the system.

[0687] Input: Name, contact information, email address

[0688] Output: User information stored in the database

[0689] Step 2:

[0690] The user takes a photo of the part or tool they have picked up and enters the necessary text information (e.g., location, date and time, and object characteristics) into their user terminal. The user terminal then sends this information to the server. The server receives the photo data and text information, analyzes the photo data using image recognition tools, and extracts the object's characteristics (color, shape, size, etc.). The extracted information and the entered text information are stored in a database.

[0691] Input: Photo of the object, location, date and time, object characteristics

[0692] Output: Object features and text information stored in a database

[0693] Step 3:

[0694] When a user drops an object, they input feature information into their user terminal or transmit it via voice input. The user terminal then transmits the input feature information to the server. The server uses natural language processing to analyze the text and voice data and compares it with object information in a database. If the server finds matching object information, it notifies the user terminal of the details.

[0695] Input: Feature information (text or audio)

[0696] Output: Matching result notification

[0697] Step 4:

[0698] If the server finds a match, it records the information on the blockchain to ensure transparency and reliability. Information is securely recorded and cannot be tampered with.

[0699] Input: Matching results

[0700] Output: Information recorded on the blockchain

[0701] Step 5:

[0702] Users who receive a matching notification can use the app's chat function to confirm information about the items they found and lost, and to communicate details about the delivery. The server also manages chat data in real time.

[0703] Input: Chat message

[0704] Output: Send a message to the other user

[0705] Step 6:

[0706] After the item has been delivered, the user confirms the delivery in the app, and the server records this information back on the blockchain, ensuring transparency and authenticity of the transaction completion.

[0707] Input: Delivery completion information

[0708] Output: Delivery completion information recorded on the blockchain

[0709] Specific examples of processing

[0710] Step 1: User Registration

[0711] The user's device transmits the name, contact information, and email address entered in the app, which is then received by the server and stored in a MySQL database.

[0712] Step 2: Register your lost property

[0713] The user's device sends a photo of the object and text information to the server, which then uses TensorFlow to analyze the photo, extract the object's features, and store them in a database.

[0714] Step 3: Find your lost item

[0715] The user inputs the characteristics of the lost item via text or voice, and the user device sends this to the server. The server analyzes it using BERT, compares it with the database, and notifies the user device of any matches.

[0716] Step 4: Record your match information

[0717] The server records the matching results on the blockchain using Ethereum, which ensures the transparency and immutability of the information.

[0718] Step 5: Chat with us

[0719] Users who receive a match notification can contact each other using the app's chat function. The server sends and receives messages in real time using WebSocket.

[0720] Step 6: Record the completion of the handover

[0721] Once the transfer is complete, the user device sends the completion information to the server, which then records this information on the blockchain again to ensure the transparency of the transfer details.

[0722] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0723] A specific embodiment of the present invention is described below. The present invention is a system that quickly and accurately connects the person who found a lost item with the person who lost it, utilizing the image recognition and natural language processing capabilities of AI, as well as blockchain technology and an emotion engine to ensure the transparency and reliability of information.

[0724] User Registration

[0725] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. This information is received by the device and sent to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete.

[0726] Registering lost items

[0727] The user takes a photo of the lost item they found, enters the necessary text information (e.g., location, date, time, characteristics of the item, etc.), and sends it from the device to the server. The server analyzes the photo data using image recognition to extract detailed information about the item (color, shape, size, brand name, etc.). This extracted information is stored in a database along with the text information. The device is then notified that registration is complete.

[0728] Search for lost items

[0729] To search for a lost item, the user inputs characteristic information as text, which the device then sends to the server. The server then uses natural language processing to analyze the input text and compare it with object information in the database. If a matching item is found, the server notifies the user of the matching result; if not, it sends a "not found" notification.

[0730] Notification of matching results

[0731] The server then sends notifications to both the person who lost the item and the person who found it based on the information matched in the database. The server also records this matching information on the blockchain, ensuring transparency and reliability of the information.

[0732] Use of emotion engine

[0733] The server is equipped with an emotion engine that recognizes emotions from user input and actions. Matching notifications and chat responses are then reflected in the emotion engine's evaluation.

[0734] Communication and delivery

[0735] The user receives a notification and uses the app's chat function to contact the finder. After contact and confirmation of the details of the return, the emotion engine analyzes the user's emotional state and adjusts the response accordingly. Once the return is complete, the return is confirmed within the app, and the server records the return information on the blockchain.

[0736] Specific examples

[0737] As a concrete example, suppose someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and registers it by entering the text "I found this on a bench at the station." The server analyzes the photo, extracts the information "black leather wallet, approximately 15cm x 10cm," and stores it in the database. Meanwhile, if the person who lost the wallet uses the app to search "I lost my black leather wallet, approximately 15cm x 10cm," the server uses this information to search the database and matches it with the registered information of the person who found it. The matching results are notified to both parties, who communicate via chat and then hand over the wallet at a designated location. During this process, the server uses its emotion engine to analyze the user's emotional state and provide appropriate responses and notifications. After the handover is complete, the server records this information on the blockchain to ensure overall transparency and reliability.

[0738] In this way, the present invention is a system that allows users to easily register and search for lost items, performs quick and accurate matching, and ensures overall transparency and reliability by utilizing blockchain technology and an emotion engine.

[0739] The processing flow will be explained below.

[0740] Handling user registration

[0741] Step 1:

[0742] User: Launches the app and accesses the user registration screen.

[0743] Step 2:

[0744] User: Enter the required information such as name, contact details, and email address, and tap the "Register" button.

[0745] Step 3:

[0746] Terminal: The entered user information is sent to the server in JSON format.

[0747] Step 4:

[0748] Server: Analyzes the received user information and stores it in a database. Generates a response indicating registration completion and sends it to the terminal.

[0749] Step 5:

[0750] Terminal: Receives notification of registration completion and displays it to the user.

[0751] Processing lost and found entries

[0752] Step 1:

[0753] User: Access the app's "Lost Item Registration" screen to register the item they found.

[0754] Step 2:

[0755] User: Take a photo of the lost item, enter the necessary text information (e.g., location, date, time, item characteristics, etc.), and tap the "Register" button.

[0756] Step 3:

[0757] Terminal: Sends photo data and text information to the server.

[0758] Step 4:

[0759] Server: The received photo data is passed to the image recognition module, which analyzes the object's detailed information (color, shape, size, brand name, etc.).

[0760] Step 5:

[0761] Server: Combines the analysis results with the text information and stores them in a database. Sends a notification of registration completion to the device.

[0762] Step 6:

[0763] Terminal: Receives notification of registration completion and displays it to the user.

[0764] Processing lost and found searches

[0765] Step 1:

[0766] User: Access the app's "Lost Item Search" screen and enter the characteristics of the lost item.

[0767] Step 2:

[0768] User: Check the information entered and tap the "Search" button.

[0769] Step 3:

[0770] Terminal: Sends the input characteristic information to the server.

[0771] Step 4:

[0772] Server: Passes the received feature information to a natural language processing (NLP) module for analysis.

[0773] Step 5:

[0774] Server: Compares the analyzed information with the lost items information in the database.

[0775] Step 6:

[0776] Server: If a matching item is found, it generates a matching result and notifies the user device. If no matching item is found, it notifies the user device.

[0777] Step 7:

[0778] Terminal: Receives the matching result notification and displays it to the user.

[0779] Processing of match result notifications

[0780] Step 1:

[0781] Server: Checks the database for a match between the item the user dropped and the item they picked up.

[0782] Step 2:

[0783] Server: Notifies the users (those who dropped the item and those who picked it up) of the matching results.

[0784] Step 3:

[0785] Server: Records matching information on the blockchain to ensure transparency and reliability.

[0786] Use of emotion engine

[0787] Step 1:

[0788] Server: The emotion engine analyzes the text and behavioral data entered by the user and recognizes the user's emotions.

[0789] Step 2:

[0790] Server: Runs a program that adjusts notification content and chat responses based on the user's emotions.

[0791] Step 3:

[0792] Server: Sends emotion-based adjustment results to the user's device.

[0793] Step 4:

[0794] Terminal: Displaying tailored notifications and responses to the user.

[0795] Contact and delivery processing

[0796] Step 1:

[0797] Users: Receive in-app notifications, check match results, and use the chat feature to contact the finder.

[0798] Step 2:

[0799] Users (those who lost the item and those who found it): Use the chat function to communicate and decide the details of the delivery.

[0800] Step 3:

[0801] User (the person who lost the item): Receives the lost item from the finder at the designated location. The completion of the delivery is confirmed within the app.

[0802] Step 4:

[0803] User (the person who found the item): Confirm in the app that the item was returned safely.

[0804] Step 5:

[0805] Server: Receives information about the completion of the transfer and records it on the blockchain.

[0806] Specific examples

[0807] As a concrete example, let's say someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and enters the text information, "I found it on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database.

[0808] Meanwhile, if the person who lost their wallet uses the app to search for "I lost a black leather wallet. It's about 15cm x 10cm," the server uses this information to search its database and finds a match with the registered information of the person who found it. The matching results are notified to both parties, who communicate via chat and then hand over the wallet at a designated location. During this process, the server uses its emotion engine to analyze the user's emotional state and provide appropriate responses and notifications. After the handover is complete, the server records this information on the blockchain to ensure overall transparency and reliability.

[0809] In this way, the present invention is a system that allows users to easily register and search for lost items, performs quick and accurate matching, and ensures overall transparency and reliability by utilizing blockchain technology and an emotion engine.

[0810] Example 2

[0811] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0812] In recent years, there has been a demand for a means to efficiently connect the finder and the person who lost a lost item. However, conventional systems lack transparency and reliability in this matching process, making it difficult to respond in a way that appropriately reflects the user's feelings. In particular, delays and misunderstandings can occur in information sharing and notification, which can detract from the overall user experience. The present invention aims to solve these issues and provide a system that connects users quickly, accurately, and reliably.

[0813] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an information processing device that uploads an image of the lost item, an information processing device that receives the uploaded image and analyzes the object's features using image recognition means, means for saving the analyzed features in a memory area, an information processing device that receives feature information entered by the person who lost the item, analyzes it using natural language processing means, and compares it with the object information in the memory area, means for notifying the information processing device of a matching result based on the compared information, means for recording the matching information in a distributed ledger to ensure transparency and reliability, and means for analyzing the user's emotions using an emotion engine and adjusting the response content. This allows users to quickly and accurately register and search for lost items, ensures information transparency and reliability, and enables appropriate responses according to the user's emotions.

[0814] An "information processing device" is a device that inputs, processes, stores, and outputs data, and includes computers, servers, and the like.

[0815] "Image recognition means" refers to technology that analyzes image data and automatically recognizes objects and features, and specifically refers to computer vision and deep learning technology.

[0816] A "storage area" is an area for storing data and information, and includes databases and cloud storage.

[0817] "Natural language processing means" refers to technology that analyzes text data, understands its content, and extracts necessary information, and includes natural language processing libraries and models.

[0818] "Verifying" refers to the process of comparing input information with existing data to see if it matches.

[0819] "Means of notification" refers to means for conveying specific information to a user, and includes push notifications, emails, etc.

[0820] A "distributed ledger" is a technology that records and manages transaction information in a distributed manner to ensure overall transparency and reliability, and includes blockchain technology.

[0821] An "emotion engine" is a system that analyzes emotions from user input and behavior and generates appropriate responses, and includes emotion analysis libraries and models.

[0822] The present invention is a system for quickly and accurately connecting the finder and the person who lost the lost item, and it uses a combination of image recognition, natural language processing, blockchain, and an emotion engine. The system is configured as follows.

[0823] User Registration

[0824] First, the user launches the application and enters the required information, such as name, contact details, and email address. The device then sends the entered information to the server. The server then stores the received information in a database and sends a notification of registration completion to the device. This process can use an SQL database or cloud storage.

[0825] Registering lost items

[0826] The user takes a photo of the lost item they found and enters text information such as the location, date, time, and characteristics of the item. The device then sends this to the server. The server then uses image recognition to analyze the photo data and extract detailed information about the object (color, shape, size, brand name, etc.). Image recognition libraries such as OpenCV and TensorFlow are used for the analysis. The extracted information is stored in a database along with the text information. A notification that registration is complete is then sent to the device.

[0827] Search for lost items

[0828] To search for a lost item, the user inputs characteristic information as text. The device then sends this to the server. The server then uses natural language processing (e.g., a natural language processing library or generative AI model) to analyze the input text information and compare it with object information in a database. If a matching item is found, the result is notified to the user's device. If not, a "not found" notification is sent.

[0829] Notification of matching results

[0830] The server then sends a notification to both the person who lost the item and the person who found it based on the information matched in the database. The server also records the matching information on a distributed ledger to ensure the transparency and reliability of the information. This process uses blockchain technology.

[0831] Use of emotion engine

[0832] The server is equipped with an emotion engine that recognizes emotions from user input and behavior. An AI model is used to analyze user emotions and reflect them in matching notifications and chat responses. Emotion analysis libraries and models are used for emotion recognition.

[0833] Communication and delivery

[0834] The user receives a notification and uses the app's chat function to contact the finder. After contact and the details of the delivery are decided, the server analyzes the user's emotional state using an emotion engine and adjusts the response accordingly. After the delivery is completed, the user confirms this in the app, and the server records the completion of the delivery again on the distributed ledger.

[0835] Specific examples

[0836] If someone finds a wallet at a station, they open the app, take a photo of the wallet, and enter the text information, "I found this on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database. Meanwhile, if the person who lost the wallet uses the app to search for "I lost my black leather wallet, approximately 15cm x 10cm," the server uses this information to search the database and matches it with the finder's registered information. The matching results are notified to both parties, who communicate via chat and then hand over the wallet at a designated location. During this process, the server uses an emotion engine to analyze the user's emotional state and provide appropriate responses and notifications. After the handover is complete, the server records this information on the blockchain to ensure transparency and reliability.

[0837] Prompt Sentence Examples

[0838] "I found a black leather wallet at the station. It measures 15cm x 10cm."

[0839] "I dropped my black leather wallet at the station. Has anyone found it?"

[0840] In this way, the system of the present invention allows users to easily register and search for lost items, performs quick and accurate matching, and utilizes blockchain technology and an emotion engine to ensure overall transparency and reliability.

[0841] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0842] Step 1: User Registration

[0843] The user launches the app and enters the required information such as name, contact details, and email address. The device assembles this input information into packets, encrypts them using SSL / TLS, and sends them to the server. The server deserializes the received information and saves it in a database (e.g., MySQL). Once saved, the server generates a registration completion status, encrypts it, and sends it back to the device. The device notifies the user that "registration is complete." Input: Name, contact details, email address. Output: Notification of registration completion.

[0844] Step 2: Register your lost property

[0845] The user uses the app to take a photo of the lost item and enters text information such as location, date and time, and characteristics. The device then assembles this into a packet and sends it to the server. The server then analyzes the received photo data using OpenCV to extract detailed information about the object (color, shape, size, brand name, etc.). This extracted information is integrated with the text information and stored in a database. Once registration is complete, the server generates a registration completion status and notifies the device. Input: photo data, location, date and time, characteristics. Output: notification that registration is complete.

[0846] Step 3: Search for lost property

[0847] The user enters the characteristic information of the dropped item as text into the app. The device sends this to the server. The server analyzes the received text information using natural language processing means (e.g., a generative AI model) and generates a search query. The server then uses this query to search a database for matching object information. If a matching item is found, details about it are sent to the user's device; if not, a "not found" message is sent. Input: Characteristic information text. Output: Search results or a "not found" message.

[0848] Step 4: Notification of match results

[0849] The server generates a notification message to both the person who dropped the item and the person who found it based on the matching results in the database. This message is encrypted and sent to the terminal. At the same time, the server records the matching information on the blockchain and generates a hash value for the transaction. Input: Matching result. Output: Notification message, hash value recorded on the blockchain.

[0850] Step 5: Use the Emotion Engine

[0851] The emotion engine built into the server analyzes user behavior and input in real time. It uses an AI model (e.g., emotion analysis library) to assess the user's emotional state and appropriately tailor responses and notifications based on this. Input: User behavior data, input text. Output: Emotion assessment, tailored responses and notifications.

[0852] Step 6: Communication and delivery

[0853] The user receives a matching notification and uses the app's chat function to contact the finder. The server collects the chat content and, if necessary, reanalyzes it using the emotion engine to adjust the response. After communication is established, the users decide on the details of the transfer. Once the transfer is complete, the user confirms it within the app, which the server receives and records on the blockchain. Input: Chat content, transfer information. Output: Emotion analysis results, transfer completion information recorded on the blockchain.

[0854] (Application example 2)

[0855] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0856] There is a need to quickly and accurately connect lost items with their owners, but existing systems do not ensure information transparency or reliability, and are unable to properly recognize and respond to emotions, resulting in a lack of sufficient improvement in the user experience. These issues are particularly likely to become apparent when managing and handing over lost items in physical stores.

[0857] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0858] In this invention, the server includes a user terminal that uploads a photo of the lost item, means for receiving the uploaded photo and analyzing the object's features using image recognition means, means for saving the analyzed features in a database, means for receiving feature information entered by the person who lost the item, analyzing it using natural language processing means, and comparing it with object information in the database, means for notifying the user terminal of a matching result based on the compared information, means for recording the matching information in a blockchain to ensure transparency and reliability, means for analyzing the user's emotional state using an emotion engine that recognizes emotions from the user's input and actions, and means for supporting the registration and search of lost items in physical stores. This enables fast and accurate registration, search, and delivery of lost items in physical stores.

[0859] "User terminal" refers to a device used to upload photos of lost items, including smartphones and smart glasses.

[0860] The "server" is a central system that receives uploaded photos, analyzes them using image recognition and natural language processing tools, and compares them with a database.

[0861] "Image recognition means" refers to technology for analyzing the characteristics of objects in uploaded photos and extracting information about them, and includes AI models and OCR (optical character recognition).

[0862] A "database" is a digital storage system used to store and later collate analyzed feature information and user-entered information.

[0863] "Natural language processing means" is a technology for analyzing text information entered by a user and comparing that information with object information in a database.

[0864] The "matching result" is the result of matching information between the lost item and the owner, which is generated based on the collated information.

[0865] "Blockchain" is a distributed ledger technology used to ensure transparency and reliability of information, with each transaction and record encrypted and connected in a chain.

[0866] An "emotion engine" is a technology for recognizing and analyzing emotions from user input and behavior.

[0867] "Physical store" refers to a sales or service establishment operated in a physical location, including commercial facilities where many lost items may occur.

[0868] "Registration and search support" means providing support to efficiently register and search for lost items in physical stores.

[0869] A specific embodiment of the present invention is described below. The present invention is a system that quickly and accurately connects the person who found a lost item in a physical store with the person who lost it, utilizing AI's image recognition and natural language processing capabilities, as well as blockchain technology and an emotion engine to ensure the transparency and reliability of information.

[0870] User Registration

[0871] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. This information is received by the device and sent to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete.

[0872] Registering lost items

[0873] The user takes a photo of the lost item, enters the necessary text information (e.g., location, date, time, characteristics of the item, etc.), and sends it from the device to the server. The server analyzes the photo data using image recognition to extract detailed information about the item (color, shape, size, brand name, etc.). This extracted information is stored in a database along with the text information. The device is then notified that registration is complete.

[0874] Search for lost items

[0875] To search for a lost item, the user inputs characteristic information as text, which the device then sends to the server. The server then uses natural language processing to analyze the input text and compare it with object information in the database. If a matching item is found, the server notifies the user of the matching result; if not, it sends a "not found" notification.

[0876] Notification of matching results

[0877] The server then sends notifications to both the person who lost the item and the person who found it based on the information matched in the database, and records this matching information on the blockchain to ensure the transparency and reliability of the information.

[0878] Use of emotion engine

[0879] The server is equipped with an emotion engine that recognizes emotions from user input and actions. Matching notifications and chat responses are then reflected in the emotion engine's evaluation.

[0880] Communication and delivery

[0881] The user receives a notification and uses the app's chat function to contact the finder. After contact and confirmation of the details of the return, the emotion engine analyzes the user's emotional state and adjusts the response accordingly. Once the return is complete, the return is confirmed within the app, and the server records the return information on the blockchain.

[0882] Hardware and software used

[0883] Hardware:

[0884] Smartphone: taking photos using the camera function, internet connection

[0885] Smart glasses: Photo and display functions

[0886] Robot: Lost item detection, voice guidance

[0887] software:

[0888] TensorFlow: for image recognition

[0889] pytesseract:OCR (optical character recognition)

[0890] Blockchain Library: Blockchain Technology

[0891] Natural Language Toolkit (NLTK): Natural language parsing and sentiment analysis

[0892] Specific examples

[0893] For example, if a black leather jacket is found in a store lobby, a staff member would take a photo of the black leather jacket using their smartphone and input information by voice or text, such as "Black leather jacket found in the store lobby." The system would then automatically extract detailed information about the item using image recognition and store it in a database. When a customer says something like "I lost my keys" at the cash register or information counter, the staff member would simply use the app to search for it. Matching is performed through image recognition and natural language processing, and the results are notified. Transparency is ensured by recording the information on the blockchain.

[0894] Example prompt:

[0895] "Analyze the image, extract the object features from the captured image, and link them with the text information. This will provide the best matching results based on the feature information and image information entered by the user."

[0896] This will enable quick and accurate registration, search, and delivery of lost items in physical stores.

[0897] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0898] Step 1:

[0899] The user launches the app and enters the required information, such as their name, contact details, and email address.

[0900] Input: Name, contact details, email address

[0901] Operation: The terminal receives the information entered by the user.

[0902] Output: User information entered

[0903] Step 2:

[0904] The terminal transmits the received user information to the server.

[0905] Input: Received user information

[0906] Operation: The terminal processes and sends user information to the server.

[0907] Output: User information sent to the server

[0908] Step 3:

[0909] The server stores the received user information in a database and sends a response indicating that registration is complete to the terminal.

[0910] Input: User information sent to the server

[0911] What happens: The server saves the user information in its database and generates a response confirming registration.

[0912] Output: Registration complete response

[0913] Step 4:

[0914] The user takes a photo of the lost item and enters text information such as the location, date, and characteristics of the item.

[0915] Input: Photo of lost item, text information

[0916] How it works: The user takes a photo on their device and enters the required information.

[0917] Output: Photo data and text information of lost items

[0918] Step 5:

[0919] The device sends photo data and text information about the lost item to the server.

[0920] Input: Photo data and text information of lost items

[0921] How it works: The device processes and sends the photo data and text information to the server.

[0922] Output: Photo data and text information sent to the server

[0923] Step 6:

[0924] The server analyzes the photo data using image recognition tools and extracts detailed information about the object (color, shape, size, brand name, etc.).

[0925] Input: Photo data sent to the server

[0926] How it works: Analyzes photo data using image recognition tools (e.g. TensorFlow) and extracts object features.

[0927] Output: Detailed information about what was parsed

[0928] Step 7:

[0929] The server stores the extracted information together with the text information in a database.

[0930] Input: Detailed information and text of the parsed object

[0931] How it works: Saves text information and extracted object features in a database.

[0932] Output: Information stored in the database

[0933] Step 8:

[0934] To search for a lost item, the user inputs characteristic information in text, which is then sent by the terminal to the server.

[0935] Input: Characteristic information text

[0936] Operation: The user inputs characteristic information, which the device sends to the server.

[0937] Output: Feature information sent to the server

[0938] Step 9:

[0939] The server analyzes the input feature information using natural language processing means and compares it with object information in the database.

[0940] Input: Feature information sent to the server

[0941] How it works: Analyzes the feature information using natural language processing tools (e.g., NLTK) and matches it with object information in a database.

[0942] Output: Search results

[0943] Step 10:

[0944] The server notifies the user of the matching results. If a matching item is found, the server notifies the user, and if not, it sends a "not found" notification.

[0945] Input: Search results

[0946] Operation: The server sends the search results to the user's device.

[0947] Output: Search results notified to the user's device

[0948] Step 11:

[0949] The server records the matching information on the blockchain to ensure transparency and reliability.

[0950] Input: Matching information

[0951] What it does: Records matching information in a blockchain library.

[0952] Output: Information recorded on the blockchain

[0953] Step 12:

[0954] The server is equipped with an emotion engine that recognizes emotions from user input and actions.

[0955] Input: User input and behavior information

[0956] How it works: Recognize and analyze user emotions with the emotion engine.

[0957] Output: Emotional state analysis result

[0958] Step 13:

[0959] The server analyzes the user's emotional state and reflects this in matching notifications and responses in the chat function.

[0960] Input: Emotional state analysis result

[0961] What it does: Adjusts response content based on emotion engine analysis.

[0962] Output: The adjusted response

[0963] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0964] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0965] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0966] [Third embodiment]

[0967] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0968] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0969] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0970] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0971] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0972] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0973] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0974] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0975] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0976] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0977] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0978] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0979] A specific embodiment of the present invention is described below. The present invention is a system that quickly and accurately connects the person who found a lost item with the person who lost it, utilizing the image recognition and natural language processing capabilities of AI, and further using blockchain technology to ensure the transparency and reliability of information.

[0980] User Registration

[0981] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. This information is received by the device and sent to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete.

[0982] Registering lost items

[0983] The user takes a photo of the lost item they found, enters the necessary text information (e.g., location, date, time, characteristics of the item, etc.), and sends it from the device to the server. The server analyzes the photo data using image recognition to extract detailed information about the item (color, shape, size, brand name, etc.). This extracted information is stored in a database along with the text information. The device is then notified that registration is complete.

[0984] Search for lost items

[0985] To search for a lost item, the user inputs characteristic information as text, which the device then sends to the server. The server then uses natural language processing to analyze the input text and compare it with object information in the database. If a matching item is found, the server notifies the user of the matching result; if not, it sends a "not found" notification.

[0986] Notification of matching results

[0987] The server then sends notifications to both the person who lost the item and the person who found it based on the information matched in the database. The server also records this matching information on the blockchain, ensuring transparency and reliability of the information.

[0988] Communication and delivery

[0989] The user receives a notification and communicates with the finder using the chat function within the app. After communicating with the finder, the details of the delivery are decided. After the delivery is completed, it is confirmed within the app, and the server records the delivery completion information again on the blockchain.

[0990] Specific examples

[0991] As a concrete example, let's say someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and enters the text information, "I found it on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database.

[0992] Meanwhile, if the person who lost their wallet uses the app to search for "I lost a black leather wallet. It's about 15cm x 10cm," the server will use this information to search the database and match it with the registered information of the person who found it. The matching results are notified to both parties, who then communicate via chat and hand over the wallet at a designated location. After the handover is complete, the server records this information on the blockchain to ensure overall transparency and reliability.

[0993] In this way, the present invention is a system that allows users to easily register and search for lost items, performs quick and accurate matching, and utilizes blockchain technology to ensure overall transparency and reliability.

[0994] The processing flow will be explained below.

[0995] Handling user registration

[0996] Step 1:

[0997] User: Launches the app and accesses the user registration screen.

[0998] Step 2:

[0999] User: Enter the required information such as name, contact details, and email address, and tap the "Register" button.

[1000] Step 3:

[1001] Terminal: The entered user information is sent to the server in JSON format.

[1002] Step 4:

[1003] Server: Analyzes the received user information and stores it in a database. Generates a response indicating registration completion and sends it to the terminal.

[1004] Step 5:

[1005] Terminal: Receives notification of registration completion and displays it to the user.

[1006] Processing lost and found entries

[1007] Step 1:

[1008] User: Access the app's "Lost Item Registration" screen to register the item they found.

[1009] Step 2:

[1010] User: Take a photo of the lost item, enter the necessary text information (e.g., location, date, time, item characteristics, etc.), and tap the "Register" button.

[1011] Step 3:

[1012] Terminal: Sends photo data and text information to the server.

[1013] Step 4:

[1014] Server: The received photo data is passed to the image recognition module, which analyzes the object's detailed information (color, shape, size, brand name, etc.).

[1015] Step 5:

[1016] Server: Combines the analysis results with the text information and stores them in a database. Sends a notification of registration completion to the device.

[1017] Step 6:

[1018] Terminal: Receives notification of registration completion and displays it to the user.

[1019] Processing lost and found searches

[1020] Step 1:

[1021] User: Access the app's "Lost Item Search" screen and enter the characteristics of the lost item.

[1022] Step 2:

[1023] User: Check the information entered and tap the "Search" button.

[1024] Step 3:

[1025] Terminal: Sends the input characteristic information to the server.

[1026] Step 4:

[1027] Server: Passes the received feature information to the natural language processing module for analysis.

[1028] Step 5:

[1029] Server: Compares the analyzed information with the lost items information in the database.

[1030] Step 6:

[1031] Server: If a matching item is found, it generates a matching result and notifies the user's device. If not found, it notifies the user.

[1032] Step 7:

[1033] Terminal: Receives the matching result notification and displays it to the user.

[1034] Processing of match result notifications

[1035] Step 1:

[1036] Server: Checks the database for a match between the user's dropped item and the item they found.

[1037] Step 2:

[1038] Server: Notifies the users (those who dropped the item and those who picked it up) of the matching results.

[1039] Step 3:

[1040] Server: Records the matching information immutably on the blockchain.

[1041] Contact and delivery processing

[1042] Step 1:

[1043] Users: Receive in-app notifications, check match results, and use the chat feature to contact the finder.

[1044] Step 2:

[1045] Users (those who lost the item and those who found it): Use the chat function to communicate and decide the details of the delivery.

[1046] Step 3:

[1047] User (the person who lost the item): Receives the lost item from the finder at the designated location. The completion of the delivery is confirmed within the app.

[1048] Step 4:

[1049] User (the person who found the item): Confirm in the app that the item was returned safely.

[1050] Step 5:

[1051] Server: Receives information about the completion of the transfer and records it on the blockchain.

[1052] The above is the specific operation of each processing step. The user, device, and server work together to efficiently complete the series of processes of registering, searching, matching, and handing over lost items.

[1053] Example 1

[1054] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1055] In conventional lost property management systems, it was difficult to quickly and accurately connect the person who found the lost item with the person who dropped it, and the transparency and reliability of information were not sufficiently ensured.In addition, the means of communication between users and information management after the handover were insufficient, which reduced the reliability of the entire system.

[1056] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1057] In this invention, the server includes a terminal for users to upload photos of lost items, an information processing device that receives the uploaded photos and analyzes the features of the object using image recognition means, means for saving the analyzed features in a storage device, an information processing device that receives feature information entered by the user who lost the item, analyzes it using natural language processing means, and compares it with the object information in the storage device, means for notifying the user of a matching result based on the generated comparison information, means for recording the matching information in a fixed ledger technology to ensure information transparency and reliability, means for users to receive the notification and communicate using a chat function, and means for recording information after the transfer is completed in the fixed ledger technology again. This allows the person who found the lost item to quickly and accurately connect with the person who dropped it, ensures information transparency and reliability, allows users to efficiently communicate with each other, and enables information management after the transfer is completed.

[1058] A "user terminal" is a device for uploading photos of lost items, and has the function of transmitting the input information to a server.

[1059] An "information processing device" is a device that has the function of analyzing input information using image recognition means and natural language processing means and storing the information in a storage device called a database.

[1060] "Image recognition means" refers to technology or software that analyzes uploaded photographs and identifies the characteristics of objects contained within them.

[1061] "Natural language processing means" refers to technology or software that analyzes text information entered by a user and understands its meaning and intent.

[1062] The "chat function" is a communication function that allows users to exchange messages and information in real time.

[1063] "Fixed ledger technology" is a technology similar to blockchain technology that preserves information once recorded with transparency and reliability, without being tampered with.

[1064] A "memory device" is a database or other storage system for storing analyzed features and other data.

[1065] "Matching results" refer to the server analyzing the characteristic information provided by the user and comparing it with information in the database to identify matching lost item information.

[1066] "Information on completion of delivery" is data indicating that the delivery of the lost item, which was determined through communication between users, has been completed.

[1067] This invention is a system that quickly and accurately connects the person who found a lost item with the person who lost it, utilizing image recognition and natural language processing capabilities, as well as blockchain technology to ensure the transparency and reliability of information.

[1068] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. The device receives this information and sends it to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete. The server uses an SQL database such as MySQL or PostgreSQL.

[1069] Next, the user takes a photo of the lost item and sends the necessary text information (e.g., location, date, time, and characteristics of the item) from the device to the server. The server analyzes the received photo using TensorFlow and OpenCV to extract detailed information about the item (color, shape, size, brand name, etc.). These details are stored in a database along with the text information, and the device is notified that registration is complete.

[1070] To search for a lost item, the user inputs feature information as text, which the device then sends to the server. The server then analyzes the input text using a natural language processing model such as BERT and compares it with object information in a database. If a matching item is found, the server notifies the user's device of the matching result. If the item is not found, the server sends a "not found" notification.

[1071] The server then sends notifications to both the person who lost the item and the person who found it based on the information collated in the database.The server also records this matching information on a blockchain such as Ethereum, ensuring the transparency and reliability of the information.

[1072] The user receives a notification and uses the chat function within the app (for example, Firebase chat service) to communicate with the finder. After communicating, they decide the details of the delivery. After the delivery is completed, this is confirmed within the app, and the server records the delivery completion information again on the blockchain.

[1073] As a concrete example, let's say someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and enters the text information, "I found it on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database.

[1074] Meanwhile, if the person who lost their wallet uses the app to search for "I lost a black leather wallet. It's about 15cm x 10cm," the server will use this information to search the database and match it with the registered information of the person who found it. The matching results are notified to both parties, who then communicate via chat and hand over the wallet at a designated location. After the handover is complete, the server records this information on the blockchain, ensuring overall transparency and reliability.

[1075] In this way, users can easily register and search for lost items, and matching can be done quickly and accurately, while blockchain technology is used to ensure overall transparency and reliability.

[1076] Example prompt sentence:

[1077] "I found a black leather wallet on a bench at the station. It measures 15cm x 10cm. How can I find the owner of this wallet?"

[1078] The above is an embodiment of the present invention.

[1079] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1080] Step 1: User Registration

[1081] The user uses the device and launches the app. The user enters the required information, such as their name, contact details, and email address. The device receives this information and sends it to the server. The server analyzes the received information and stores it in a database. It then generates a response indicating registration is complete and sends it to the device.

[1082] Input: Name, contact details, and email address entered by the user.

[1083] Data processing: Sending data from the device to the server.

[1084] Data calculation: Data analysis and database storage on the server.

[1085] Output: Registration complete response.

[1086] What happens: When a user clicks the "Register" button, the information is sent to the server and confirmed to be saved in the database. A notification of successful registration is displayed on the device.

[1087] Step 2: Register your lost property

[1088] The user uses their device to take a photo of the lost item and uploads it to the app. The user then enters text information such as the location, date, and item characteristics, and sends it from the device to the server. The server then analyzes the received photo data using image recognition techniques (e.g., TensorFlow or OpenCV) to extract detailed information about the item (color, shape, size, brand name, etc.). The extracted information and text information are then stored in a database, and the user is notified that registration is complete.

[1089] Input: A photo taken by the user and text information.

[1090] Data processing: Sending photos and text information from the device to the server.

[1091] Data calculation: Analyzing photo data and extracting detailed information using image recognition methods.

[1092] Output: Notification that registration is complete.

[1093] Specific operation: When a user presses the "Register" button, the photo and text information are sent to the server, where image analysis is performed. The results are stored in a database, and a notification of registration completion is displayed on the device.

[1094] Step 3: Search for lost property

[1095] The user uses their device to input textual information about the features of the dropped item, which is then sent from the device to the server. The server then uses natural language processing (e.g., the BERT model) to analyze the input feature information and compare it with object information in the database. If a matching item is found, a matching result is generated and notified to the user's device.

[1096] Input: Feature information entered by the user.

[1097] Data processing: Sending characteristic information from the device to the server.

[1098] Data Computing: Analysis of feature information and database search using natural language processing tools.

[1099] Output: Matching result or "not found" notification.

[1100] Specific operation: When the user presses the "Search" button, the characteristic information is analyzed and compared with the information in the database. The results are displayed on the device.

[1101] Step 4: Notification of match results

[1102] The server then sends notifications to both the person who lost the item and the person who found it based on the matched information in the database, and records this matching information on the blockchain to ensure the transparency and reliability of the information.

[1103] Input: The matched information.

[1104] Data processing: Creating notifications of matching results.

[1105] Data calculation: Blockchain recording of matching information.

[1106] Output: Notification of matching results and recording to the blockchain.

[1107] Specific operation: The server checks the matching information, generates a notification and sends it to the user. The result is displayed on the device and simultaneously recorded on the blockchain.

[1108] Step 5: Communication and delivery

[1109] The user receives a notification and uses the chat function within the app (for example, Firebase chat service) to communicate with the finder. After communicating, they decide the details of the delivery. After the delivery is completed, this is confirmed within the app, and the server records the delivery completion information again on the blockchain.

[1110] Input: Completed handover information.

[1111] Data processing: Communication via chat and confirmation of completion of delivery.

[1112] Data calculation: Blockchain recording of delivery completion information.

[1113] Output: A confirmation that the transaction has been completed and recorded on the blockchain.

[1114] How it works: Users communicate and decide on the details of the transaction. Once the transaction is complete, the information is recorded on the blockchain.

[1115] The above is a detailed description of each processing step of the system.

[1116] (Application example 1)

[1117] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1118] In modern factory environments, lost parts and tools have a significant impact on production efficiency. Work is often delayed when workers are unable to find the parts they need. Managing and locating lost items requires a great deal of time and effort, creating a need for a system that can quickly and efficiently resolve this issue. Furthermore, measures to ensure transparency and reliability are insufficient, and this also requires improvement.

[1119] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1120] In this invention, the server includes a means for preventing and quickly locating parts and tools lost in a factory environment, a means for searching for parts information by voice input or text input and analyzing it with natural language processing means, and a means for notifying the owner of the lost item with an internal notification function. This makes it possible to quickly and efficiently locate lost parts and tools in the factory, ensuring transparency and reliability.

[1121] Key Word Definitions

[1122] A "user terminal" is a computing device used by a user, such as a smartphone or tablet.

[1123] A "server" is a computer system that communicates with user terminals over a network and processes and stores data.

[1124] "Image recognition means" is a technology for detecting objects from photographs and videos taken with a camera and analyzing their features.

[1125] A "database" is a system for efficiently storing, retrieving, and managing information.

[1126] "Natural language processing means" refers to technology that enables computers to understand, interpret, and generate human language.

[1127] "Blockchain" is a distributed digital ledger and technology that increases the transparency and reliability of information.

[1128] An "internal notification function" is a system that sends messages and alerts to specific users or devices.

[1129] "Factory environment" means the physical and working environment within a factory where products are manufactured.

[1130] "Voice input" is a method of inputting voice as data through a microphone.

[1131] "Text entry" is a method of entering characters and symbols as data using a keyboard or software.

[1132] MODE FOR CARRYING OUT THE INVENTION

[1133] The system for implementing this invention aims to efficiently prevent and quickly locate parts and tools lost in a factory environment. The technical elements that make up this system and their specific processing will be described below.

[1134] System Components

[1135] User terminal

[1136] User devices can be smartphones, tablets, or factory robots. These devices are used to take photos of parts and tools, and provide an interface for registering information, searching, and receiving notifications. The main software components are a mobile app and a client application for data transfer.

[1137] server

[1138] The server is the central component that communicates with user devices over the network, processes uploaded photos and text information, manages the database, notifies matching results, and records information on the blockchain. It includes the following main software and hardware components:

[1139] Image recognition: Uses image processing libraries such as TensorFlow and OpenCV to extract object features from uploaded photos.

[1140] Database: A relational database management system such as MySQL or PostgreSQL is used to store the analyzed feature information.

[1141] Natural language processing tools: NLP libraries such as BERT and spaCy are used to analyze the text information entered by the user and match it with object information in the database.

[1142] Notifications: Use notification services such as Firebase Cloud Messaging to notify users of match results and other important information.

[1143] Blockchain: Using Ethereum, Hyperledger, etc., matching information and delivery completion information are recorded on the blockchain to ensure transparency and reliability.

[1144] Usage and Operation Procedures

[1145] First, the user registers with the system using their user terminal, and the necessary information is stored in a database. Next, when the user registers information about a lost or found item, they take a photo using their terminal and send the photo along with detailed text information to the server. The server receives this data and analyzes it using image recognition and natural language processing. The analyzed data is then stored in a database and, if necessary, notifies the user of the matching results.

[1146] After the delivery is complete, the delivery details are also recorded on the blockchain again, ensuring overall transparency and reliability. This prevents loss of production efficiency due to lost parts or tools in the factory and ensures that needed items can be found quickly.

[1147] Specific examples

[1148] For example, if a "blue drill bit (50mm)" is lost in a factory, a worker launches the smartphone app and searches for "lost drill bit." They voice-type "blue drill bit, 50mm" into the app, and the AI ​​converts the voice to text, then searches the database for a matching part. The matching part information is notified, and the blue drill bit picked up by another worker is found. They communicate using the built-in chat function to complete the part transfer. The transfer completion information is then recorded on the blockchain.

[1149] Prompt sentence for generative AI model

[1150] User Input: "Blue drill bit, 50mm"

[1151] AI process: Converts voice input into text and searches a database for matching parts.

[1152] Result: Notify the customer of the matching part information and communicate with them via the internal chat function.

[1153] The above is a specific embodiment for implementing this system, which makes it possible to efficiently manage parts and tools in a factory environment and to manage information in a transparent and reliable manner.

[1154] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1155] System processing steps

[1156] Processing flow

[1157] Step 1:

[1158] A user launches the app on a user device such as a smartphone or tablet and enters their own information (name, contact details, email address, etc.). This input information is sent from the user device to the server. The server receives the sent information and stores it in a database. This process registers the user's basic information within the system.

[1159] Input: Name, contact information, email address

[1160] Output: User information stored in the database

[1161] Step 2:

[1162] The user takes a photo of the part or tool they have picked up and enters the necessary text information (e.g., location, date and time, and object characteristics) into their user terminal. The user terminal then sends this information to the server. The server receives the photo data and text information, analyzes the photo data using image recognition tools, and extracts the object's characteristics (color, shape, size, etc.). The extracted information and the entered text information are stored in a database.

[1163] Input: Photo of the object, location, date and time, object characteristics

[1164] Output: Object features and text information stored in a database

[1165] Step 3:

[1166] When a user drops an object, they input feature information into their user terminal or transmit it via voice input. The user terminal then transmits the input feature information to the server. The server uses natural language processing to analyze the text and voice data and compares it with object information in a database. If the server finds matching object information, it notifies the user terminal of the details.

[1167] Input: Feature information (text or audio)

[1168] Output: Matching result notification

[1169] Step 4:

[1170] If the server finds a match, it records the information on the blockchain to ensure transparency and reliability. Information is securely recorded and cannot be tampered with.

[1171] Input: Matching results

[1172] Output: Information recorded on the blockchain

[1173] Step 5:

[1174] Users who receive a matching notification can use the app's chat function to confirm information about the items they found and lost, and to communicate details about the delivery. The server also manages chat data in real time.

[1175] Input: Chat message

[1176] Output: Send a message to the other user

[1177] Step 6:

[1178] After the item has been delivered, the user confirms the delivery in the app, and the server records this information back on the blockchain, ensuring transparency and authenticity of the transaction completion.

[1179] Input: Delivery completion information

[1180] Output: Delivery completion information recorded on the blockchain

[1181] Specific examples of processing

[1182] Step 1: User Registration

[1183] The user's device transmits the name, contact information, and email address entered in the app, which is then received by the server and stored in a MySQL database.

[1184] Step 2: Register your lost property

[1185] The user's device sends a photo of the object and text information to the server, which then uses TensorFlow to analyze the photo, extract the object's features, and store them in a database.

[1186] Step 3: Find your lost item

[1187] The user inputs the characteristics of the lost item via text or voice, and the user device sends this to the server. The server analyzes it using BERT, compares it with the database, and notifies the user device of any matches.

[1188] Step 4: Record your match information

[1189] The server records the matching results on the blockchain using Ethereum, which ensures the transparency and immutability of the information.

[1190] Step 5: Chat with us

[1191] Users who receive a match notification can contact each other using the app's chat function. The server sends and receives messages in real time using WebSocket.

[1192] Step 6: Record the completion of the handover

[1193] Once the transfer is complete, the user device sends the completion information to the server, which then records this information on the blockchain again to ensure the transparency of the transfer details.

[1194] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1195] A specific embodiment of the present invention is described below. The present invention is a system that quickly and accurately connects the person who found a lost item with the person who lost it, utilizing the image recognition and natural language processing capabilities of AI, as well as blockchain technology and an emotion engine to ensure the transparency and reliability of information.

[1196] User Registration

[1197] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. This information is received by the device and sent to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete.

[1198] Registering lost items

[1199] The user takes a photo of the lost item they found, enters the necessary text information (e.g., location, date, time, characteristics of the item, etc.), and sends it from the device to the server. The server analyzes the photo data using image recognition to extract detailed information about the item (color, shape, size, brand name, etc.). This extracted information is stored in a database along with the text information. The device is then notified that registration is complete.

[1200] Search for lost items

[1201] To search for a lost item, the user inputs characteristic information as text, which the device then sends to the server. The server then uses natural language processing to analyze the input text and compare it with object information in the database. If a matching item is found, the server notifies the user of the matching result; if not, it sends a "not found" notification.

[1202] Notification of matching results

[1203] The server then sends notifications to both the person who lost the item and the person who found it based on the information matched in the database. The server also records this matching information on the blockchain, ensuring transparency and reliability of the information.

[1204] Use of emotion engine

[1205] The server is equipped with an emotion engine that recognizes emotions from user input and actions. Matching notifications and chat responses are then reflected in the emotion engine's evaluation.

[1206] Communication and delivery

[1207] The user receives a notification and uses the app's chat function to contact the finder. After contact and confirmation of the details of the return, the emotion engine analyzes the user's emotional state and adjusts the response accordingly. Once the return is complete, the return is confirmed within the app, and the server records the return information on the blockchain.

[1208] Specific examples

[1209] As a concrete example, suppose someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and registers it by entering the text "I found this on a bench at the station." The server analyzes the photo, extracts the information "black leather wallet, approximately 15cm x 10cm," and stores it in the database. Meanwhile, if the person who lost the wallet uses the app to search "I lost my black leather wallet, approximately 15cm x 10cm," the server uses this information to search the database and matches it with the registered information of the person who found it. The matching results are notified to both parties, who communicate via chat and then hand over the wallet at a designated location. During this process, the server uses its emotion engine to analyze the user's emotional state and provide appropriate responses and notifications. After the handover is complete, the server records this information on the blockchain to ensure overall transparency and reliability.

[1210] In this way, the present invention is a system that allows users to easily register and search for lost items, performs quick and accurate matching, and ensures overall transparency and reliability by utilizing blockchain technology and an emotion engine.

[1211] The processing flow will be explained below.

[1212] Handling user registration

[1213] Step 1:

[1214] User: Launches the app and accesses the user registration screen.

[1215] Step 2:

[1216] User: Enter the required information such as name, contact details, and email address, and tap the "Register" button.

[1217] Step 3:

[1218] Terminal: The entered user information is sent to the server in JSON format.

[1219] Step 4:

[1220] Server: Analyzes the received user information and stores it in a database. Generates a response indicating registration completion and sends it to the terminal.

[1221] Step 5:

[1222] Terminal: Receives notification of registration completion and displays it to the user.

[1223] Processing lost and found entries

[1224] Step 1:

[1225] User: Access the app's "Lost Item Registration" screen to register the item they found.

[1226] Step 2:

[1227] User: Take a photo of the lost item, enter the necessary text information (e.g., location, date, time, item characteristics, etc.), and tap the "Register" button.

[1228] Step 3:

[1229] Terminal: Sends photo data and text information to the server.

[1230] Step 4:

[1231] Server: The received photo data is passed to the image recognition module, which analyzes the object's detailed information (color, shape, size, brand name, etc.).

[1232] Step 5:

[1233] Server: Combines the analysis results with the text information and stores them in a database. Sends a notification of registration completion to the device.

[1234] Step 6:

[1235] Terminal: Receives notification of registration completion and displays it to the user.

[1236] Processing lost and found searches

[1237] Step 1:

[1238] User: Access the app's "Lost Item Search" screen and enter the characteristics of the lost item.

[1239] Step 2:

[1240] User: Check the information entered and tap the "Search" button.

[1241] Step 3:

[1242] Terminal: Sends the input characteristic information to the server.

[1243] Step 4:

[1244] Server: Passes the received feature information to a natural language processing (NLP) module for analysis.

[1245] Step 5:

[1246] Server: Compares the analyzed information with the lost items information in the database.

[1247] Step 6:

[1248] Server: If a matching item is found, it generates a matching result and notifies the user device. If no matching item is found, it notifies the user device.

[1249] Step 7:

[1250] Terminal: Receives the matching result notification and displays it to the user.

[1251] Processing of match result notifications

[1252] Step 1:

[1253] Server: Checks the database for a match between the item the user dropped and the item they picked up.

[1254] Step 2:

[1255] Server: Notifies the users (those who dropped the item and those who picked it up) of the matching results.

[1256] Step 3:

[1257] Server: Records matching information on the blockchain to ensure transparency and reliability.

[1258] Use of emotion engine

[1259] Step 1:

[1260] Server: The emotion engine analyzes the text and behavioral data entered by the user and recognizes the user's emotions.

[1261] Step 2:

[1262] Server: Runs a program that adjusts notification content and chat responses based on the user's emotions.

[1263] Step 3:

[1264] Server: Sends emotion-based adjustment results to the user's device.

[1265] Step 4:

[1266] Terminal: Displaying tailored notifications and responses to the user.

[1267] Contact and delivery processing

[1268] Step 1:

[1269] Users: Receive in-app notifications, check match results, and use the chat feature to contact the finder.

[1270] Step 2:

[1271] Users (those who lost the item and those who found it): Use the chat function to communicate and decide the details of the delivery.

[1272] Step 3:

[1273] User (the person who lost the item): Receives the lost item from the finder at the designated location. The completion of the delivery is confirmed within the app.

[1274] Step 4:

[1275] User (the person who found the item): Confirm in the app that the item was returned safely.

[1276] Step 5:

[1277] Server: Receives information about the completion of the transfer and records it on the blockchain.

[1278] Specific examples

[1279] As a concrete example, let's say someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and enters the text information, "I found it on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database.

[1280] Meanwhile, if the person who lost their wallet uses the app to search for "I lost a black leather wallet. It's about 15cm x 10cm," the server uses this information to search its database and finds a match with the registered information of the person who found it. The matching results are notified to both parties, who communicate via chat and then hand over the wallet at a designated location. During this process, the server uses its emotion engine to analyze the user's emotional state and provide appropriate responses and notifications. After the handover is complete, the server records this information on the blockchain to ensure overall transparency and reliability.

[1281] In this way, the present invention is a system that allows users to easily register and search for lost items, performs quick and accurate matching, and ensures overall transparency and reliability by utilizing blockchain technology and an emotion engine.

[1282] Example 2

[1283] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1284] In recent years, there has been a demand for a means to efficiently connect the finder and the person who lost a lost item. However, conventional systems lack transparency and reliability in this matching process, making it difficult to respond in a way that appropriately reflects the user's feelings. In particular, delays and misunderstandings can occur in information sharing and notification, which can detract from the overall user experience. The present invention aims to solve these issues and provide a system that connects users quickly, accurately, and reliably.

[1285] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an information processing device that uploads an image of the lost item, an information processing device that receives the uploaded image and analyzes the object's features using image recognition means, means for saving the analyzed features in a memory area, an information processing device that receives feature information entered by the person who lost the item, analyzes it using natural language processing means, and compares it with the object information in the memory area, means for notifying the information processing device of a matching result based on the compared information, means for recording the matching information in a distributed ledger to ensure transparency and reliability, and means for analyzing the user's emotions using an emotion engine and adjusting the response content. This allows users to quickly and accurately register and search for lost items, ensures information transparency and reliability, and enables appropriate responses according to the user's emotions.

[1286] An "information processing device" is a device that inputs, processes, stores, and outputs data, and includes computers, servers, and the like.

[1287] "Image recognition means" refers to technology that analyzes image data and automatically recognizes objects and features, and specifically refers to computer vision and deep learning technology.

[1288] A "storage area" is an area for storing data and information, and includes databases and cloud storage.

[1289] "Natural language processing means" refers to technology that analyzes text data, understands its content, and extracts necessary information, and includes natural language processing libraries and models.

[1290] "Verifying" refers to the process of comparing input information with existing data to see if it matches.

[1291] "Means of notification" refers to means for conveying specific information to a user, and includes push notifications, emails, etc.

[1292] A "distributed ledger" is a technology that records and manages transaction information in a distributed manner to ensure overall transparency and reliability, and includes blockchain technology.

[1293] An "emotion engine" is a system that analyzes emotions from user input and behavior and generates appropriate responses, and includes emotion analysis libraries and models.

[1294] The present invention is a system for quickly and accurately connecting the finder and the person who lost the lost item, and it uses a combination of image recognition, natural language processing, blockchain, and an emotion engine. The system is configured as follows.

[1295] User Registration

[1296] First, the user launches the application and enters the required information, such as name, contact details, and email address. The device then sends the entered information to the server. The server then stores the received information in a database and sends a notification of registration completion to the device. This process can use an SQL database or cloud storage.

[1297] Registering lost items

[1298] The user takes a photo of the lost item they found and enters text information such as the location, date, time, and characteristics of the item. The device then sends this to the server. The server then uses image recognition to analyze the photo data and extract detailed information about the object (color, shape, size, brand name, etc.). Image recognition libraries such as OpenCV and TensorFlow are used for the analysis. The extracted information is stored in a database along with the text information. A notification that registration is complete is then sent to the device.

[1299] Search for lost items

[1300] To search for a lost item, the user inputs characteristic information as text. The device then sends this to the server. The server then uses natural language processing (e.g., a natural language processing library or generative AI model) to analyze the input text information and compare it with object information in a database. If a matching item is found, the result is notified to the user's device. If not, a "not found" notification is sent.

[1301] Notification of matching results

[1302] The server then sends a notification to both the person who lost the item and the person who found it based on the information matched in the database. The server also records the matching information on a distributed ledger to ensure the transparency and reliability of the information. This process uses blockchain technology.

[1303] Use of emotion engine

[1304] The server is equipped with an emotion engine that recognizes emotions from user input and behavior. An AI model is used to analyze user emotions and reflect them in matching notifications and chat responses. Emotion analysis libraries and models are used for emotion recognition.

[1305] Communication and delivery

[1306] The user receives a notification and uses the app's chat function to contact the finder. After contact and the details of the delivery are decided, the server analyzes the user's emotional state using an emotion engine and adjusts the response accordingly. After the delivery is completed, the user confirms this in the app, and the server records the completion of the delivery again on the distributed ledger.

[1307] Specific examples

[1308] If someone finds a wallet at a station, they open the app, take a photo of the wallet, and enter the text information, "I found this on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database. Meanwhile, if the person who lost the wallet uses the app to search for "I lost my black leather wallet, approximately 15cm x 10cm," the server uses this information to search the database and matches it with the finder's registered information. The matching results are notified to both parties, who communicate via chat and then hand over the wallet at a designated location. During this process, the server uses an emotion engine to analyze the user's emotional state and provide appropriate responses and notifications. After the handover is complete, the server records this information on the blockchain to ensure transparency and reliability.

[1309] Prompt Sentence Examples

[1310] "I found a black leather wallet at the station. It measures 15cm x 10cm."

[1311] "I dropped my black leather wallet at the station. Has anyone found it?"

[1312] In this way, the system of the present invention allows users to easily register and search for lost items, performs quick and accurate matching, and utilizes blockchain technology and an emotion engine to ensure overall transparency and reliability.

[1313] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1314] Step 1: User Registration

[1315] The user launches the app and enters the required information such as name, contact details, and email address. The device assembles this input information into packets, encrypts them using SSL / TLS, and sends them to the server. The server deserializes the received information and saves it in a database (e.g., MySQL). Once saved, the server generates a registration completion status, encrypts it, and sends it back to the device. The device notifies the user that "registration is complete." Input: Name, contact details, email address. Output: Notification of registration completion.

[1316] Step 2: Register your lost property

[1317] The user uses the app to take a photo of the lost item and enters text information such as location, date and time, and characteristics. The device then assembles this into a packet and sends it to the server. The server then analyzes the received photo data using OpenCV to extract detailed information about the object (color, shape, size, brand name, etc.). This extracted information is integrated with the text information and stored in a database. Once registration is complete, the server generates a registration completion status and notifies the device. Input: photo data, location, date and time, characteristics. Output: notification that registration is complete.

[1318] Step 3: Search for lost property

[1319] The user enters the characteristic information of the dropped item as text into the app. The device sends this to the server. The server analyzes the received text information using natural language processing means (e.g., a generative AI model) and generates a search query. The server then uses this query to search a database for matching object information. If a matching item is found, details about it are sent to the user's device; if not, a "not found" message is sent. Input: Characteristic information text. Output: Search results or a "not found" message.

[1320] Step 4: Notification of match results

[1321] The server generates a notification message to both the person who dropped the item and the person who found it based on the matching results in the database. This message is encrypted and sent to the terminal. At the same time, the server records the matching information on the blockchain and generates a hash value for the transaction. Input: Matching result. Output: Notification message, hash value recorded on the blockchain.

[1322] Step 5: Use the Emotion Engine

[1323] The emotion engine built into the server analyzes user behavior and input in real time. It uses an AI model (e.g., emotion analysis library) to assess the user's emotional state and appropriately tailor responses and notifications based on this. Input: User behavior data, input text. Output: Emotion assessment, tailored responses and notifications.

[1324] Step 6: Communication and delivery

[1325] The user receives a matching notification and uses the app's chat function to contact the finder. The server collects the chat content and, if necessary, reanalyzes it using the emotion engine to adjust the response. After communication is established, the users decide on the details of the transfer. Once the transfer is complete, the user confirms it within the app, which the server receives and records on the blockchain. Input: Chat content, transfer information. Output: Emotion analysis results, transfer completion information recorded on the blockchain.

[1326] (Application example 2)

[1327] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1328] There is a need to quickly and accurately connect lost items with their owners, but existing systems do not ensure information transparency or reliability, and are unable to properly recognize and respond to emotions, resulting in a lack of sufficient improvement in the user experience. These issues are particularly likely to become apparent when managing and handing over lost items in physical stores.

[1329] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1330] In this invention, the server includes a user terminal that uploads a photo of the lost item, means for receiving the uploaded photo and analyzing the object's features using image recognition means, means for saving the analyzed features in a database, means for receiving feature information entered by the person who lost the item, analyzing it using natural language processing means, and comparing it with object information in the database, means for notifying the user terminal of a matching result based on the compared information, means for recording the matching information in a blockchain to ensure transparency and reliability, means for analyzing the user's emotional state using an emotion engine that recognizes emotions from the user's input and actions, and means for supporting the registration and search of lost items in physical stores. This enables fast and accurate registration, search, and delivery of lost items in physical stores.

[1331] "User terminal" refers to a device used to upload photos of lost items, including smartphones and smart glasses.

[1332] The "server" is a central system that receives uploaded photos, analyzes them using image recognition and natural language processing tools, and compares them with a database.

[1333] "Image recognition means" refers to technology for analyzing the characteristics of objects in uploaded photos and extracting information about them, and includes AI models and OCR (optical character recognition).

[1334] A "database" is a digital storage system used to store and later collate analyzed feature information and user-entered information.

[1335] "Natural language processing means" is a technology for analyzing text information entered by a user and comparing that information with object information in a database.

[1336] The "matching result" is the result of matching information between the lost item and the owner, which is generated based on the collated information.

[1337] "Blockchain" is a distributed ledger technology used to ensure transparency and reliability of information, with each transaction and record encrypted and connected in a chain.

[1338] An "emotion engine" is a technology for recognizing and analyzing emotions from user input and behavior.

[1339] "Physical store" refers to a sales or service establishment operated in a physical location, including commercial facilities where many lost items may occur.

[1340] "Registration and search support" means providing support to efficiently register and search for lost items in physical stores.

[1341] A specific embodiment of the present invention is described below. The present invention is a system that quickly and accurately connects the person who found a lost item in a physical store with the person who lost it, utilizing AI's image recognition and natural language processing capabilities, as well as blockchain technology and an emotion engine to ensure the transparency and reliability of information.

[1342] User Registration

[1343] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. This information is received by the device and sent to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete.

[1344] Registering lost items

[1345] The user takes a photo of the lost item, enters the necessary text information (e.g., location, date, time, characteristics of the item, etc.), and sends it from the device to the server. The server analyzes the photo data using image recognition to extract detailed information about the item (color, shape, size, brand name, etc.). This extracted information is stored in a database along with the text information. The device is then notified that registration is complete.

[1346] Search for lost items

[1347] To search for a lost item, the user inputs characteristic information as text, which the device then sends to the server. The server then uses natural language processing to analyze the input text and compare it with object information in the database. If a matching item is found, the server notifies the user of the matching result; if not, it sends a "not found" notification.

[1348] Notification of matching results

[1349] The server then sends notifications to both the person who lost the item and the person who found it based on the information matched in the database, and records this matching information on the blockchain to ensure the transparency and reliability of the information.

[1350] Use of emotion engine

[1351] The server is equipped with an emotion engine that recognizes emotions from user input and actions. Matching notifications and chat responses are then reflected in the emotion engine's evaluation.

[1352] Communication and delivery

[1353] The user receives a notification and uses the app's chat function to contact the finder. After contact and confirmation of the details of the return, the emotion engine analyzes the user's emotional state and adjusts the response accordingly. Once the return is complete, the return is confirmed within the app, and the server records the return information on the blockchain.

[1354] Hardware and software used

[1355] Hardware:

[1356] Smartphone: taking photos using the camera function, internet connection

[1357] Smart glasses: Photo and display functions

[1358] Robot: Lost item detection, voice guidance

[1359] software:

[1360] TensorFlow: for image recognition

[1361] pytesseract:OCR (optical character recognition)

[1362] Blockchain Library: Blockchain Technology

[1363] Natural Language Toolkit (NLTK): Natural language parsing and sentiment analysis

[1364] Specific examples

[1365] For example, if a black leather jacket is found in a store lobby, a staff member would take a photo of the black leather jacket using their smartphone and input information by voice or text, such as "Black leather jacket found in the store lobby." The system would then automatically extract detailed information about the item using image recognition and store it in a database. When a customer says something like "I lost my keys" at the cash register or information counter, the staff member would simply use the app to search for it. Matching is performed through image recognition and natural language processing, and the results are notified. Transparency is ensured by recording the information on the blockchain.

[1366] Example prompt:

[1367] "Analyze the image, extract the object features from the captured image, and link them with the text information. This will provide the best matching results based on the feature information and image information entered by the user."

[1368] This will enable quick and accurate registration, search, and delivery of lost items in physical stores.

[1369] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1370] Step 1:

[1371] The user launches the app and enters the required information, such as their name, contact details, and email address.

[1372] Input: Name, contact details, email address

[1373] Operation: The terminal receives the information entered by the user.

[1374] Output: User information entered

[1375] Step 2:

[1376] The terminal transmits the received user information to the server.

[1377] Input: Received user information

[1378] Operation: The terminal processes and sends user information to the server.

[1379] Output: User information sent to the server

[1380] Step 3:

[1381] The server stores the received user information in a database and sends a response indicating that registration is complete to the terminal.

[1382] Input: User information sent to the server

[1383] What happens: The server saves the user information in its database and generates a response confirming registration.

[1384] Output: Registration complete response

[1385] Step 4:

[1386] The user takes a photo of the lost item and enters text information such as the location, date, and characteristics of the item.

[1387] Input: Photo of lost item, text information

[1388] How it works: The user takes a photo on their device and enters the required information.

[1389] Output: Photo data and text information of lost items

[1390] Step 5:

[1391] The device sends photo data and text information about the lost item to the server.

[1392] Input: Photo data and text information of lost items

[1393] How it works: The device processes and sends the photo data and text information to the server.

[1394] Output: Photo data and text information sent to the server

[1395] Step 6:

[1396] The server analyzes the photo data using image recognition tools and extracts detailed information about the object (color, shape, size, brand name, etc.).

[1397] Input: Photo data sent to the server

[1398] How it works: Analyzes photo data using image recognition tools (e.g. TensorFlow) and extracts object features.

[1399] Output: Detailed information about what was parsed

[1400] Step 7:

[1401] The server stores the extracted information together with the text information in a database.

[1402] Input: Detailed information and text of the parsed object

[1403] How it works: Saves text information and extracted object features in a database.

[1404] Output: Information stored in the database

[1405] Step 8:

[1406] To search for a lost item, the user inputs characteristic information in text, which is then sent by the terminal to the server.

[1407] Input: Characteristic information text

[1408] Operation: The user inputs characteristic information, which the device sends to the server.

[1409] Output: Feature information sent to the server

[1410] Step 9:

[1411] The server analyzes the input feature information using natural language processing means and compares it with object information in the database.

[1412] Input: Feature information sent to the server

[1413] How it works: Analyzes the feature information using natural language processing tools (e.g., NLTK) and matches it with object information in a database.

[1414] Output: Search results

[1415] Step 10:

[1416] The server notifies the user of the matching results. If a matching item is found, the server notifies the user, and if not, it sends a "not found" notification.

[1417] Input: Search results

[1418] Operation: The server sends the search results to the user's device.

[1419] Output: Search results notified to the user's device

[1420] Step 11:

[1421] The server records the matching information on the blockchain to ensure transparency and reliability.

[1422] Input: Matching information

[1423] What it does: Records matching information in a blockchain library.

[1424] Output: Information recorded on the blockchain

[1425] Step 12:

[1426] The server is equipped with an emotion engine that recognizes emotions from user input and actions.

[1427] Input: User input and behavior information

[1428] How it works: Recognize and analyze user emotions with the emotion engine.

[1429] Output: Emotional state analysis result

[1430] Step 13:

[1431] The server analyzes the user's emotional state and reflects this in matching notifications and responses in the chat function.

[1432] Input: Emotional state analysis result

[1433] What it does: Adjusts response content based on emotion engine analysis.

[1434] Output: The adjusted response

[1435] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1436] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1437] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1438] [Fourth embodiment]

[1439] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1440] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1441] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1442] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1443] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1444] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1445] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1446] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1447] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1448] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1449] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1450] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1451] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1452] A specific embodiment of the present invention is described below. The present invention is a system that quickly and accurately connects the person who found a lost item with the person who lost it, utilizing the image recognition and natural language processing capabilities of AI, and further using blockchain technology to ensure the transparency and reliability of information.

[1453] User Registration

[1454] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. This information is received by the device and sent to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete.

[1455] Registering lost items

[1456] The user takes a photo of the lost item they found, enters the necessary text information (e.g., location, date, time, characteristics of the item, etc.), and sends it from the device to the server. The server analyzes the photo data using image recognition to extract detailed information about the item (color, shape, size, brand name, etc.). This extracted information is stored in a database along with the text information. The device is then notified that registration is complete.

[1457] Search for lost items

[1458] To search for a lost item, the user inputs characteristic information as text, which the device then sends to the server. The server then uses natural language processing to analyze the input text and compare it with object information in the database. If a matching item is found, the server notifies the user of the matching result; if not, it sends a "not found" notification.

[1459] Notification of matching results

[1460] The server then sends notifications to both the person who lost the item and the person who found it based on the information matched in the database. The server also records this matching information on the blockchain, ensuring transparency and reliability of the information.

[1461] Communication and delivery

[1462] The user receives a notification and communicates with the finder using the chat function within the app. After communicating with the finder, the details of the delivery are decided. After the delivery is completed, it is confirmed within the app, and the server records the delivery completion information again on the blockchain.

[1463] Specific examples

[1464] As a concrete example, let's say someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and enters the text information, "I found it on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database.

[1465] Meanwhile, if the person who lost their wallet uses the app to search for "I lost a black leather wallet. It's about 15cm x 10cm," the server will use this information to search the database and match it with the registered information of the person who found it. The matching results are notified to both parties, who then communicate via chat and hand over the wallet at a designated location. After the handover is complete, the server records this information on the blockchain to ensure overall transparency and reliability.

[1466] In this way, the present invention is a system that allows users to easily register and search for lost items, performs quick and accurate matching, and utilizes blockchain technology to ensure overall transparency and reliability.

[1467] The processing flow will be explained below.

[1468] Handling user registration

[1469] Step 1:

[1470] User: Launches the app and accesses the user registration screen.

[1471] Step 2:

[1472] User: Enter the required information such as name, contact details, and email address, and tap the "Register" button.

[1473] Step 3:

[1474] Terminal: The entered user information is sent to the server in JSON format.

[1475] Step 4:

[1476] Server: Analyzes the received user information and stores it in a database. Generates a response indicating registration completion and sends it to the terminal.

[1477] Step 5:

[1478] Terminal: Receives notification of registration completion and displays it to the user.

[1479] Processing lost and found entries

[1480] Step 1:

[1481] User: Access the app's "Lost Item Registration" screen to register the item they found.

[1482] Step 2:

[1483] User: Take a photo of the lost item, enter the necessary text information (e.g., location, date, time, item characteristics, etc.), and tap the "Register" button.

[1484] Step 3:

[1485] Terminal: Sends photo data and text information to the server.

[1486] Step 4:

[1487] Server: The received photo data is passed to the image recognition module, which analyzes the object's detailed information (color, shape, size, brand name, etc.).

[1488] Step 5:

[1489] Server: Combines the analysis results with the text information and stores them in a database. Sends a notification of registration completion to the device.

[1490] Step 6:

[1491] Terminal: Receives notification of registration completion and displays it to the user.

[1492] Processing lost and found searches

[1493] Step 1:

[1494] User: Access the app's "Lost Item Search" screen and enter the characteristics of the lost item.

[1495] Step 2:

[1496] User: Check the information entered and tap the "Search" button.

[1497] Step 3:

[1498] Terminal: Sends the input characteristic information to the server.

[1499] Step 4:

[1500] Server: Passes the received feature information to the natural language processing module for analysis.

[1501] Step 5:

[1502] Server: Compares the analyzed information with the lost items information in the database.

[1503] Step 6:

[1504] Server: If a matching item is found, it generates a matching result and notifies the user's device. If not found, it notifies the user.

[1505] Step 7:

[1506] Terminal: Receives the matching result notification and displays it to the user.

[1507] Processing of match result notifications

[1508] Step 1:

[1509] Server: Checks the database for a match between the user's dropped item and the item they found.

[1510] Step 2:

[1511] Server: Notifies the users (those who dropped the item and those who picked it up) of the matching results.

[1512] Step 3:

[1513] Server: Records the matching information immutably on the blockchain.

[1514] Contact and delivery processing

[1515] Step 1:

[1516] Users: Receive in-app notifications, check match results, and use the chat feature to contact the finder.

[1517] Step 2:

[1518] Users (those who lost the item and those who found it): Use the chat function to communicate and decide the details of the delivery.

[1519] Step 3:

[1520] User (the person who lost the item): Receives the lost item from the finder at the designated location. The completion of the delivery is confirmed within the app.

[1521] Step 4:

[1522] User (the person who found the item): Confirm in the app that the item was returned safely.

[1523] Step 5:

[1524] Server: Receives information about the completion of the transfer and records it on the blockchain.

[1525] The above is the specific operation of each processing step. The user, device, and server work together to efficiently complete the series of processes of registering, searching, matching, and handing over lost items.

[1526] Example 1

[1527] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1528] In conventional lost property management systems, it was difficult to quickly and accurately connect the person who found the lost item with the person who dropped it, and the transparency and reliability of information were not sufficiently ensured.In addition, the means of communication between users and information management after the handover were insufficient, which reduced the reliability of the entire system.

[1529] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1530] In this invention, the server includes a terminal for users to upload photos of lost items, an information processing device that receives the uploaded photos and analyzes the features of the object using image recognition means, means for saving the analyzed features in a storage device, an information processing device that receives feature information entered by the user who lost the item, analyzes it using natural language processing means, and compares it with the object information in the storage device, means for notifying the user of a matching result based on the generated comparison information, means for recording the matching information in a fixed ledger technology to ensure information transparency and reliability, means for users to receive the notification and communicate using a chat function, and means for recording information after the transfer is completed in the fixed ledger technology again. This allows the person who found the lost item to quickly and accurately connect with the person who dropped it, ensures information transparency and reliability, allows users to efficiently communicate with each other, and enables information management after the transfer is completed.

[1531] A "user terminal" is a device for uploading photos of lost items, and has the function of transmitting the input information to a server.

[1532] An "information processing device" is a device that has the function of analyzing input information using image recognition means and natural language processing means and storing the information in a storage device called a database.

[1533] "Image recognition means" refers to technology or software that analyzes uploaded photographs and identifies the characteristics of objects contained within them.

[1534] "Natural language processing means" refers to technology or software that analyzes text information entered by a user and understands its meaning and intent.

[1535] The "chat function" is a communication function that allows users to exchange messages and information in real time.

[1536] "Fixed ledger technology" is a technology similar to blockchain technology that preserves information once recorded with transparency and reliability, without being tampered with.

[1537] A "memory device" is a database or other storage system for storing analyzed features and other data.

[1538] "Matching results" refer to the server analyzing the characteristic information provided by the user and comparing it with information in the database to identify matching lost item information.

[1539] "Information on completion of delivery" is data indicating that the delivery of the lost item, which was determined through communication between users, has been completed.

[1540] This invention is a system that quickly and accurately connects the person who found a lost item with the person who lost it, utilizing image recognition and natural language processing capabilities, as well as blockchain technology to ensure the transparency and reliability of information.

[1541] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. The device receives this information and sends it to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete. The server uses an SQL database such as MySQL or PostgreSQL.

[1542] Next, the user takes a photo of the lost item and sends the necessary text information (e.g., location, date, time, and characteristics of the item) from the device to the server. The server analyzes the received photo using TensorFlow and OpenCV to extract detailed information about the item (color, shape, size, brand name, etc.). These details are stored in a database along with the text information, and the device is notified that registration is complete.

[1543] To search for a lost item, the user inputs feature information as text, which the device then sends to the server. The server then analyzes the input text using a natural language processing model such as BERT and compares it with object information in a database. If a matching item is found, the server notifies the user's device of the matching result. If the item is not found, the server sends a "not found" notification.

[1544] The server then sends notifications to both the person who lost the item and the person who found it based on the information collated in the database.The server also records this matching information on a blockchain such as Ethereum, ensuring the transparency and reliability of the information.

[1545] The user receives a notification and uses the chat function within the app (for example, Firebase chat service) to communicate with the finder. After communicating, they decide the details of the delivery. After the delivery is completed, this is confirmed within the app, and the server records the delivery completion information again on the blockchain.

[1546] As a concrete example, let's say someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and enters the text information, "I found it on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database.

[1547] Meanwhile, if the person who lost their wallet uses the app to search for "I lost a black leather wallet. It's about 15cm x 10cm," the server will use this information to search the database and match it with the registered information of the person who found it. The matching results are notified to both parties, who then communicate via chat and hand over the wallet at a designated location. After the handover is complete, the server records this information on the blockchain, ensuring overall transparency and reliability.

[1548] In this way, users can easily register and search for lost items, and matching can be done quickly and accurately, while blockchain technology is used to ensure overall transparency and reliability.

[1549] Example prompt sentence:

[1550] "I found a black leather wallet on a bench at the station. It measures 15cm x 10cm. How can I find the owner of this wallet?"

[1551] The above is an embodiment of the present invention.

[1552] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1553] Step 1: User Registration

[1554] The user uses the device and launches the app. The user enters the required information, such as their name, contact details, and email address. The device receives this information and sends it to the server. The server analyzes the received information and stores it in a database. It then generates a response indicating registration is complete and sends it to the device.

[1555] Input: Name, contact details, and email address entered by the user.

[1556] Data processing: Sending data from the device to the server.

[1557] Data calculation: Data analysis and database storage on the server.

[1558] Output: Registration complete response.

[1559] What happens: When a user clicks the "Register" button, the information is sent to the server and confirmed to be saved in the database. A notification of successful registration is displayed on the device.

[1560] Step 2: Register your lost property

[1561] The user uses their device to take a photo of the lost item and uploads it to the app. The user then enters text information such as the location, date, and item characteristics, and sends it from the device to the server. The server then analyzes the received photo data using image recognition techniques (e.g., TensorFlow or OpenCV) to extract detailed information about the item (color, shape, size, brand name, etc.). The extracted information and text information are then stored in a database, and the user is notified that registration is complete.

[1562] Input: A photo taken by the user and text information.

[1563] Data processing: Sending photos and text information from the device to the server.

[1564] Data calculation: Analyzing photo data and extracting detailed information using image recognition methods.

[1565] Output: Notification that registration is complete.

[1566] Specific operation: When a user presses the "Register" button, the photo and text information are sent to the server, where image analysis is performed. The results are stored in a database, and a notification of registration completion is displayed on the device.

[1567] Step 3: Search for lost property

[1568] The user uses their device to input textual information about the features of the dropped item, which is then sent from the device to the server. The server then uses natural language processing (e.g., the BERT model) to analyze the input feature information and compare it with object information in the database. If a matching item is found, a matching result is generated and notified to the user's device.

[1569] Input: Feature information entered by the user.

[1570] Data processing: Sending characteristic information from the device to the server.

[1571] Data Computing: Analysis of feature information and database search using natural language processing tools.

[1572] Output: Matching result or "not found" notification.

[1573] Specific operation: When the user presses the "Search" button, the characteristic information is analyzed and compared with the information in the database. The results are displayed on the device.

[1574] Step 4: Notification of match results

[1575] The server then sends notifications to both the person who lost the item and the person who found it based on the matched information in the database, and records this matching information on the blockchain to ensure the transparency and reliability of the information.

[1576] Input: The matched information.

[1577] Data processing: Creating notifications of matching results.

[1578] Data calculation: Blockchain recording of matching information.

[1579] Output: Notification of matching results and recording to the blockchain.

[1580] Specific operation: The server checks the matching information, generates a notification and sends it to the user. The result is displayed on the device and simultaneously recorded on the blockchain.

[1581] Step 5: Communication and delivery

[1582] The user receives a notification and uses the chat function within the app (for example, Firebase chat service) to communicate with the finder. After communicating, they decide the details of the delivery. After the delivery is completed, this is confirmed within the app, and the server records the delivery completion information again on the blockchain.

[1583] Input: Completed handover information.

[1584] Data processing: Communication via chat and confirmation of completion of delivery.

[1585] Data calculation: Blockchain recording of delivery completion information.

[1586] Output: A confirmation that the transaction has been completed and recorded on the blockchain.

[1587] How it works: Users communicate and decide on the details of the transaction. Once the transaction is complete, the information is recorded on the blockchain.

[1588] The above is a detailed description of each processing step of the system.

[1589] (Application example 1)

[1590] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1591] In modern factory environments, lost parts and tools have a significant impact on production efficiency. Work is often delayed when workers are unable to find the parts they need. Managing and locating lost items requires a great deal of time and effort, creating a need for a system that can quickly and efficiently resolve this issue. Furthermore, measures to ensure transparency and reliability are insufficient, and this also requires improvement.

[1592] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1593] In this invention, the server includes a means for preventing and quickly locating parts and tools lost in a factory environment, a means for searching for parts information by voice input or text input and analyzing it with natural language processing means, and a means for notifying the owner of the lost item with an internal notification function. This makes it possible to quickly and efficiently locate lost parts and tools in the factory, ensuring transparency and reliability.

[1594] Key Word Definitions

[1595] A "user terminal" is a computing device used by a user, such as a smartphone or tablet.

[1596] A "server" is a computer system that communicates with user terminals over a network and processes and stores data.

[1597] "Image recognition means" is a technology for detecting objects from photographs and videos taken with a camera and analyzing their features.

[1598] A "database" is a system for efficiently storing, retrieving, and managing information.

[1599] "Natural language processing means" refers to technology that enables computers to understand, interpret, and generate human language.

[1600] "Blockchain" is a distributed digital ledger and technology that increases the transparency and reliability of information.

[1601] An "internal notification function" is a system that sends messages and alerts to specific users or devices.

[1602] "Factory environment" means the physical and working environment within a factory where products are manufactured.

[1603] "Voice input" is a method of inputting voice as data through a microphone.

[1604] "Text entry" is a method of entering characters and symbols as data using a keyboard or software.

[1605] MODE FOR CARRYING OUT THE INVENTION

[1606] The system for implementing this invention aims to efficiently prevent and quickly locate parts and tools lost in a factory environment. The technical elements that make up this system and their specific processing will be described below.

[1607] System Components

[1608] User terminal

[1609] User devices can be smartphones, tablets, or factory robots. These devices are used to take photos of parts and tools, and provide an interface for registering information, searching, and receiving notifications. The main software components are a mobile app and a client application for data transfer.

[1610] server

[1611] The server is the central component that communicates with user devices over the network, processes uploaded photos and text information, manages the database, notifies matching results, and records information on the blockchain. It includes the following main software and hardware components:

[1612] Image recognition: Uses image processing libraries such as TensorFlow and OpenCV to extract object features from uploaded photos.

[1613] Database: A relational database management system such as MySQL or PostgreSQL is used to store the analyzed feature information.

[1614] Natural language processing tools: NLP libraries such as BERT and spaCy are used to analyze the text information entered by the user and match it with object information in the database.

[1615] Notifications: Use notification services such as Firebase Cloud Messaging to notify users of match results and other important information.

[1616] Blockchain: Using Ethereum, Hyperledger, etc., matching information and delivery completion information are recorded on the blockchain to ensure transparency and reliability.

[1617] Usage and Operation Procedures

[1618] First, the user registers with the system using their user terminal, and the necessary information is stored in a database. Next, when the user registers information about a lost or found item, they take a photo using their terminal and send the photo along with detailed text information to the server. The server receives this data and analyzes it using image recognition and natural language processing. The analyzed data is then stored in a database and, if necessary, notifies the user of the matching results.

[1619] After the delivery is complete, the delivery details are also recorded on the blockchain again, ensuring overall transparency and reliability. This prevents loss of production efficiency due to lost parts or tools in the factory and ensures that needed items can be found quickly.

[1620] Specific examples

[1621] For example, if a "blue drill bit (50mm)" is lost in a factory, a worker launches the smartphone app and searches for "lost drill bit." They voice-type "blue drill bit, 50mm" into the app, and the AI ​​converts the voice to text, then searches the database for a matching part. The matching part information is notified, and the blue drill bit picked up by another worker is found. They communicate using the built-in chat function to complete the part transfer. The transfer completion information is then recorded on the blockchain.

[1622] Prompt sentence for generative AI model

[1623] User Input: "Blue drill bit, 50mm"

[1624] AI process: Converts voice input into text and searches a database for matching parts.

[1625] Result: Notify the customer of the matching part information and communicate with them via the internal chat function.

[1626] The above is a specific embodiment for implementing this system, which makes it possible to efficiently manage parts and tools in a factory environment and to manage information in a transparent and reliable manner.

[1627] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1628] System processing steps

[1629] Processing flow

[1630] Step 1:

[1631] A user launches the app on a user device such as a smartphone or tablet and enters their own information (name, contact details, email address, etc.). This input information is sent from the user device to the server. The server receives the sent information and stores it in a database. This process registers the user's basic information within the system.

[1632] Input: Name, contact information, email address

[1633] Output: User information stored in the database

[1634] Step 2:

[1635] The user takes a photo of the part or tool they have picked up and enters the necessary text information (e.g., location, date and time, and object characteristics) into their user terminal. The user terminal then sends this information to the server. The server receives the photo data and text information, analyzes the photo data using image recognition tools, and extracts the object's characteristics (color, shape, size, etc.). The extracted information and the entered text information are stored in a database.

[1636] Input: Photo of the object, location, date and time, object characteristics

[1637] Output: Object features and text information stored in a database

[1638] Step 3:

[1639] When a user drops an object, they input feature information into their user terminal or transmit it via voice input. The user terminal then transmits the input feature information to the server. The server uses natural language processing to analyze the text and voice data and compares it with object information in a database. If the server finds matching object information, it notifies the user terminal of the details.

[1640] Input: Feature information (text or audio)

[1641] Output: Matching result notification

[1642] Step 4:

[1643] If the server finds a match, it records the information on the blockchain to ensure transparency and reliability. Information is securely recorded and cannot be tampered with.

[1644] Input: Matching results

[1645] Output: Information recorded on the blockchain

[1646] Step 5:

[1647] Users who receive a matching notification can use the app's chat function to confirm information about the items they found and lost, and to communicate details about the delivery. The server also manages chat data in real time.

[1648] Input: Chat message

[1649] Output: Send a message to the other user

[1650] Step 6:

[1651] After the item has been delivered, the user confirms the delivery in the app, and the server records this information back on the blockchain, ensuring transparency and authenticity of the transaction completion.

[1652] Input: Delivery completion information

[1653] Output: Delivery completion information recorded on the blockchain

[1654] Specific examples of processing

[1655] Step 1: User Registration

[1656] The user's device transmits the name, contact information, and email address entered in the app, which is then received by the server and stored in a MySQL database.

[1657] Step 2: Register your lost property

[1658] The user's device sends a photo of the object and text information to the server, which then uses TensorFlow to analyze the photo, extract the object's features, and store them in a database.

[1659] Step 3: Find your lost item

[1660] The user inputs the characteristics of the lost item via text or voice, and the user device sends this to the server. The server analyzes it using BERT, compares it with the database, and notifies the user device of any matches.

[1661] Step 4: Record your match information

[1662] The server records the matching results on the blockchain using Ethereum, which ensures the transparency and immutability of the information.

[1663] Step 5: Chat with us

[1664] Users who receive a match notification can contact each other using the app's chat function. The server sends and receives messages in real time using WebSocket.

[1665] Step 6: Record the completion of the handover

[1666] Once the transfer is complete, the user device sends the completion information to the server, which then records this information on the blockchain again to ensure the transparency of the transfer details.

[1667] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1668] A specific embodiment of the present invention is described below. The present invention is a system that quickly and accurately connects the person who found a lost item with the person who lost it, utilizing the image recognition and natural language processing capabilities of AI, as well as blockchain technology and an emotion engine to ensure the transparency and reliability of information.

[1669] User Registration

[1670] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. This information is received by the device and sent to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete.

[1671] Registering lost items

[1672] The user takes a photo of the lost item they found, enters the necessary text information (e.g., location, date, time, characteristics of the item, etc.), and sends it from the device to the server. The server analyzes the photo data using image recognition to extract detailed information about the item (color, shape, size, brand name, etc.). This extracted information is stored in a database along with the text information. The device is then notified that registration is complete.

[1673] Search for lost items

[1674] To search for a lost item, the user inputs characteristic information as text, which the device then sends to the server. The server then uses natural language processing to analyze the input text and compare it with object information in the database. If a matching item is found, the server notifies the user of the matching result; if not, it sends a "not found" notification.

[1675] Notification of matching results

[1676] The server then sends notifications to both the person who lost the item and the person who found it based on the information matched in the database. The server also records this matching information on the blockchain, ensuring transparency and reliability of the information.

[1677] Use of emotion engine

[1678] The server is equipped with an emotion engine that recognizes emotions from user input and actions. Matching notifications and chat responses are then reflected in the emotion engine's evaluation.

[1679] Communication and delivery

[1680] The user receives a notification and uses the app's chat function to contact the finder. After contact and confirmation of the details of the return, the emotion engine analyzes the user's emotional state and adjusts the response accordingly. Once the return is complete, the return is confirmed within the app, and the server records the return information on the blockchain.

[1681] Specific examples

[1682] As a concrete example, suppose someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and registers it by entering the text "I found this on a bench at the station." The server analyzes the photo, extracts the information "black leather wallet, approximately 15cm x 10cm," and stores it in the database. Meanwhile, if the person who lost the wallet uses the app to search "I lost my black leather wallet, approximately 15cm x 10cm," the server uses this information to search the database and matches it with the registered information of the person who found it. The matching results are notified to both parties, who communicate via chat and then hand over the wallet at a designated location. During this process, the server uses its emotion engine to analyze the user's emotional state and provide appropriate responses and notifications. After the handover is complete, the server records this information on the blockchain to ensure overall transparency and reliability.

[1683] In this way, the present invention is a system that allows users to easily register and search for lost items, performs quick and accurate matching, and ensures overall transparency and reliability by utilizing blockchain technology and an emotion engine.

[1684] The processing flow will be explained below.

[1685] Handling user registration

[1686] Step 1:

[1687] User: Launches the app and accesses the user registration screen.

[1688] Step 2:

[1689] User: Enter the required information such as name, contact details, and email address, and tap the "Register" button.

[1690] Step 3:

[1691] Terminal: The entered user information is sent to the server in JSON format.

[1692] Step 4:

[1693] Server: Analyzes the received user information and stores it in a database. Generates a response indicating registration completion and sends it to the terminal.

[1694] Step 5:

[1695] Terminal: Receives notification of registration completion and displays it to the user.

[1696] Processing lost and found entries

[1697] Step 1:

[1698] User: Access the app's "Lost Item Registration" screen to register the item they found.

[1699] Step 2:

[1700] User: Take a photo of the lost item, enter the necessary text information (e.g., location, date, time, item characteristics, etc.), and tap the "Register" button.

[1701] Step 3:

[1702] Terminal: Sends photo data and text information to the server.

[1703] Step 4:

[1704] Server: The received photo data is passed to the image recognition module, which analyzes the object's detailed information (color, shape, size, brand name, etc.).

[1705] Step 5:

[1706] Server: Combines the analysis results with the text information and stores them in a database. Sends a notification of registration completion to the device.

[1707] Step 6:

[1708] Terminal: Receives notification of registration completion and displays it to the user.

[1709] Processing lost and found searches

[1710] Step 1:

[1711] User: Access the app's "Lost Item Search" screen and enter the characteristics of the lost item.

[1712] Step 2:

[1713] User: Check the information entered and tap the "Search" button.

[1714] Step 3:

[1715] Terminal: Sends the input characteristic information to the server.

[1716] Step 4:

[1717] Server: Passes the received feature information to a natural language processing (NLP) module for analysis.

[1718] Step 5:

[1719] Server: Compares the analyzed information with the lost items information in the database.

[1720] Step 6:

[1721] Server: If a matching item is found, it generates a matching result and notifies the user device. If no matching item is found, it notifies the user device.

[1722] Step 7:

[1723] Terminal: Receives the matching result notification and displays it to the user.

[1724] Processing of match result notifications

[1725] Step 1:

[1726] Server: Checks the database for a match between the item the user dropped and the item they picked up.

[1727] Step 2:

[1728] Server: Notifies the users (those who dropped the item and those who picked it up) of the matching results.

[1729] Step 3:

[1730] Server: Records matching information on the blockchain to ensure transparency and reliability.

[1731] Use of emotion engine

[1732] Step 1:

[1733] Server: The emotion engine analyzes the text and behavioral data entered by the user and recognizes the user's emotions.

[1734] Step 2:

[1735] Server: Runs a program that adjusts notification content and chat responses based on the user's emotions.

[1736] Step 3:

[1737] Server: Sends emotion-based adjustment results to the user's device.

[1738] Step 4:

[1739] Terminal: Displaying tailored notifications and responses to the user.

[1740] Contact and delivery processing

[1741] Step 1:

[1742] Users: Receive in-app notifications, check match results, and use the chat feature to contact the finder.

[1743] Step 2:

[1744] Users (those who lost the item and those who found it): Use the chat function to communicate and decide the details of the delivery.

[1745] Step 3:

[1746] User (the person who lost the item): Receives the lost item from the finder at the designated location. The completion of the delivery is confirmed within the app.

[1747] Step 4:

[1748] User (the person who found the item): Confirm in the app that the item was returned safely.

[1749] Step 5:

[1750] Server: Receives information about the completion of the transfer and records it on the blockchain.

[1751] Specific examples

[1752] As a concrete example, let's say someone finds a wallet at a train station. The person opens the app, takes a photo of the wallet, and enters the text information, "I found it on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database.

[1753] Meanwhile, if the person who lost their wallet uses the app to search for "I lost a black leather wallet. It's about 15cm x 10cm," the server uses this information to search its database and finds a match with the registered information of the person who found it. The matching results are notified to both parties, who communicate via chat and then hand over the wallet at a designated location. During this process, the server uses its emotion engine to analyze the user's emotional state and provide appropriate responses and notifications. After the handover is complete, the server records this information on the blockchain to ensure overall transparency and reliability.

[1754] In this way, the present invention is a system that allows users to easily register and search for lost items, performs quick and accurate matching, and ensures overall transparency and reliability by utilizing blockchain technology and an emotion engine.

[1755] Example 2

[1756] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1757] In recent years, there has been a demand for a means to efficiently connect the finder and the person who lost a lost item. However, conventional systems lack transparency and reliability in this matching process, making it difficult to respond in a way that appropriately reflects the user's feelings. In particular, delays and misunderstandings can occur in information sharing and notification, which can detract from the overall user experience. The present invention aims to solve these issues and provide a system that connects users quickly, accurately, and reliably.

[1758] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an information processing device that uploads an image of the lost item, an information processing device that receives the uploaded image and analyzes the object's features using image recognition means, means for saving the analyzed features in a memory area, an information processing device that receives feature information entered by the person who lost the item, analyzes it using natural language processing means, and compares it with the object information in the memory area, means for notifying the information processing device of a matching result based on the compared information, means for recording the matching information in a distributed ledger to ensure transparency and reliability, and means for analyzing the user's emotions using an emotion engine and adjusting the response content. This allows users to quickly and accurately register and search for lost items, ensures information transparency and reliability, and enables appropriate responses according to the user's emotions.

[1759] An "information processing device" is a device that inputs, processes, stores, and outputs data, and includes computers, servers, and the like.

[1760] "Image recognition means" refers to technology that analyzes image data and automatically recognizes objects and features, and specifically refers to computer vision and deep learning technology.

[1761] A "storage area" is an area for storing data and information, and includes databases and cloud storage.

[1762] "Natural language processing means" refers to technology that analyzes text data, understands its content, and extracts necessary information, and includes natural language processing libraries and models.

[1763] "Verifying" refers to the process of comparing input information with existing data to see if it matches.

[1764] "Means of notification" refers to means for conveying specific information to a user, and includes push notifications, emails, etc.

[1765] A "distributed ledger" is a technology that records and manages transaction information in a distributed manner to ensure overall transparency and reliability, and includes blockchain technology.

[1766] An "emotion engine" is a system that analyzes emotions from user input and behavior and generates appropriate responses, and includes emotion analysis libraries and models.

[1767] The present invention is a system for quickly and accurately connecting the finder and the person who lost the lost item, and it uses a combination of image recognition, natural language processing, blockchain, and an emotion engine. The system is configured as follows.

[1768] User Registration

[1769] First, the user launches the application and enters the required information, such as name, contact details, and email address. The device then sends the entered information to the server. The server then stores the received information in a database and sends a notification of registration completion to the device. This process can use an SQL database or cloud storage.

[1770] Registering lost items

[1771] The user takes a photo of the lost item they found and enters text information such as the location, date, time, and characteristics of the item. The device then sends this to the server. The server then uses image recognition to analyze the photo data and extract detailed information about the object (color, shape, size, brand name, etc.). Image recognition libraries such as OpenCV and TensorFlow are used for the analysis. The extracted information is stored in a database along with the text information. A notification that registration is complete is then sent to the device.

[1772] Search for lost items

[1773] To search for a lost item, the user inputs characteristic information as text. The device then sends this to the server. The server then uses natural language processing (e.g., a natural language processing library or generative AI model) to analyze the input text information and compare it with object information in a database. If a matching item is found, the result is notified to the user's device. If not, a "not found" notification is sent.

[1774] Notification of matching results

[1775] The server then sends a notification to both the person who lost the item and the person who found it based on the information matched in the database. The server also records the matching information on a distributed ledger to ensure the transparency and reliability of the information. This process uses blockchain technology.

[1776] Use of emotion engine

[1777] The server is equipped with an emotion engine that recognizes emotions from user input and behavior. An AI model is used to analyze user emotions and reflect them in matching notifications and chat responses. Emotion analysis libraries and models are used for emotion recognition.

[1778] Communication and delivery

[1779] The user receives a notification and uses the app's chat function to contact the finder. After contact and the details of the delivery are decided, the server analyzes the user's emotional state using an emotion engine and adjusts the response accordingly. After the delivery is completed, the user confirms this in the app, and the server records the completion of the delivery again on the distributed ledger.

[1780] Specific examples

[1781] If someone finds a wallet at a station, they open the app, take a photo of the wallet, and enter the text information, "I found this on a bench at the station," to register it. The server analyzes the photo, extracts the information, "Black leather wallet, approximately 15cm x 10cm," and stores it in the database. Meanwhile, if the person who lost the wallet uses the app to search for "I lost my black leather wallet, approximately 15cm x 10cm," the server uses this information to search the database and matches it with the finder's registered information. The matching results are notified to both parties, who communicate via chat and then hand over the wallet at a designated location. During this process, the server uses an emotion engine to analyze the user's emotional state and provide appropriate responses and notifications. After the handover is complete, the server records this information on the blockchain to ensure transparency and reliability.

[1782] Prompt Sentence Examples

[1783] "I found a black leather wallet at the station. It measures 15cm x 10cm."

[1784] "I dropped my black leather wallet at the station. Has anyone found it?"

[1785] In this way, the system of the present invention allows users to easily register and search for lost items, performs quick and accurate matching, and utilizes blockchain technology and an emotion engine to ensure overall transparency and reliability.

[1786] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1787] Step 1: User Registration

[1788] The user launches the app and enters the required information such as name, contact details, and email address. The device assembles this input information into packets, encrypts them using SSL / TLS, and sends them to the server. The server deserializes the received information and saves it in a database (e.g., MySQL). Once saved, the server generates a registration completion status, encrypts it, and sends it back to the device. The device notifies the user that "registration is complete." Input: Name, contact details, email address. Output: Notification of registration completion.

[1789] Step 2: Register your lost property

[1790] The user uses the app to take a photo of the lost item and enters text information such as location, date and time, and characteristics. The device then assembles this into a packet and sends it to the server. The server then analyzes the received photo data using OpenCV to extract detailed information about the object (color, shape, size, brand name, etc.). This extracted information is integrated with the text information and stored in a database. Once registration is complete, the server generates a registration completion status and notifies the device. Input: photo data, location, date and time, characteristics. Output: notification that registration is complete.

[1791] Step 3: Search for lost property

[1792] The user enters the characteristic information of the dropped item as text into the app. The device sends this to the server. The server analyzes the received text information using natural language processing means (e.g., a generative AI model) and generates a search query. The server then uses this query to search a database for matching object information. If a matching item is found, details about it are sent to the user's device; if not, a "not found" message is sent. Input: Characteristic information text. Output: Search results or a "not found" message.

[1793] Step 4: Notification of match results

[1794] The server generates a notification message to both the person who dropped the item and the person who found it based on the matching results in the database. This message is encrypted and sent to the terminal. At the same time, the server records the matching information on the blockchain and generates a hash value for the transaction. Input: Matching result. Output: Notification message, hash value recorded on the blockchain.

[1795] Step 5: Use the Emotion Engine

[1796] The emotion engine built into the server analyzes user behavior and input in real time. It uses an AI model (e.g., emotion analysis library) to assess the user's emotional state and appropriately tailor responses and notifications based on this. Input: User behavior data, input text. Output: Emotion assessment, tailored responses and notifications.

[1797] Step 6: Communication and delivery

[1798] The user receives a matching notification and uses the app's chat function to contact the finder. The server collects the chat content and, if necessary, reanalyzes it using the emotion engine to adjust the response. After communication is established, the users decide on the details of the transfer. Once the transfer is complete, the user confirms it within the app, which the server receives and records on the blockchain. Input: Chat content, transfer information. Output: Emotion analysis results, transfer completion information recorded on the blockchain.

[1799] (Application example 2)

[1800] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1801] There is a need to quickly and accurately connect lost items with their owners, but existing systems do not ensure information transparency or reliability, and are unable to properly recognize and respond to emotions, resulting in a lack of sufficient improvement in the user experience. These issues are particularly likely to become apparent when managing and handing over lost items in physical stores.

[1802] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1803] In this invention, the server includes a user terminal that uploads a photo of the lost item, means for receiving the uploaded photo and analyzing the object's features using image recognition means, means for saving the analyzed features in a database, means for receiving feature information entered by the person who lost the item, analyzing it using natural language processing means, and comparing it with object information in the database, means for notifying the user terminal of a matching result based on the compared information, means for recording the matching information in a blockchain to ensure transparency and reliability, means for analyzing the user's emotional state using an emotion engine that recognizes emotions from the user's input and actions, and means for supporting the registration and search of lost items in physical stores. This enables fast and accurate registration, search, and delivery of lost items in physical stores.

[1804] "User terminal" refers to a device used to upload photos of lost items, including smartphones and smart glasses.

[1805] The "server" is a central system that receives uploaded photos, analyzes them using image recognition and natural language processing tools, and compares them with a database.

[1806] "Image recognition means" refers to technology for analyzing the characteristics of objects in uploaded photos and extracting information about them, and includes AI models and OCR (optical character recognition).

[1807] A "database" is a digital storage system used to store and later collate analyzed feature information and user-entered information.

[1808] "Natural language processing means" is a technology for analyzing text information entered by a user and comparing that information with object information in a database.

[1809] The "matching result" is the result of matching information between the lost item and the owner, which is generated based on the collated information.

[1810] "Blockchain" is a distributed ledger technology used to ensure transparency and reliability of information, with each transaction and record encrypted and connected in a chain.

[1811] An "emotion engine" is a technology for recognizing and analyzing emotions from user input and behavior.

[1812] "Physical store" refers to a sales or service establishment operated in a physical location, including commercial facilities where many lost items may occur.

[1813] "Registration and search support" means providing support to efficiently register and search for lost items in physical stores.

[1814] A specific embodiment of the present invention is described below. The present invention is a system that quickly and accurately connects the person who found a lost item in a physical store with the person who lost it, utilizing AI's image recognition and natural language processing capabilities, as well as blockchain technology and an emotion engine to ensure the transparency and reliability of information.

[1815] User Registration

[1816] First, the user launches the app and enters the necessary information, such as their name, contact details, and email address. This information is received by the device and sent to the server. The server stores the received information in a database and sends a response to the device indicating that registration is complete.

[1817] Registering lost items

[1818] The user takes a photo of the lost item, enters the necessary text information (e.g., location, date, time, characteristics of the item, etc.), and sends it from the device to the server. The server analyzes the photo data using image recognition to extract detailed information about the item (color, shape, size, brand name, etc.). This extracted information is stored in a database along with the text information. The device is then notified that registration is complete.

[1819] Search for lost items

[1820] To search for a lost item, the user inputs characteristic information as text, which the device then sends to the server. The server then uses natural language processing to analyze the input text and compare it with object information in the database. If a matching item is found, the server notifies the user of the matching result; if not, it sends a "not found" notification.

[1821] Notification of matching results

[1822] The server then sends notifications to both the person who lost the item and the person who found it based on the information matched in the database, and records this matching information on the blockchain to ensure the transparency and reliability of the information.

[1823] Use of emotion engine

[1824] The server is equipped with an emotion engine that recognizes emotions from user input and actions. Matching notifications and chat responses are then reflected in the emotion engine's evaluation.

[1825] Communication and delivery

[1826] The user receives a notification and uses the app's chat function to contact the finder. After contact and confirmation of the details of the return, the emotion engine analyzes the user's emotional state and adjusts the response accordingly. Once the return is complete, the return is confirmed within the app, and the server records the return information on the blockchain.

[1827] Hardware and software used

[1828] Hardware:

[1829] Smartphone: taking photos using the camera function, internet connection

[1830] Smart glasses: Photo and display functions

[1831] Robot: Lost item detection, voice guidance

[1832] software:

[1833] TensorFlow: for image recognition

[1834] pytesseract:OCR (optical character recognition)

[1835] Blockchain Library: Blockchain Technology

[1836] Natural Language Toolkit (NLTK): Natural language parsing and sentiment analysis

[1837] Specific examples

[1838] For example, if a black leather jacket is found in a store lobby, a staff member would take a photo of the black leather jacket using their smartphone and input information by voice or text, such as "Black leather jacket found in the store lobby." The system would then automatically extract detailed information about the item using image recognition and store it in a database. When a customer says something like "I lost my keys" at the cash register or information counter, the staff member would simply use the app to search for it. Matching is performed through image recognition and natural language processing, and the results are notified. Transparency is ensured by recording the information on the blockchain.

[1839] Example prompt:

[1840] "Analyze the image, extract the object features from the captured image, and link them with the text information. This will provide the best matching results based on the feature information and image information entered by the user."

[1841] This will enable quick and accurate registration, search, and delivery of lost items in physical stores.

[1842] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1843] Step 1:

[1844] The user launches the app and enters the required information, such as their name, contact details, and email address.

[1845] Input: Name, contact details, email address

[1846] Operation: The terminal receives the information entered by the user.

[1847] Output: User information entered

[1848] Step 2:

[1849] The terminal transmits the received user information to the server.

[1850] Input: Received user information

[1851] Operation: The terminal processes and sends user information to the server.

[1852] Output: User information sent to the server

[1853] Step 3:

[1854] The server stores the received user information in a database and sends a response indicating that registration is complete to the terminal.

[1855] Input: User information sent to the server

[1856] What happens: The server saves the user information in its database and generates a response confirming registration.

[1857] Output: Registration complete response

[1858] Step 4:

[1859] The user takes a photo of the lost item and enters text information such as the location, date, and characteristics of the item.

[1860] Input: Photo of lost item, text information

[1861] How it works: The user takes a photo on their device and enters the required information.

[1862] Output: Photo data and text information of lost items

[1863] Step 5:

[1864] The device sends photo data and text information about the lost item to the server.

[1865] Input: Photo data and text information of lost items

[1866] How it works: The device processes and sends the photo data and text information to the server.

[1867] Output: Photo data and text information sent to the server

[1868] Step 6:

[1869] The server analyzes the photo data using image recognition tools and extracts detailed information about the object (color, shape, size, brand name, etc.).

[1870] Input: Photo data sent to the server

[1871] How it works: Analyzes photo data using image recognition tools (e.g. TensorFlow) and extracts object features.

[1872] Output: Detailed information about what was parsed

[1873] Step 7:

[1874] The server stores the extracted information together with the text information in a database.

[1875] Input: Detailed information and text of the parsed object

[1876] How it works: Saves text information and extracted object features in a database.

[1877] Output: Information stored in the database

[1878] Step 8:

[1879] To search for a lost item, the user inputs characteristic information in text, which is then sent by the terminal to the server.

[1880] Input: Characteristic information text

[1881] Operation: The user inputs characteristic information, which the device sends to the server.

[1882] Output: Feature information sent to the server

[1883] Step 9:

[1884] The server analyzes the input feature information using natural language processing means and compares it with object information in the database.

[1885] Input: Feature information sent to the server

[1886] How it works: Analyzes the feature information using natural language processing tools (e.g., NLTK) and matches it with object information in a database.

[1887] Output: Search results

[1888] Step 10:

[1889] The server notifies the user of the matching results. If a matching item is found, the server notifies the user, and if not, it sends a "not found" notification.

[1890] Input: Search results

[1891] Operation: The server sends the search results to the user's device.

[1892] Output: Search results notified to the user's device

[1893] Step 11:

[1894] The server records the matching information on the blockchain to ensure transparency and reliability.

[1895] Input: Matching information

[1896] What it does: Records matching information in a blockchain library.

[1897] Output: Information recorded on the blockchain

[1898] Step 12:

[1899] The server is equipped with an emotion engine that recognizes emotions from user input and actions.

[1900] Input: User input and behavior information

[1901] How it works: Recognize and analyze user emotions with the emotion engine.

[1902] Output: Emotional state analysis result

[1903] Step 13:

[1904] The server analyzes the user's emotional state and reflects this in matching notifications and responses in the chat function.

[1905] Input: Emotional state analysis result

[1906] What it does: Adjusts response content based on emotion engine analysis.

[1907] Output: The adjusted response

[1908] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1909] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1910] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1911] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1912] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort...

Claims

1. a user terminal for uploading photos of lost items; a server that receives the uploaded photograph and analyzes the characteristics of the object using image recognition means; means for storing the analyzed features in a database; a server that receives the characteristic information input by the person who dropped the object, analyzes it using natural language processing means, and compares it with object information in a database; means for notifying a user terminal of a matching result based on the collated information; A system that includes a means to record matching information on the blockchain and ensure transparency and reliability.

2. The system of claim 1 , further comprising means for providing a chat function to enable users to stay in contact after matching.

3. The system according to claim 1, further comprising means for recording information after the transfer is completed in the blockchain again.

Images