System

Abstract

A system is provided.SOLUTION: The system includes a means for acquiring face image data, a means for digitizing the face image data, a means for storing the digitized face data in a database, a means for inputting a face photograph of a parent when a missing child occurs, a means for collating the face photograph data of the parent with the face data of the database, a means for acquiring images of a plurality of monitoring cameras in a facility, a means for collating the images of the monitoring cameras with the face data of the database to specify the position of the missing child, and a means for notifying the specified position information of the missing child.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In large facilities, there is a high risk of children getting lost, and the inability to quickly reunite parents and children creates anxiety. Current systems rely primarily on human resources to conduct searches, which takes time, making it difficult to quickly reunite parents and children. Another issue is that they cannot efficiently monitor a wide area within a facility, and can only respond in specific locations. [Means for solving the problem]

[0005] This invention relates to a system including a means for acquiring facial image data, a means for digitizing the facial image data, a means for storing the digitized facial data in a database, a means for inputting a parent's facial photograph when a child gets lost, a means for comparing the parent's facial photograph data with the facial data in the database, a means for acquiring footage from multiple surveillance cameras within a facility, a means for locating the lost child by comparing the surveillance camera footage with the parent's facial photograph data, and a means for notifying the location of the identified lost child. The system acquires facial image data in real time using a facial recognition camera, digitizes it, and stores it in a database. Furthermore, when a child gets lost, a parent's facial photograph is input and compared with the database. Furthermore, by comparing the parent's facial photograph data with footage from multiple surveillance cameras within the facility, the system quickly locates the lost child and notifies the parent and facility staff, enabling a rapid reunion.

[0006] "Facial image data" is data that records an image of a visitor's face in digital format.

[0007] "Digitalization" is the process of converting analog information into digital information.

[0008] A "database" is a system that organizes information and allows it to be stored and searched efficiently.

[0009] "When a child gets lost" refers to a situation where a parent and child become separated within the facility and the child is temporarily separated from their parents.

[0010] "Facial photo input" means taking a photo of the parent's face and inputting that data into the system.

[0011] "Matching" is the process of comparing two or more pieces of data to see if they match.

[0012] "Monitoring camera video" refers to video data captured by a monitoring camera.

[0013] "Location information" is data about the geographic location of a particular object or person.

[0014] "Notification" means the communication of specific information to an interested party.

[0015] A "system" is a set of devices or software in which multiple elements work together to perform a specific function. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention provides a system for quickly finding a child who has become lost in a large facility by using facial recognition and video data from a surveillance camera. A specific embodiment of the present invention will be described in detail below.

[0038] Acquiring facial image data

[0039] The server captures visitors' facial images in real time from facial recognition cameras installed at the facility entrances. The facial recognition cameras provide high-resolution images, which are necessary for generating accurate digital values ​​in the facial recognition system. The captured facial image data is then digitized and encoded on the spot.

[0040] Storing digitized facial data

[0041] The server stores the encoded facial data in a database, which is used to safely and efficiently manage existing visitors' facial data, allowing for data storage and rapid retrieval.

[0042] Parents' photos can be entered when a child gets lost

[0043] When a child gets lost, the user (facility staff) enters a photo of the parent's face into a dedicated terminal at the lost child center. This photo is then digitized and sent to the server. The server encodes the parent's face data and compares it with the facial data in the database.

[0044] Verifying parent-child relationships

[0045] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database. This process verifies whether the parent and child entered the facility together. The AI ​​checks the degree of similarity of facial features and identifies the parent-child relationship with high accuracy.

[0046] Matching and tracking with surveillance camera footage

[0047] The server acquires video footage from security cameras installed in each area of ​​the facility. The server analyzes each frame of the security camera footage in real time and compares it with the matched parent's facial data. By matching the security camera video data with the parent's facial data, the lost child can be quickly identified.

[0048] Location notification

[0049] If a lost child's location is identified, the server notifies the lost child center, which then provides the specific location information to the parents and facility staff, enabling the child and parent to be reunited quickly and providing peace of mind.

[0050] As a concrete example, consider a large theme park. A facial recognition camera is installed at the entrance, and facial data of visitors is collected when they enter the park. When a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the surveillance camera footage within the park, and if a specific camera captures the lost child, it notifies the child of the child's location. Using this information, the parent and facility staff can quickly reunite the child.

[0051] The above is a specific embodiment of the present invention. This system allows lost children to be quickly reunited with their parents, and increases safety and security for facility users.

[0052] The processing flow will be explained below.

[0053] Step 1:

[0054] The server acquires visitors' facial images in real time from a facial recognition camera installed at the entrance of the facility. The facial images are captured by a high-resolution camera.

[0055] Step 2:

[0056] The server sends the acquired facial image data to the facial recognition system and converts it into digital values, which digitize facial feature points and enable subsequent matching.

[0057] Step 3:

[0058] The server stores the digitized facial data in a database, along with the date, time, and camera position at the time of storage.

[0059] Step 4:

[0060] When a child is lost, the user (facility staff) enters a parent's face photo into the system using a dedicated terminal at the lost child center. The photo is taken using a high-resolution camera.

[0061] Step 5:

[0062] The server digitizes the parent's facial photo and converts it into encoded data, which serves as the basis for matching the lost child.

[0063] Step 6:

[0064] The server uses AI to compare the facial data stored at the facility entrance with the parent's facial data. The AI ​​analyzes the degree of similarity of facial feature points, and if there is a match, it recognizes the parent-child relationship.

[0065] Step 7:

[0066] The server acquires video in real time from surveillance cameras installed in each area of ​​the facility, and each surveillance camera periodically transmits video frames to the server.

[0067] Step 8:

[0068] The server analyzes the video frames from the security cameras and compares them with the stored facial data of the parents, checking whether the facial data matches the specific camera footage.

[0069] Step 9:

[0070] The server identifies the child's location from the surveillance camera footage that matches the parent's facial data, and records the identified location as specific area information.

[0071] Step 10:

[0072] The server then notifies the lost child center of the child's location, including detailed location information and the latest camera footage.

[0073] Step 11:

[0074] The user (facility staff) will then communicate the location information received at the lost child center to the parents, who can then head to the designated area based on the specific location information and quickly reunite with their child.

[0075] This series of processes enables lost children to be quickly reunited with their parents, increasing the safety and security of the facility.

[0076] Example 1

[0077] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0078] When a child gets lost in a large facility, it poses a serious problem for the safety and security of parents and children. With the current system, it is difficult to quickly and accurately find a lost child, which increases the burden on facility staff. In addition, the process of collecting and confirming information when a child gets lost is complicated and time-consuming, so a quick response is required.

[0079] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0080] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a parent's facial photo when a child gets lost, means for digitizing the parent's facial photo data and transmitting it to the server, means for comparing the parent's facial photo data with the facial data in the database using artificial intelligence, means for acquiring footage from multiple surveillance cameras in the facility in real time, means for comparing each frame of the surveillance camera footage with the parent's facial data to identify the location of the lost child, and means for sending a push notification of the identified location of the lost child to the lost child center.This makes it possible to quickly and accurately find a lost child and quickly reunite the parent and child.

[0081] "Facial image data" is digital data that captures the facial features of a visitor.

[0082] "Digitization" is the process of electronically processing analog facial image data to convert it into digital form.

[0083] "Database" means a computer system for storing and managing captured and encoded facial data.

[0084] "Method for inputting a photograph of the parents' faces when a child becomes lost" refers to a method in which facility staff digitally input a photograph of the parents' faces using a dedicated terminal.

[0085] The "means for digitizing facial photograph data and transmitting it to a server" refers to a means for converting a photograph of a face into a digital format and transmitting it to a server.

[0086] "Artificial intelligence" refers to machine learning and deep learning models that perform complex processes such as facial recognition and data matching.

[0087] "Means for acquiring surveillance camera footage in real time" refers to means for continuously and immediately acquiring footage from multiple surveillance cameras installed within a facility.

[0088] The "means for collating each frame of surveillance camera video with parent's facial data" refers to a means for comparing each frame of surveillance camera video acquired in real time with parent's facial data.

[0089] The "means for identifying the location of a lost child" is a means for identifying the location of a lost child based on the location information of surveillance camera footage that matches the facial data of the parent.

[0090] "Push notification" is a communication method for proactively sending the location information of a identified lost child to the lost child center.

[0091] This invention is a system that uses facial recognition and video data from surveillance cameras to quickly find a child who has become lost in a large facility. Specific embodiments of this system will be described in detail below, including the hardware and software used to process and calculate data.

[0092] The server captures visitors' facial images in real time using a high-resolution facial recognition camera (e.g., a typical high-performance camera) installed at the entrance to the facility. The facial recognition camera analyzes facial features and encodes them as digital data. This data is temporarily stored in the server's memory. The server then stores this encoded facial data in a database. This database can be built using, for example, MySQL (registered trademark).

[0093] When a child gets lost, the user (facility staff) uses a dedicated terminal at the lost child center to input a parent's facial photo. This terminal digitizes the parent's facial photo and sends the image data to a server. In this transmission process, the image captured by the terminal is transferred to the server via a network. The server then encodes the received parent's facial photo and stores it in a database.

[0094] The server uses artificial intelligence (AI models, such as Amazon Rekognition) to compare the facial data stored in the database with the parent's facial photo data. The AI ​​model evaluates the degree of match of facial features, and if a high match is detected, the parent-child relationship is identified.

[0095] The server captures and analyzes video footage from surveillance cameras (e.g., standard surveillance cameras) installed in each area of ​​the facility in real time. The server then applies a facial recognition algorithm to each frame of video footage from the security cameras and compares it with the parent's facial data. If a match is confirmed, the server records the location information of that frame and identifies the lost child.

[0096] Once the location of a lost child is identified, the server sends the information to the lost child center as a push notification. The lost child center's device receives this information and displays the location information to facility staff and parents. For example, by notifying them that the lost child is in a specific area, it helps to quickly reunite the child.

[0097] As a concrete example, imagine a large theme park. When visitors enter the park, a facial recognition camera at the entrance captures and stores their facial data. If a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the footage from the facility's security cameras, and if a specific security camera captures the lost child, it notifies them of the child's location. Based on this information, the parents and facility staff can quickly reunite the child.

[0098] This system will enable lost children to be quickly reunited with their parents, improving safety and peace of mind for facility users.

[0099] Example prompts to input to a generative AI model:

[0100] "Please explain how the system works by inputting a photo of the parent's face when a child goes missing and matching it with the security camera footage within the facility to locate the child."

[0101] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0102] Step 1: Acquire facial image data

[0103] The server captures visitors' facial images in real time using a high-resolution facial recognition camera installed at the entrance of the facility. The input data is the video stream from the camera. The server extracts individual facial images from this stream and encodes them into digital data. The converted output data is temporarily stored in the server's memory.

[0104] Specific behavior:

[0105] The server receives the video stream transmitted from the camera in real time.

[0106] The server extracts facial images from the video stream using a facial recognition algorithm.

[0107] The server encodes the extracted facial image and converts it into digital data.

[0108] Step 2: Save the digitized face data

[0109] The server stores the encoded facial data in a database. The input data for this step is the digitized facial data acquired in step 1. The server performs error checks before storing to ensure data integrity. Once integrity is confirmed, the data is stored in the database.

[0110] Specific behavior:

[0111] The server performs error checking on the encoded face data.

[0112] The server generates SQL queries to store the digitized face data in a database.

[0113] The server sends an SQL query to the database and receives a response confirming the save.

[0114] Step 3: Enter a photo of the parents' faces in case the child gets lost

[0115] When a child gets lost, the user (facility staff) uses a dedicated terminal at the lost child center to input a parent's facial photo. The input data for this step is the parent's facial photo. The terminal digitizes the input facial photo and sends the image data to the server. The server then encodes the received facial photo and stores it in a database.

[0116] Specific behavior:

[0117] The user (facility staff) takes a photo of the parent's face using a dedicated terminal.

[0118] The terminal digitizes the captured facial photograph.

[0119] The terminal transmits the digitized facial photograph to the server.

[0120] The server encodes the received parent's facial photograph and stores it in a database.

[0121] Step 4: Verify parent-child relationships

[0122] The server uses an artificial intelligence model to match the facial data of the parent entered with the facial data stored in the database at the entrance. The input data for this step is the facial data of the parent and the facial data acquired at the entrance. The server provides the input data to the AI ​​model and evaluates the degree of match of facial features. If a high match is detected, the server identifies the parent-child relationship.

[0123] Specific behavior:

[0124] The server inputs the parents' facial data into the AI ​​model.

[0125] The AI ​​model compares the parent's facial data with multiple facial data stored in the entrance database.

[0126] The server receives the match assessment from the AI ​​model and identifies parent-child relationships.

[0127] Step 5: Matching with CCTV footage and tracking

[0128] The server captures and analyzes real-time video from security cameras installed in each area of ​​the facility. The input data for this step are the video stream from the security cameras and the parent's facial data. The server applies a facial recognition algorithm to each frame from the security cameras and compares it with the parent's facial data. If a match is confirmed, the server records the location information of that frame.

[0129] Specific behavior:

[0130] The server receives the video streams from each surveillance camera.

[0131] The server applies a facial recognition algorithm to each video frame.

[0132] The server records the position information of the frame where a match is confirmed by comparing it with the parent's face data.

[0133] Step 6: Location Notification

[0134] Once the location of the lost child is identified, the server pushes the information to the lost child center. The input data for this step is the location information of the identified lost child. The server converts the location information into an explainable format and sends it to the lost child center's terminal. The lost child center's terminal receives the notification and displays the location information to facility staff and parents.

[0135] Specific behavior:

[0136] The server converts the identified location information of the lost child into an explainable format.

[0137] The server sends the location information to the lost child center's terminal as a push notification.

[0138] The terminal at the lost child center displays the location information to facility staff and parents based on the received location information.

[0139] (Application example 1)

[0140] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0141] When a child gets lost in a large facility, a system that can quickly and accurately locate the child is extremely important. However, with existing systems, it takes time to identify the lost child, causing anxiety and stress for parents and facility staff. Furthermore, there is a lack of a way for parents to directly obtain information about the lost child's location, further extending the time until the child is reunited. Furthermore, information sharing with lost child centers and security guards is often not smooth. A system that can solve these problems and respond quickly and efficiently when a child gets lost is needed.

[0142] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0143] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a parent's facial photograph when a child gets lost, means for comparing the parent's facial photograph data with the facial data in the database, means for acquiring footage from multiple surveillance cameras within the facility, means for comparing the surveillance camera footage with the parent's facial photograph data to identify the location of the lost child, means for notifying the parent's mobile device of the identified location information of the lost child, and means for sharing the location information notified to the parent's mobile device with the facility's lost child center and security guard terminals in real time. This allows parents to quickly determine the child's location and share the information with the lost child center and security guard, enabling a prompt reunion.

[0144] "Facial image data" refers to facial image information captured by a facial recognition camera or other camera device.

[0145] "Digitalized facial data" refers to facial image data that has been digitally digitized and converted into a form that is easy for a computer to handle.

[0146] A "database" refers to a system that efficiently manages accumulated digital data and quickly searches and obtains necessary information.

[0147] "Parent's face photo" refers to a photo of the face provided by the parent when the child gets lost. This photo is digitized and sent to the server.

[0148] "Matching" refers to the process of comparing one piece of data with another to see if they match.

[0149] A "surveillance camera" refers to a camera installed within a facility to monitor and record the movements of people and objects in real time.

[0150] "Location information of lost children" refers to information showing the current location of a lost child. This information is obtained by comparing video data from surveillance cameras with facial data.

[0151] "Parent's mobile device" refers to a mobile information device such as a smartphone or tablet owned by the parent. This device will be notified of the lost child's location.

[0152] A "lost child center" is a department located within a large facility that specializes in dealing with lost children.

[0153] "Security guard devices" refer to the mobile information devices owned by security guards at the facility. The location information of lost children is also shared with these devices.

[0154] This invention is a system for quickly reuniting parents and children when they get lost in a large facility. This system is composed of a facial recognition camera, a surveillance camera, a parent's mobile device, a lost child center, and a security guard's terminal.

[0155] The server first uses a facial recognition camera installed at the entrance of the facility to obtain facial image data. The facial recognition camera provides high-resolution images that allow the facial recognition system to generate accurate digital values. The facial image data is digitized and transmitted to the server in real time.

[0156] The server stores the captured digitized facial data in a database designed to securely and efficiently manage visitor facial data and enable rapid search and analysis.

[0157] When a child gets lost, the parent enters a facial photo via a dedicated terminal at the lost child center. This photo is then digitized and sent to a server. The server encodes the parent's facial photo data and compares it with facial data in a database. This identifies the parent-child relationship and confirms that the child has entered the facility.

[0158] The server captures and analyzes video footage from multiple surveillance cameras installed within the facility in real time. By comparing the video data from the surveillance cameras with facial photos of the parents, the location of the lost child can be identified. AI technology is used to confirm the degree of match of facial features with high accuracy.

[0159] When a lost child's location is identified, the server notifies the parent's mobile device of the location information. The location information is also shared in real time with the lost child center and security guards' devices. This function allows parents to quickly determine the child's location, and the lost child center and security guards can take action at the same time.

[0160] As an example of how this system can be put into practical use, consider a large shopping mall. As a parent and child pass through the entrance, facial image data is captured by a facial recognition camera. If a child gets lost, the parent provides a facial photo to the lost child center, and the data is collated on a server. After that, an analysis of the surveillance camera identifies the lost child as being in the food court, and the parent's smartphone is notified of the child's location. At the same time, this information is shared with the lost child center and security guards, allowing the parent and child to be quickly reunited.

[0161] An example of an input prompt for a generative AI model is:

[0162] "Please create a system to quickly find children who get lost in a shopping mall. The system will use facial recognition cameras and surveillance cameras to capture and match the facial data of parents and children. Please also implement a function to notify the parents' smartphones of the child's location when the child gets lost."

[0163] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0164] Step 1:

[0165] The server acquires the visitor's facial image data in real time using a facial recognition camera installed at the facility entrance. The facial recognition camera provides high-resolution images and generates accurate digital values ​​in the facial recognition system. The server acquires the facial image data as input and obtains the digitized facial data as output.

[0166] Step 2:

[0167] The server digitizes and encodes the acquired facial image data, converting it into a digital format that is easy for a computer to handle. It receives facial image data as input and obtains encoded facial data as output.

[0168] Step 3:

[0169] The server stores the encoded facial data in a database designed to efficiently manage visitor facial data and enable rapid search and analysis. The server receives digitized facial data as input and stores it in the database.

[0170] Step 4:

[0171] When a child gets lost, the user (parent) enters a facial photo through a dedicated terminal at the lost child center. The parent's facial photo is digitized on the spot and sent to the server. The parent's facial photo is obtained as input, and the digitized parent's facial photo data is obtained as output.

[0172] Step 5:

[0173] The server encodes the parent's facial photo data and matches it with facial data in the database. This process identifies the parent-child relationship and verifies whether the parent and child entered the facility together. The server receives digitized parent facial data as input and provides information on the degree of match between the facial data as output.

[0174] Step 6:

[0175] The server acquires video footage from multiple surveillance cameras installed within the facility in real time. The video data from the surveillance cameras is used to match the parent's facial data. The surveillance camera video is received as input, and the video data for analysis is obtained as output.

[0176] Step 7:

[0177] The server analyzes the surveillance camera footage and compares it with the parent's facial data. This analysis is performed using AI technology to confirm the degree of match of facial features with high accuracy. The server receives the surveillance camera footage and the parent's facial data as input, and obtains specific information about the matching parts as output.

[0178] Step 8:

[0179] The server identifies the location of the lost child and notifies the parent's mobile device of the location information. At the same time, the server also shares the location information with the lost child center and security guards' devices in real time. The server receives the matching part of the location information as input and outputs the location information notification.

[0180] Step 9:

[0181] The parents check the location information of their lost child that is notified to their mobile device, and then use that information to work with facility staff to find the lost child.The location information notification is received as input, and the actual on-site confirmation and response action are output.

[0182] The above are the specific processing steps of the system that realizes the application example.

[0183] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0184] This invention is a system that, when a child gets lost in a large facility, quickly finds the child using facial recognition and video data from a surveillance camera, and further recognizes the emotional states of the parent and child using an emotion engine, and provides appropriate support. Specific embodiments of the present invention will be described in detail below.

[0185] Acquiring facial image data

[0186] The server acquires visitors' facial images in real time from a facial recognition camera installed at the entrance of the facility. The facial images are captured by a high-resolution camera, which allows for detailed analysis of facial feature points.

[0187] Storing digitized facial data

[0188] The server sends the acquired facial image data to the facial recognition system and converts it into digital values. This conversion quantifies facial feature points, enabling subsequent matching. The digitized data is stored in a database and managed along with metadata such as date, time, and camera position.

[0189] Parents' photos can be entered when a child gets lost

[0190] When a child gets lost, the user (facility staff) enters a parent's facial photo into the system via a dedicated terminal at the lost child center. The photo is taken using a high-resolution camera and sent to the server. The server digitizes the parent's facial photo and converts it into encoded data.

[0191] Verifying parent-child relationships

[0192] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database. This process verifies whether the parent and child entered the facility together. The AI ​​checks the degree of similarity of facial features and identifies the parent-child relationship with high accuracy.

[0193] Matching and tracking with surveillance camera footage

[0194] The server acquires video in real time from surveillance cameras installed in each area of ​​the facility. Each surveillance camera periodically transmits video frames to the server. The server analyzes the video frames from the surveillance cameras and compares them with the stored facial data of the parents. By comparing the video data from the surveillance cameras with the facial data of the parents, lost children can be quickly identified.

[0195] Location notification

[0196] If a lost child's location is identified, the server notifies the lost child center with the location information, including the child's detailed location and the latest camera footage, providing parents and facility staff with specific location information and facilitating a speedy reunion.

[0197] Emotion recognition by emotion engine

[0198] The server is equipped with an emotion engine that recognizes the emotional state of parents and lost children in real time. The emotion engine analyzes facial expression and voice data acquired through a facial recognition camera and sensors on the dedicated device to estimate the emotional state. For example, if a parent is feeling stressed, the emotion engine will notify the server based on that information.

[0199] Supported by an emotional engine

[0200] The server provides appropriate support based on the emotional data acquired by the emotion engine. For example, if a parent is in a very unstable state, the server notifies the facility staff of this information and instructs them on how to provide psychological support and take appropriate measures. This reduces the anxiety of both parents and children and supports rapid reunification.

[0201] As a concrete example, consider a large theme park. A facial recognition camera is installed at the entrance, and facial data of visitors is collected when they enter the park. When a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the surveillance camera footage within the park, and if a specific camera captures the lost child, it notifies the location information and the parent's emotional state. Based on this information, the parent and facility staff can quickly reunite, and at the same time, the parent can receive appropriate support according to their emotional state.

[0202] The above is a specific embodiment of the present invention. This system allows lost children to be quickly reunited with their parents, enhancing safety and a sense of security for facility users. Furthermore, the use of an emotion engine reduces anxiety for parents and children, supporting a safe and secure facility experience.

[0203] The processing flow will be explained below.

[0204] Step 1:

[0205] The server acquires visitors' facial images in real time from a facial recognition camera installed at the entrance of the facility. The facial images are captured by a high-resolution camera.

[0206] Step 2:

[0207] The server sends the acquired facial image data to the facial recognition system and converts it into digital values. By quantifying facial features in detail, subsequent matching can be performed.

[0208] Step 3:

[0209] The server stores the digitized facial data in a database, along with metadata such as date, time, and camera position.

[0210] Step 4:

[0211] When a child gets lost, the user (facility staff) enters a photo of the parent's face into the system via a dedicated terminal at the lost child center. The photo is taken using a high-resolution camera.

[0212] Step 5:

[0213] The server digitizes the parent's facial photo data and converts it into encoded data, which is used as the basis for matching lost children.

[0214] Step 6:

[0215] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database. The AI ​​analyzes the degree of similarity of facial features and, if there is a match, recognizes the parent-child relationship.

[0216] Step 7:

[0217] The server acquires video in real time from surveillance cameras installed in each area of ​​the facility, and each surveillance camera periodically transmits video frames to the server.

[0218] Step 8:

[0219] The server analyzes the video frames from the security cameras and compares them with the stored facial data of the parents, checking whether the facial data matches the specific camera footage.

[0220] Step 9:

[0221] The server identifies the child's location from the surveillance camera footage that matches the parent's facial data, and records the identified location as specific area information.

[0222] Step 10:

[0223] The server then notifies the lost child center of the location of the identified lost child, including detailed location information and the latest camera footage.

[0224] Step 11:

[0225] The user (facility staff) will then communicate the location information received at the lost child center to the parents, who can then head to the designated area based on the specific location information and quickly reunite with their child.

[0226] Step 12:

[0227] The server activates an emotion engine that recognizes the emotional state of the parent and the lost child in real time. The emotion engine analyzes facial expression and voice data acquired through a facial recognition camera and sensors on the dedicated device to estimate the emotional state.

[0228] Step 13:

[0229] The server provides appropriate support based on the parent's emotional data analyzed by the emotion engine. For example, if the parent is feeling stressed, the server notifies facility staff of this information and instructs them on how to provide psychological support and take appropriate measures.

[0230] Step 14:

[0231] Based on the analysis results of the emotion engine, the user (facility staff) communicates with the parents in a way that gives them a sense of security, thereby reducing their anxiety and supporting a speedy reunion.

[0232] This series of processes not only enables lost children to be quickly reunited with their parents, but also reduces the anxiety of both parents and children and increases the sense of safety and security for facility users.

[0233] Example 2

[0234] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0235] Having a child get lost in a large facility is a major source of anxiety for parents. It is also extremely important for facility staff to quickly and accurately locate lost children, but conventional methods often require a lot of time and effort. Furthermore, understanding the emotional state of lost children and their parents and providing appropriate support has also been a challenge. Therefore, the objective of this invention is to provide a system that uses facial recognition technology and surveillance camera footage to quickly identify lost children and recognize and respond to the emotional state of parents and children.

[0236] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0237] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a parent's facial photo when a child gets lost, means for using artificial intelligence to compare the parent's facial photo data with the facial data in the database, means for acquiring footage from multiple surveillance cameras within the facility, means for locating the lost child by comparing the surveillance camera footage with the parent's facial photo data, means for notifying the location of the located lost child, means including an emotion engine for recognizing the emotional states of the parent and the lost child in real time, and means for providing appropriate support based on the emotion data acquired by the emotion engine. This enables rapid location identification of the lost child and appropriate support taking into account the emotional states of the parent and child.

[0238] "Means for acquiring facial image data" refers to devices or technologies for capturing high-resolution images of visitors' faces, and specifically includes facial recognition cameras.

[0239] "Means for digitizing facial image data" refers to software or systems for analyzing acquired facial images and converting them into numerical data.

[0240] "Means for storing digitized facial data in a database" refers to technologies or systems that securely and effectively store facial data converted into a numerical data format in a database along with certain metadata.

[0241] "Means for inputting a parent's facial photograph when a child gets lost" refers to a device or method for inputting and transmitting a parent's facial photograph into the system via a dedicated terminal.

[0242] "Means using artificial intelligence to match parent facial photograph data with facial data in the database" refers to a matching system that uses AI technology to compare parent facial photographs with facial data in the database and determine the degree of match with a high degree of accuracy.

[0243] "Means for acquiring footage from multiple surveillance cameras within a facility" refers to systems and technologies for collecting real-time video data from multiple surveillance cameras installed within a facility.

[0244] "Means for locating a lost child by comparing surveillance camera footage with facial photographic data of parents" refers to technology or systems for locating a lost child by comparing surveillance camera footage with facial photographic data of parents.

[0245] "Means for notifying the location of identified lost children" refers to a notification system that quickly notifies facility personnel of the precise location of identified lost children.

[0246] "Means including an emotion engine for recognizing the emotional states of the parent and the lost child in real time" refers to an emotion recognition engine for analyzing facial expressions and voice data and detecting the emotional states of the parent and the lost child in real time.

[0247] "Means for providing appropriate support based on emotional data obtained by the emotion engine" refers to a support system that uses data from the emotion recognition engine to give instructions to facility staff and suggest specific countermeasures.

[0248] This invention is a system that, when a child gets lost in a large facility, quickly finds the child using facial recognition and video data from a surveillance camera, and further recognizes the emotional states of the parent and child using an emotion engine, and provides appropriate support. Specific embodiments are described below.

[0249] System configuration

[0250] The system includes the following hardware and software:

[0251] Hardware

[0252] High-resolution facial recognition camera (captures facial image data)

[0253] Surveillance cameras (capture footage within the facility)

[0254] Dedicated terminal (parent's photo input)

[0255] software

[0256] Facial recognition system (analysis and digitization of facial image data)

[0257] Database (storage of digitized facial data)

[0258] Artificial intelligence (matching parental facial data with facial data in the database)

[0259] Emotion engine (recognition and analysis of emotional states)

[0260] Notification system (notifying the location of identified lost children)

[0261] Process Overview

[0262] 1. Acquiring facial image data

[0263] The server captures visitors' facial images in real time from a high-resolution facial recognition camera, allowing detailed facial feature analysis.

[0264] 2. Storage of digitized facial data

[0265] The server sends the acquired facial image data to a facial recognition system and converts it into numerical data, which is then stored in a database along with metadata.

[0266] 3. Parental photo input in case of lost child

[0267] When a child gets lost, the user (facility staff) enters a parent's facial photo into the system using a dedicated terminal at the lost child center. The entered facial photo is sent to the server and digitized.

[0268] 4. Confirming parent-child relationships

[0269] The server uses artificial intelligence to match the facial data stored in the database with the facial data of the parent entered, making it possible to verify whether the parent and child entered the facility together.

[0270] 5. Matching with surveillance camera footage and tracking

[0271] The server receives real-time footage from surveillance cameras installed in each area and matches it with stored facial data of parents to identify lost children.

[0272] 6. Location Notifications

[0273] If a lost child's location is identified, the server notifies the lost child center with the information, including detailed location information and the latest camera footage.

[0274] 7. Emotion Recognition with Emotion Engine

[0275] The server uses an emotion engine to analyze facial expression and voice data acquired through a facial recognition camera and dedicated terminals, and recognizes the emotional state of parents and lost children in real time.

[0276] 8. Supported by Emotion Engine

[0277] The server provides appropriate support based on the analysis results of the emotion engine. For example, if the server determines that a parent is in an extremely unstable state, it will notify facility staff and instruct them on how to provide psychological support and take appropriate action.

[0278] Specific examples

[0279] Let's take a large theme park as an example. Facial recognition cameras installed at the entrance collect facial data of visitors. When a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the surveillance camera footage within the park, and if a specific camera captures a lost child, it notifies the child's location and the parent's emotional state. Using this information, the parent and facility staff can quickly reunite, and the parent can also receive psychological support.

[0280] Example prompts for generative AI models

[0281] "Please explain the specific process flow of a system that uses video data from facial recognition cameras and security cameras to quickly locate children who become lost in large facilities. Please also provide details on the hardware and software used, the data processing method, and the emotional state recognition by the emotion engine."

[0282] This system allows lost children to be quickly reunited with their parents, increasing the safety and sense of security of facility users.The use of an emotion engine also reduces anxiety for parents and children, allowing them to use the facility with peace of mind.

[0283] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0284] Step 1:

[0285] The server acquires face image data in real time from the face recognition camera.

[0286] Specifically, the facial recognition camera photographs the face of a visitor as they pass through the entrance and immediately sends the image data to a server.

[0287] Input: Real-time facial images of visitors passing through the entrance

[0288] Output: Facial image data sent to the server

[0289] Step 2:

[0290] The server passes the received facial image data to a facial authentication system and digitizes it.

[0291] A facial recognition system analyzes facial feature points and encodes them as numerical data.

[0292] Specifically, the facial recognition system analyzes the position and shape of the eyes, nose, and mouth and converts this into numerical data.

[0293] Input: Facial image data stored on the server

[0294] Output: Digitized face data

[0295] Step 3:

[0296] The server stores the digitized facial data in a database.

[0297] When saving, metadata such as date, time, and camera location are also recorded.

[0298] Specifically, the server stores the numerical data in a database and manages the data efficiently.

[0299] Input: Digitized face data, metadata

[0300] Output: Face data stored in a database

[0301] Step 4:

[0302] When a child gets lost, the user (facility staff) takes a photo of the parent's face using a dedicated terminal and enters it into the system.

[0303] The terminal transmits the captured facial photograph data to the server.

[0304] Specifically, the user takes a photo of the parent's face using a dedicated terminal, and the terminal then sends the data to the server.

[0305] Input: Parent's photo taken by facility staff

[0306] Output: Parent face photo data sent to the server

[0307] Step 5:

[0308] The server receives and digitizes the facial photograph data of the parents.

[0309] The facial recognition system encodes the parent's facial data as numerical data.

[0310] Specifically, the server passes the parent's facial photo data to the facial recognition system, which then analyzes it and converts it into numerical data.

[0311] Input: Parent's facial photo data received by the server

[0312] Output: Digitized parent face data

[0313] Step 6:

[0314] The server uses artificial intelligence to compare the facial data of the parent stored in the database with the facial data of the person at the entrance.

[0315] The AI ​​calculates the degree of similarity of facial features and confirms the parent-child relationship.

[0316] Specifically, the server uses an AI matching system to compare the parent's facial data with the facial data in the database and determine the degree of match.

[0317] Input: Digitized parent face data

[0318] Output: Confirmation of parent-child relationship based on the degree of matching of facial feature points

[0319] Step 7:

[0320] The server acquires real-time images from surveillance cameras installed in each area of ​​the facility.

[0321] Each surveillance camera transmits video frames to the server at regular intervals.

[0322] Specifically, the server continuously receives video data transmitted from multiple surveillance cameras.

[0323] Input: Security camera footage from inside the facility

[0324] Output: Real-time video of each area sent to the server

[0325] Step 8:

[0326] The server compares the surveillance camera footage with the stored facial data of the parents to identify the lost child.

[0327] The surveillance camera video frames are analyzed to detect matches between the facial data of the parent and child.

[0328] Specifically, the server uses a video analysis system to compare surveillance camera footage with the parent's facial data to confirm the location of the lost child.

[0329] Input: Surveillance camera footage, digitized parental facial data

[0330] Output: Location information of the lost child as a result of matching with surveillance camera footage

[0331] Step 9:

[0332] If the location of the lost child is determined, the server notifies the lost child center of this information.

[0333] The notification will include detailed location information and the latest camera footage.

[0334] Specifically, the server generates a notification message based on the identified location information and sends it to the lost child center.

[0335] Input: Matched location of lost child

[0336] Output: Location notification sent to the lost child center

[0337] Step 10:

[0338] The server is equipped with an emotion engine that recognizes the emotional state of the parent and the lost child in real time.

[0339] The emotion engine analyzes facial expression and voice data to estimate the emotional state.

[0340] Specifically, the server uses an emotion engine to analyze the emotional state of the parent and the lost child in real time.

[0341] Input: facial expression data, voice data

[0342] Output: Parsed emotional state data

[0343] Step 11:

[0344] The server provides appropriate support based on the emotion data acquired by the emotion engine.

[0345] For example, if a parent is in a very unstable state, the server will notify facility staff and instruct them on psychological support and appropriate measures.

[0346] As a specific operation, the server analyzes the emotion data and notifies the facility staff of the specific support required.

[0347] Input: Emotional state data

[0348] Output: Support instruction notification to facility staff

[0349] (Application example 2)

[0350] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0351] When a child gets lost in a large facility, there is a need to quickly find the child and reunite them with their parents safely and securely. However, current systems take time to locate a lost child, making it difficult to alleviate the anxiety of the parent and child. Furthermore, simply providing location information without considering the emotional state of the parent and child does not provide sufficient support. Therefore, there is a need for a system that can quickly locate a lost child while also providing appropriate support based on the emotional state of the parent and child.

[0352] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0353] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a facial photo of a parent when a child gets lost, means for comparing the facial photo data of the parent with the facial data in the database, means for acquiring footage from multiple surveillance cameras within the facility, means for comparing the surveillance camera footage with the facial photo data of the parent to identify the location of the lost child, means for notifying the location information of the identified lost child, emotion analysis means for recognizing the emotional states of the parent and the lost child, and means for providing support based on the emotional states, thereby enabling the quick discovery of a lost child and support according to the emotional states of the parent and child.

[0354] The "means for acquiring facial image data" refers to a device and method for capturing facial images of visitors using a facial recognition camera installed at the entrance of the facility or in each area.

[0355] The "means for digitizing the facial image data" refers to a process and software for analyzing the acquired facial image and converting facial feature points into digital values.

[0356] The "means for storing the digitized facial data in a database" refers to a system and method for storing facial data converted into digital values ​​in a database and adding metadata such as date, time, and camera position.

[0357] The "means for inputting a parent's facial photograph when a child gets lost" refers to a device and method for taking a parent's facial photograph using a dedicated terminal or smart device and inputting it into the system.

[0358] The "means for matching parent's facial photograph data with the facial data in the database" refers to the process and software for using artificial intelligence to compare and match parent's facial photograph data with the facial data obtained upon entry into the facility.

[0359] The "means for acquiring images from multiple surveillance cameras within a facility" refers to a system and method for transmitting images from surveillance cameras installed in each area within a facility to a server in real time.

[0360] The "means for locating a lost child by comparing the surveillance camera footage with the facial photograph data of the parent" refers to the process and software for comparing and matching the surveillance camera footage with the facial photograph data of the parent to locate a lost child.

[0361] The "means for notifying the location information of the identified lost child" refers to a system and method for notifying the location information of the identified lost child to a lost child center or a smart device.

[0362] The "emotion analysis means for recognizing the emotional states of parents and lost children" is a device and method that uses an emotion engine to analyze facial expression data and voice data acquired and estimate the emotional states of parents and children.

[0363] The "means for providing support based on the emotional state" refers to a system and method for providing appropriate psychological support and responses based on the emotional states of the parent and the lost child.

[0364] The present invention is a system that, when a child gets lost in a large facility, quickly finds the child by utilizing facial recognition and video data from a surveillance camera, and further recognizes the emotional states of the parent and child using an emotion engine, and provides appropriate support. Specific embodiments of the present invention will be described in detail below.

[0365] Acquiring facial image data

[0366] The server captures visitors' facial images in real time through a facial recognition camera installed at the entrance of the facility. This camera has high resolution and can analyze facial feature points in detail. The facial images are converted into digital values ​​and stored in a database.

[0367] Storing digitized facial data

[0368] The server sends the acquired facial image data to a facial recognition system (e.g., Amazon Rekognition, Microsoft® Azure® Face API) and converts it into digital values. This conversion digitizes facial feature points, facilitating subsequent matching. The digitized data is stored in a database using cloud services such as AWS® or Google® Cloud. When the data is saved, metadata such as the date, time, and camera position is also added.

[0369] Parents' photos can be entered when a child gets lost

[0370] When a child gets lost, the user (facility staff) uses a dedicated terminal or smart device to input a parent's facial photo into the system. The parent's facial photo is taken with a high-resolution camera and sent to the server. The server digitizes the parent's facial photo and converts it into encoded data.

[0371] Verifying parent-child relationships

[0372] The server uses artificial intelligence (AI) to compare the facial data stored in the database at the entrance with the facial data of the parent entered, making it possible to confirm with high accuracy whether the parent and child entered the facility together.

[0373] Matching and tracking with surveillance camera footage

[0374] The server captures video in real time from multiple surveillance cameras installed within the facility. Each surveillance camera periodically transmits video frames to the server. The server analyzes the video frames and matches them with the facial data of the parents to identify the lost child.

[0375] Location notification

[0376] If the server locates a lost child, the information is sent to a dedicated terminal or smart device, along with detailed location information and the latest camera footage, helping parents and facility staff quickly reunite the child.

[0377] Emotion recognition by emotion engine

[0378] The server recognizes the emotional state using an emotion engine (e.g., Affectiva Emotion AI, Microsoft Azure Text Analytics). This engine analyzes facial expression and voice data acquired from a facial recognition camera and sensors on the dedicated device to estimate the emotional state of the parent and lost child.

[0379] Supported by an emotional engine

[0380] The server provides appropriate support based on the emotional data acquired by the emotion engine. For example, if a parent is extremely unstable, the server notifies facility staff of this information and instructs them on how to provide psychological support and take appropriate measures.

[0381] Specific examples and prompts for generative AI models

[0382] As a concrete example, consider a large shopping mall. A facial recognition camera is installed at the entrance, and facial data of visitors is collected. When a child gets lost, the parent uses smart glasses to go to the lost child center and input a photo of their face. The server matches the parent and child's faces, and by analyzing the surveillance camera footage, locates the child's location and notifies the smart glasses.

[0383] Example prompts for generative AI models:

[0384] ---

[0385] Please provide a detailed description of a child loss prevention system that uses facial recognition and security cameras. Explain the functionality of the smart glasses application, which compares the facial data of parents and children to quickly identify a child if they get lost and notify the parent of their location and emotional state. Include the specific processing steps and technologies used.

[0386] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0387] Step 1: Acquire facial image data

[0388] The server captures visitors' facial images through high-resolution facial recognition cameras installed at the entrances of the facilities. The input is facial image data, which is then converted into a digital format. Specifically, the server takes the facial image captured by the camera sensor as initial data, and processes the high-quality image through an AI-based facial recognition algorithm.

[0389] Step 2: Digitizing the face data

[0390] The server sends the acquired facial image data to the facial recognition system, where an AI algorithm analyzes the facial feature points and converts them into digital values. The input is the captured facial image, and the output is the digitized facial feature point data. This process uses a high-speed image analysis algorithm and is performed in real time.

[0391] Step 3: Save the digitized data

[0392] The server stores the digitized facial feature data in a cloud database, along with metadata such as date, time, and camera position. The input is the digitized facial feature data and metadata, and the output is a record in the database.

[0393] Step 4: Enter a photo of the parents' faces in case the child gets lost

[0394] The user (facility staff) uses a dedicated terminal or smart device to input a facial photo of the parent of the lost child into the system. The parent's facial photo is taken with a high-resolution camera and sent to the server. The input is the parent's facial image data, and the output is digitized facial feature point data of the parent.

[0395] Step 5: Verify parent-child relationships

[0396] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database at the entrance. The input is the parent's digitized facial data and the facial data in the database, and the output is the parent-child relationship identification result with a high degree of match. In this process, the AI ​​compares facial feature points and determines the parent-child relationship with a high degree of accuracy.

[0397] Step 6: Matching with CCTV footage and tracking

[0398] The server receives real-time video footage from multiple surveillance cameras installed within the facility. It analyzes these videos and matches them with the facial data of the parents to identify lost children. The input is the video frames from each surveillance camera and the parents' digitized facial data, and the output is the identified location information of the lost child.

[0399] Step 7: Location Notification

[0400] When the server identifies the location of a lost child, it notifies the dedicated terminal or smart device of that information. The notification includes the latest camera footage and detailed location information. The input is the location information and video data of the lost child, and the output is a notification message.

[0401] Step 8: Emotion Recognition with the Emotion Engine

[0402] The server uses an emotion engine to recognize the emotional state of the parent and the lost child in real time. The input is facial expression and voice data acquired from a facial recognition camera and sensors on a dedicated device, and the output is an estimated emotional state. This allows for a detailed understanding of the emotional states of the parent and child.

[0403] Step 9: Support from the Emotion Engine

[0404] The server provides appropriate support based on the emotional data acquired by the emotion engine. For example, if a parent is in a very unstable state, the server notifies facility staff of this information and instructs them on how to provide psychological support and appropriate measures for the parent. The input is emotional data, and the output is instructions and provision of support measures.

[0405] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0406] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0407] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0408] [Second embodiment]

[0409] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0410] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0411] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0412] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0413] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0414] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0415] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0416] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0417] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0418] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0419] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0420] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0421] The present invention provides a system for quickly finding a child who has become lost in a large facility by using facial recognition and video data from a surveillance camera. A specific embodiment of the present invention will be described in detail below.

[0422] Acquiring facial image data

[0423] The server captures visitors' facial images in real time from facial recognition cameras installed at the facility entrances. The facial recognition cameras provide high-resolution images, which are necessary for generating accurate digital values ​​in the facial recognition system. The captured facial image data is then digitized and encoded on the spot.

[0424] Storing digitized facial data

[0425] The server stores the encoded facial data in a database, which is used to safely and efficiently manage existing visitors' facial data, allowing for data storage and rapid retrieval.

[0426] Parents' photos can be entered when a child gets lost

[0427] When a child gets lost, the user (facility staff) enters a photo of the parent's face into a dedicated terminal at the lost child center. This photo is then digitized and sent to the server. The server encodes the parent's face data and compares it with the facial data in the database.

[0428] Verifying parent-child relationships

[0429] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database. This process verifies whether the parent and child entered the facility together. The AI ​​checks the degree of similarity of facial features and identifies the parent-child relationship with high accuracy.

[0430] Matching and tracking with surveillance camera footage

[0431] The server acquires video footage from security cameras installed in each area of ​​the facility. The server analyzes each frame of the security camera footage in real time and compares it with the matched parent's facial data. By matching the security camera video data with the parent's facial data, the lost child can be quickly identified.

[0432] Location notification

[0433] If a lost child's location is identified, the server notifies the lost child center, which then provides the specific location information to the parents and facility staff, enabling the child and parent to be reunited quickly and providing peace of mind.

[0434] As a concrete example, consider a large theme park. A facial recognition camera is installed at the entrance, and facial data of visitors is collected when they enter the park. When a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the surveillance camera footage within the park, and if a specific camera captures the lost child, it notifies the child of the child's location. Using this information, the parent and facility staff can quickly reunite the child.

[0435] The above is a specific embodiment of the present invention. This system allows lost children to be quickly reunited with their parents, and increases safety and security for facility users.

[0436] The processing flow will be explained below.

[0437] Step 1:

[0438] The server acquires visitors' facial images in real time from a facial recognition camera installed at the entrance of the facility. The facial images are captured by a high-resolution camera.

[0439] Step 2:

[0440] The server sends the acquired facial image data to the facial recognition system and converts it into digital values, which digitize facial feature points and enable subsequent matching.

[0441] Step 3:

[0442] The server stores the digitized facial data in a database, along with the date, time, and camera position at the time of storage.

[0443] Step 4:

[0444] When a child is lost, the user (facility staff) enters a parent's face photo into the system using a dedicated terminal at the lost child center. The photo is taken using a high-resolution camera.

[0445] Step 5:

[0446] The server digitizes the parent's facial photo and converts it into encoded data, which serves as the basis for matching the lost child.

[0447] Step 6:

[0448] The server uses AI to compare the facial data stored at the facility entrance with the parent's facial data. The AI ​​analyzes the degree of similarity of facial feature points, and if there is a match, it recognizes the parent-child relationship.

[0449] Step 7:

[0450] The server acquires video in real time from surveillance cameras installed in each area of ​​the facility, and each surveillance camera periodically transmits video frames to the server.

[0451] Step 8:

[0452] The server analyzes the video frames from the security cameras and compares them with the stored facial data of the parents, checking whether the facial data matches the specific camera footage.

[0453] Step 9:

[0454] The server identifies the child's location from the surveillance camera footage that matches the parent's facial data, and records the identified location as specific area information.

[0455] Step 10:

[0456] The server then notifies the lost child center of the child's location, including detailed location information and the latest camera footage.

[0457] Step 11:

[0458] The user (facility staff) will then communicate the location information received at the lost child center to the parents, who can then head to the designated area based on the specific location information and quickly reunite with their child.

[0459] This series of processes enables lost children to be quickly reunited with their parents, increasing the safety and security of the facility.

[0460] Example 1

[0461] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0462] When a child gets lost in a large facility, it poses a serious problem for the safety and security of parents and children. With the current system, it is difficult to quickly and accurately find a lost child, which increases the burden on facility staff. In addition, the process of collecting and confirming information when a child gets lost is complicated and time-consuming, so a quick response is required.

[0463] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0464] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a parent's facial photo when a child gets lost, means for digitizing the parent's facial photo data and transmitting it to the server, means for comparing the parent's facial photo data with the facial data in the database using artificial intelligence, means for acquiring footage from multiple surveillance cameras in the facility in real time, means for comparing each frame of the surveillance camera footage with the parent's facial data to identify the location of the lost child, and means for sending a push notification of the identified location of the lost child to the lost child center.This makes it possible to quickly and accurately find a lost child and quickly reunite the parent and child.

[0465] "Facial image data" is digital data that captures the facial features of a visitor.

[0466] "Digitization" is the process of electronically processing analog facial image data to convert it into digital form.

[0467] "Database" means a computer system for storing and managing captured and encoded facial data.

[0468] "Method for inputting a photograph of the parents' faces when a child becomes lost" refers to a method in which facility staff digitally input a photograph of the parents' faces using a dedicated terminal.

[0469] The "means for digitizing facial photograph data and transmitting it to a server" refers to a means for converting a photograph of a face into a digital format and transmitting it to a server.

[0470] "Artificial intelligence" refers to machine learning and deep learning models that perform complex processes such as facial recognition and data matching.

[0471] "Means for acquiring surveillance camera footage in real time" refers to means for continuously and immediately acquiring footage from multiple surveillance cameras installed within a facility.

[0472] The "means for collating each frame of surveillance camera video with parent's facial data" refers to a means for comparing each frame of surveillance camera video acquired in real time with parent's facial data.

[0473] The "means for identifying the location of a lost child" is a means for identifying the location of a lost child based on the location information of surveillance camera footage that matches the facial data of the parent.

[0474] "Push notification" is a communication method for proactively sending the location information of a identified lost child to the lost child center.

[0475] This invention is a system that uses facial recognition and video data from surveillance cameras to quickly find a child who has become lost in a large facility. Specific embodiments of this system will be described in detail below, including the hardware and software used to process and calculate data.

[0476] The server captures visitors' facial images in real time using a high-resolution facial recognition camera (e.g., a typical high-performance camera) installed at the entrance to the facility. The facial recognition camera analyzes facial features and encodes them as digital data. This data is temporarily stored in the server's memory. The server then stores this encoded facial data in a database. This database can be built using MySQL, for example.

[0477] When a child gets lost, the user (facility staff) uses a dedicated terminal at the lost child center to input a parent's facial photo. This terminal digitizes the parent's facial photo and sends the image data to a server. In this transmission process, the image captured by the terminal is transferred to the server via a network. The server then encodes the received parent's facial photo and stores it in a database.

[0478] The server uses artificial intelligence (AI models, such as Amazon Rekognition) to compare the facial data stored in the database with the parent's facial photo data. The AI ​​model evaluates the degree of match of facial features, and if a high match is detected, the parent-child relationship is identified.

[0479] The server captures and analyzes video footage from surveillance cameras (e.g., standard surveillance cameras) installed in each area of ​​the facility in real time. The server then applies a facial recognition algorithm to each frame of video footage from the security cameras and compares it with the parent's facial data. If a match is confirmed, the server records the location information of that frame and identifies the lost child.

[0480] Once the location of a lost child is identified, the server sends the information to the lost child center as a push notification. The lost child center's device receives this information and displays the location information to facility staff and parents. For example, by notifying them that the lost child is in a specific area, it helps to quickly reunite the child.

[0481] As a concrete example, imagine a large theme park. When visitors enter the park, a facial recognition camera at the entrance captures and stores their facial data. If a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the footage from the facility's security cameras, and if a specific security camera captures the lost child, it notifies them of the child's location. Based on this information, the parents and facility staff can quickly reunite the child.

[0482] This system will enable lost children to be quickly reunited with their parents, improving safety and peace of mind for facility users.

[0483] Example prompts to input to a generative AI model:

[0484] "Please explain how the system works by inputting a photo of the parent's face when a child goes missing and matching it with the security camera footage within the facility to locate the child."

[0485] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0486] Step 1: Acquire facial image data

[0487] The server captures visitors' facial images in real time using a high-resolution facial recognition camera installed at the entrance of the facility. The input data is the video stream from the camera. The server extracts individual facial images from this stream and encodes them into digital data. The converted output data is temporarily stored in the server's memory.

[0488] Specific behavior:

[0489] The server receives the video stream transmitted from the camera in real time.

[0490] The server extracts facial images from the video stream using a facial recognition algorithm.

[0491] The server encodes the extracted facial image and converts it into digital data.

[0492] Step 2: Save the digitized face data

[0493] The server stores the encoded facial data in a database. The input data for this step is the digitized facial data acquired in step 1. The server performs error checks before storing to ensure data integrity. Once integrity is confirmed, the data is stored in the database.

[0494] Specific behavior:

[0495] The server performs error checking on the encoded face data.

[0496] The server generates SQL queries to store the digitized face data in a database.

[0497] The server sends an SQL query to the database and receives a response confirming the save.

[0498] Step 3: Enter a photo of the parents' faces in case the child gets lost

[0499] When a child gets lost, the user (facility staff) uses a dedicated terminal at the lost child center to input a parent's facial photo. The input data for this step is the parent's facial photo. The terminal digitizes the input facial photo and sends the image data to the server. The server then encodes the received facial photo and stores it in a database.

[0500] Specific behavior:

[0501] The user (facility staff) takes a photo of the parent's face using a dedicated terminal.

[0502] The terminal digitizes the captured facial photograph.

[0503] The terminal transmits the digitized facial photograph to the server.

[0504] The server encodes the received parent's facial photograph and stores it in a database.

[0505] Step 4: Verify parent-child relationships

[0506] The server uses an artificial intelligence model to match the facial data of the parent entered with the facial data stored in the database at the entrance. The input data for this step is the facial data of the parent and the facial data acquired at the entrance. The server provides the input data to the AI ​​model and evaluates the degree of match of facial features. If a high match is detected, the server identifies the parent-child relationship.

[0507] Specific behavior:

[0508] The server inputs the parents' facial data into the AI ​​model.

[0509] The AI ​​model compares the parent's facial data with multiple facial data stored in the entrance database.

[0510] The server receives the match assessment from the AI ​​model and identifies parent-child relationships.

[0511] Step 5: Matching with CCTV footage and tracking

[0512] The server captures and analyzes real-time video from security cameras installed in each area of ​​the facility. The input data for this step are the video stream from the security cameras and the parent's facial data. The server applies a facial recognition algorithm to each frame from the security cameras and compares it with the parent's facial data. If a match is confirmed, the server records the location information of that frame.

[0513] Specific behavior:

[0514] The server receives the video streams from each surveillance camera.

[0515] The server applies a facial recognition algorithm to each video frame.

[0516] The server records the position information of the frame where a match is confirmed by comparing it with the parent's face data.

[0517] Step 6: Location Notification

[0518] Once the location of the lost child is identified, the server pushes the information to the lost child center. The input data for this step is the location information of the identified lost child. The server converts the location information into an explainable format and sends it to the lost child center's terminal. The lost child center's terminal receives the notification and displays the location information to facility staff and parents.

[0519] Specific behavior:

[0520] The server converts the identified location information of the lost child into an explainable format.

[0521] The server sends the location information to the lost child center's terminal as a push notification.

[0522] The terminal at the lost child center displays the location information to facility staff and parents based on the received location information.

[0523] (Application example 1)

[0524] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0525] When a child gets lost in a large facility, a system that can quickly and accurately locate the child is extremely important. However, with existing systems, it takes time to identify the lost child, causing anxiety and stress for parents and facility staff. Furthermore, there is a lack of a way for parents to directly obtain information about the lost child's location, further extending the time until the child is reunited. Furthermore, information sharing with lost child centers and security guards is often not smooth. A system that can solve these problems and respond quickly and efficiently when a child gets lost is needed.

[0526] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0527] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a parent's facial photograph when a child gets lost, means for comparing the parent's facial photograph data with the facial data in the database, means for acquiring footage from multiple surveillance cameras within the facility, means for comparing the surveillance camera footage with the parent's facial photograph data to identify the location of the lost child, means for notifying the parent's mobile device of the identified location information of the lost child, and means for sharing the location information notified to the parent's mobile device with the facility's lost child center and security guard terminals in real time. This allows parents to quickly determine the child's location and share the information with the lost child center and security guard, enabling a prompt reunion.

[0528] "Facial image data" refers to facial image information captured by a facial recognition camera or other camera device.

[0529] "Digitalized facial data" refers to facial image data that has been digitally digitized and converted into a form that is easy for a computer to handle.

[0530] A "database" refers to a system that efficiently manages accumulated digital data and quickly searches and obtains necessary information.

[0531] "Parent's face photo" refers to a photo of the face provided by the parent when the child gets lost. This photo is digitized and sent to the server.

[0532] "Matching" refers to the process of comparing one piece of data with another to see if they match.

[0533] A "surveillance camera" refers to a camera installed within a facility to monitor and record the movements of people and objects in real time.

[0534] "Location information of lost children" refers to information showing the current location of a lost child. This information is obtained by comparing video data from surveillance cameras with facial data.

[0535] "Parent's mobile device" refers to a mobile information device such as a smartphone or tablet owned by the parent. This device will be notified of the lost child's location.

[0536] A "lost child center" is a department located within a large facility that specializes in dealing with lost children.

[0537] "Security guard devices" refer to the mobile information devices owned by security guards at the facility. The location information of lost children is also shared with these devices.

[0538] This invention is a system for quickly reuniting parents and children when they get lost in a large facility. This system is composed of a facial recognition camera, a surveillance camera, a parent's mobile device, a lost child center, and a security guard's terminal.

[0539] The server first uses a facial recognition camera installed at the entrance of the facility to obtain facial image data. The facial recognition camera provides high-resolution images that allow the facial recognition system to generate accurate digital values. The facial image data is digitized and transmitted to the server in real time.

[0540] The server stores the captured digitized facial data in a database designed to securely and efficiently manage visitor facial data and enable rapid search and analysis.

[0541] When a child gets lost, the parent enters a facial photo via a dedicated terminal at the lost child center. This photo is then digitized and sent to a server. The server encodes the parent's facial photo data and compares it with facial data in a database. This identifies the parent-child relationship and confirms that the child has entered the facility.

[0542] The server captures and analyzes video footage from multiple surveillance cameras installed within the facility in real time. By comparing the video data from the surveillance cameras with facial photos of the parents, the location of the lost child can be identified. AI technology is used to confirm the degree of match of facial features with high accuracy.

[0543] When a lost child's location is identified, the server notifies the parent's mobile device of the location information. The location information is also shared in real time with the lost child center and security guards' devices. This function allows parents to quickly determine the child's location, and the lost child center and security guards can take action at the same time.

[0544] As an example of how this system can be put into practical use, consider a large shopping mall. As a parent and child pass through the entrance, facial image data is captured by a facial recognition camera. If a child gets lost, the parent provides a facial photo to the lost child center, and the data is collated on a server. After that, an analysis of the surveillance camera identifies the lost child as being in the food court, and the parent's smartphone is notified of the child's location. At the same time, this information is shared with the lost child center and security guards, allowing the parent and child to be quickly reunited.

[0545] An example of an input prompt for a generative AI model is:

[0546] "Please create a system to quickly find children who get lost in a shopping mall. The system will use facial recognition cameras and surveillance cameras to capture and match the facial data of parents and children. Please also implement a function to notify the parents' smartphones of the child's location when the child gets lost."

[0547] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0548] Step 1:

[0549] The server acquires the visitor's facial image data in real time using a facial recognition camera installed at the facility entrance. The facial recognition camera provides high-resolution images and generates accurate digital values ​​in the facial recognition system. The server acquires the facial image data as input and obtains the digitized facial data as output.

[0550] Step 2:

[0551] The server digitizes and encodes the acquired facial image data, converting it into a digital format that is easy for a computer to handle. It receives facial image data as input and obtains encoded facial data as output.

[0552] Step 3:

[0553] The server stores the encoded facial data in a database designed to efficiently manage visitor facial data and enable rapid search and analysis. The server receives digitized facial data as input and stores it in the database.

[0554] Step 4:

[0555] When a child gets lost, the user (parent) enters a facial photo through a dedicated terminal at the lost child center. The parent's facial photo is digitized on the spot and sent to the server. The parent's facial photo is obtained as input, and the digitized parent's facial photo data is obtained as output.

[0556] Step 5:

[0557] The server encodes the parent's facial photo data and matches it with facial data in the database. This process identifies the parent-child relationship and verifies whether the parent and child entered the facility together. The server receives digitized parent facial data as input and provides information on the degree of match between the facial data as output.

[0558] Step 6:

[0559] The server acquires video footage from multiple surveillance cameras installed within the facility in real time. The video data from the surveillance cameras is used to match the parent's facial data. The surveillance camera video is received as input, and the video data for analysis is obtained as output.

[0560] Step 7:

[0561] The server analyzes the surveillance camera footage and compares it with the parent's facial data. This analysis is performed using AI technology to confirm the degree of match of facial features with high accuracy. The server receives the surveillance camera footage and the parent's facial data as input, and obtains specific information about the matching parts as output.

[0562] Step 8:

[0563] The server identifies the location of the lost child and notifies the parent's mobile device of the location information. At the same time, the server also shares the location information with the lost child center and security guards' devices in real time. The server receives the matching part of the location information as input and outputs the location information notification.

[0564] Step 9:

[0565] The parents check the location information of their lost child that is notified to their mobile device, and then use that information to work with facility staff to find the lost child.The location information notification is received as input, and the actual on-site confirmation and response action are output.

[0566] The above are the specific processing steps of the system that realizes the application example.

[0567] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0568] This invention is a system that, when a child gets lost in a large facility, quickly finds the child using facial recognition and video data from a surveillance camera, and further recognizes the emotional states of the parent and child using an emotion engine, and provides appropriate support. Specific embodiments of the present invention will be described in detail below.

[0569] Acquiring facial image data

[0570] The server acquires visitors' facial images in real time from a facial recognition camera installed at the entrance of the facility. The facial images are captured by a high-resolution camera, which allows for detailed analysis of facial feature points.

[0571] Storing digitized facial data

[0572] The server sends the acquired facial image data to the facial recognition system and converts it into digital values. This conversion quantifies facial feature points, enabling subsequent matching. The digitized data is stored in a database and managed along with metadata such as date, time, and camera position.

[0573] Parents' photos can be entered when a child gets lost

[0574] When a child gets lost, the user (facility staff) enters a parent's facial photo into the system via a dedicated terminal at the lost child center. The photo is taken using a high-resolution camera and sent to the server. The server digitizes the parent's facial photo and converts it into encoded data.

[0575] Verifying parent-child relationships

[0576] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database. This process verifies whether the parent and child entered the facility together. The AI ​​checks the degree of similarity of facial features and identifies the parent-child relationship with high accuracy.

[0577] Matching and tracking with surveillance camera footage

[0578] The server acquires video in real time from surveillance cameras installed in each area of ​​the facility. Each surveillance camera periodically transmits video frames to the server. The server analyzes the video frames from the surveillance cameras and compares them with the stored facial data of the parents. By comparing the video data from the surveillance cameras with the facial data of the parents, lost children can be quickly identified.

[0579] Location notification

[0580] If a lost child's location is identified, the server notifies the lost child center with the location information, including the child's detailed location and the latest camera footage, providing parents and facility staff with specific location information and facilitating a speedy reunion.

[0581] Emotion recognition by emotion engine

[0582] The server is equipped with an emotion engine that recognizes the emotional state of parents and lost children in real time. The emotion engine analyzes facial expression and voice data acquired through a facial recognition camera and sensors on the dedicated device to estimate the emotional state. For example, if a parent is feeling stressed, the emotion engine will notify the server based on that information.

[0583] Supported by an emotional engine

[0584] The server provides appropriate support based on the emotional data acquired by the emotion engine. For example, if a parent is in a very unstable state, the server notifies the facility staff of this information and instructs them on how to provide psychological support and take appropriate measures. This reduces the anxiety of both parents and children and supports rapid reunification.

[0585] As a concrete example, consider a large theme park. A facial recognition camera is installed at the entrance, and facial data of visitors is collected when they enter the park. When a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the surveillance camera footage within the park, and if a specific camera captures the lost child, it notifies the location information and the parent's emotional state. Based on this information, the parent and facility staff can quickly reunite, and at the same time, the parent can receive appropriate support according to their emotional state.

[0586] The above is a specific embodiment of the present invention. This system allows lost children to be quickly reunited with their parents, enhancing safety and a sense of security for facility users. Furthermore, the use of an emotion engine reduces anxiety for parents and children, supporting a safe and secure facility experience.

[0587] The processing flow will be explained below.

[0588] Step 1:

[0589] The server acquires visitors' facial images in real time from a facial recognition camera installed at the entrance of the facility. The facial images are captured by a high-resolution camera.

[0590] Step 2:

[0591] The server sends the acquired facial image data to the facial recognition system and converts it into digital values. By quantifying facial features in detail, subsequent matching can be performed.

[0592] Step 3:

[0593] The server stores the digitized facial data in a database, along with metadata such as date, time, and camera position.

[0594] Step 4:

[0595] When a child gets lost, the user (facility staff) enters a photo of the parent's face into the system via a dedicated terminal at the lost child center. The photo is taken using a high-resolution camera.

[0596] Step 5:

[0597] The server digitizes the parent's facial photo data and converts it into encoded data, which is used as the basis for matching lost children.

[0598] Step 6:

[0599] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database. The AI ​​analyzes the degree of similarity of facial features and, if there is a match, recognizes the parent-child relationship.

[0600] Step 7:

[0601] The server acquires video in real time from surveillance cameras installed in each area of ​​the facility, and each surveillance camera periodically transmits video frames to the server.

[0602] Step 8:

[0603] The server analyzes the video frames from the security cameras and compares them with the stored facial data of the parents, checking whether the facial data matches the specific camera footage.

[0604] Step 9:

[0605] The server identifies the child's location from the surveillance camera footage that matches the parent's facial data, and records the identified location as specific area information.

[0606] Step 10:

[0607] The server then notifies the lost child center of the location of the identified lost child, including detailed location information and the latest camera footage.

[0608] Step 11:

[0609] The user (facility staff) will then communicate the location information received at the lost child center to the parents, who can then head to the designated area based on the specific location information and quickly reunite with their child.

[0610] Step 12:

[0611] The server activates an emotion engine that recognizes the emotional state of the parent and the lost child in real time. The emotion engine analyzes facial expression and voice data acquired through a facial recognition camera and sensors on the dedicated device to estimate the emotional state.

[0612] Step 13:

[0613] The server provides appropriate support based on the parent's emotional data analyzed by the emotion engine. For example, if the parent is feeling stressed, the server notifies facility staff of this information and instructs them on how to provide psychological support and take appropriate measures.

[0614] Step 14:

[0615] Based on the analysis results of the emotion engine, the user (facility staff) communicates with the parents in a way that gives them a sense of security, thereby reducing their anxiety and supporting a speedy reunion.

[0616] This series of processes not only enables lost children to be quickly reunited with their parents, but also reduces the anxiety of both parents and children and increases the sense of safety and security for facility users.

[0617] Example 2

[0618] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0619] Having a child get lost in a large facility is a major source of anxiety for parents. It is also extremely important for facility staff to quickly and accurately locate lost children, but conventional methods often require a lot of time and effort. Furthermore, understanding the emotional state of lost children and their parents and providing appropriate support has also been a challenge. Therefore, the objective of this invention is to provide a system that uses facial recognition technology and surveillance camera footage to quickly identify lost children and recognize and respond to the emotional state of parents and children.

[0620] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0621] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a parent's facial photo when a child gets lost, means for using artificial intelligence to compare the parent's facial photo data with the facial data in the database, means for acquiring footage from multiple surveillance cameras within the facility, means for locating the lost child by comparing the surveillance camera footage with the parent's facial photo data, means for notifying the location of the located lost child, means including an emotion engine for recognizing the emotional states of the parent and the lost child in real time, and means for providing appropriate support based on the emotion data acquired by the emotion engine. This enables rapid location identification of the lost child and appropriate support taking into account the emotional states of the parent and child.

[0622] "Means for acquiring facial image data" refers to devices or technologies for capturing high-resolution images of visitors' faces, and specifically includes facial recognition cameras.

[0623] "Means for digitizing facial image data" refers to software or systems for analyzing acquired facial images and converting them into numerical data.

[0624] "Means for storing digitized facial data in a database" refers to technologies or systems that securely and effectively store facial data converted into a numerical data format in a database along with certain metadata.

[0625] "Means for inputting a parent's facial photograph when a child gets lost" refers to a device or method for inputting and transmitting a parent's facial photograph into the system via a dedicated terminal.

[0626] "Means using artificial intelligence to match parent facial photograph data with facial data in the database" refers to a matching system that uses AI technology to compare parent facial photographs with facial data in the database and determine the degree of match with a high degree of accuracy.

[0627] "Means for acquiring footage from multiple surveillance cameras within a facility" refers to systems and technologies for collecting real-time video data from multiple surveillance cameras installed within a facility.

[0628] "Means for locating a lost child by comparing surveillance camera footage with facial photographic data of parents" refers to technology or systems for locating a lost child by comparing surveillance camera footage with facial photographic data of parents.

[0629] "Means for notifying the location of identified lost children" refers to a notification system that quickly notifies facility personnel of the precise location of identified lost children.

[0630] "Means including an emotion engine for recognizing the emotional states of the parent and the lost child in real time" refers to an emotion recognition engine for analyzing facial expressions and voice data and detecting the emotional states of the parent and the lost child in real time.

[0631] "Means for providing appropriate support based on emotional data obtained by the emotion engine" refers to a support system that uses data from the emotion recognition engine to give instructions to facility staff and suggest specific countermeasures.

[0632] This invention is a system that, when a child gets lost in a large facility, quickly finds the child using facial recognition and video data from a surveillance camera, and further recognizes the emotional states of the parent and child using an emotion engine, and provides appropriate support. Specific embodiments are described below.

[0633] System configuration

[0634] The system includes the following hardware and software:

[0635] Hardware

[0636] High-resolution facial recognition camera (captures facial image data)

[0637] Surveillance cameras (capture footage within the facility)

[0638] Dedicated terminal (parent's photo input)

[0639] software

[0640] Facial recognition system (analysis and digitization of facial image data)

[0641] Database (storage of digitized facial data)

[0642] Artificial intelligence (matching parental facial data with facial data in the database)

[0643] Emotion engine (recognition and analysis of emotional states)

[0644] Notification system (notifying the location of identified lost children)

[0645] Process Overview

[0646] 1. Acquiring facial image data

[0647] The server captures visitors' facial images in real time from a high-resolution facial recognition camera, allowing detailed facial feature analysis.

[0648] 2. Storage of digitized facial data

[0649] The server sends the acquired facial image data to a facial recognition system and converts it into numerical data, which is then stored in a database along with metadata.

[0650] 3. Parental photo input in case of lost child

[0651] When a child gets lost, the user (facility staff) enters a parent's facial photo into the system using a dedicated terminal at the lost child center. The entered facial photo is sent to the server and digitized.

[0652] 4. Confirming parent-child relationships

[0653] The server uses artificial intelligence to match the facial data stored in the database with the facial data of the parent entered, making it possible to verify whether the parent and child entered the facility together.

[0654] 5. Matching with surveillance camera footage and tracking

[0655] The server receives real-time footage from surveillance cameras installed in each area and matches it with stored facial data of parents to identify lost children.

[0656] 6. Location Notifications

[0657] If a lost child's location is identified, the server notifies the lost child center with the information, including detailed location information and the latest camera footage.

[0658] 7. Emotion Recognition with Emotion Engine

[0659] The server uses an emotion engine to analyze facial expression and voice data acquired through a facial recognition camera and dedicated terminals, and recognizes the emotional state of parents and lost children in real time.

[0660] 8. Supported by Emotion Engine

[0661] The server provides appropriate support based on the analysis results of the emotion engine. For example, if the server determines that a parent is in an extremely unstable state, it will notify facility staff and instruct them on how to provide psychological support and take appropriate action.

[0662] Specific examples

[0663] Let's take a large theme park as an example. Facial recognition cameras installed at the entrance collect facial data of visitors. When a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the surveillance camera footage within the park, and if a specific camera captures a lost child, it notifies the child's location and the parent's emotional state. Using this information, the parent and facility staff can quickly reunite, and the parent can also receive psychological support.

[0664] Example prompts for generative AI models

[0665] "Please explain the specific process flow of a system that uses video data from facial recognition cameras and security cameras to quickly locate children who become lost in large facilities. Please also provide details on the hardware and software used, the data processing method, and the emotional state recognition by the emotion engine."

[0666] This system allows lost children to be quickly reunited with their parents, increasing the safety and sense of security of facility users.The use of an emotion engine also reduces anxiety for parents and children, allowing them to use the facility with peace of mind.

[0667] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0668] Step 1:

[0669] The server acquires face image data in real time from the face recognition camera.

[0670] Specifically, the facial recognition camera photographs the face of a visitor as they pass through the entrance and immediately sends the image data to a server.

[0671] Input: Real-time facial images of visitors passing through the entrance

[0672] Output: Facial image data sent to the server

[0673] Step 2:

[0674] The server passes the received facial image data to a facial authentication system and digitizes it.

[0675] A facial recognition system analyzes facial feature points and encodes them as numerical data.

[0676] Specifically, the facial recognition system analyzes the position and shape of the eyes, nose, and mouth and converts this into numerical data.

[0677] Input: Facial image data stored on the server

[0678] Output: Digitized face data

[0679] Step 3:

[0680] The server stores the digitized facial data in a database.

[0681] When saving, metadata such as date, time, and camera location are also recorded.

[0682] Specifically, the server stores the numerical data in a database and manages the data efficiently.

[0683] Input: Digitized face data, metadata

[0684] Output: Face data stored in a database

[0685] Step 4:

[0686] When a child gets lost, the user (facility staff) takes a photo of the parent's face using a dedicated terminal and enters it into the system.

[0687] The terminal transmits the captured facial photograph data to the server.

[0688] Specifically, the user takes a photo of the parent's face using a dedicated terminal, and the terminal then sends the data to the server.

[0689] Input: Parent's photo taken by facility staff

[0690] Output: Parent face photo data sent to the server

[0691] Step 5:

[0692] The server receives and digitizes the facial photograph data of the parents.

[0693] The facial recognition system encodes the parent's facial data as numerical data.

[0694] Specifically, the server passes the parent's facial photo data to the facial recognition system, which then analyzes it and converts it into numerical data.

[0695] Input: Parent's facial photo data received by the server

[0696] Output: Digitized parent face data

[0697] Step 6:

[0698] The server uses artificial intelligence to compare the facial data of the parent stored in the database with the facial data of the person at the entrance.

[0699] The AI ​​calculates the degree of similarity of facial features and confirms the parent-child relationship.

[0700] Specifically, the server uses an AI matching system to compare the parent's facial data with the facial data in the database and determine the degree of match.

[0701] Input: Digitized parent face data

[0702] Output: Confirmation of parent-child relationship based on the degree of matching of facial feature points

[0703] Step 7:

[0704] The server acquires real-time images from surveillance cameras installed in each area of ​​the facility.

[0705] Each surveillance camera transmits video frames to the server at regular intervals.

[0706] Specifically, the server continuously receives video data transmitted from multiple surveillance cameras.

[0707] Input: Security camera footage from inside the facility

[0708] Output: Real-time video of each area sent to the server

[0709] Step 8:

[0710] The server compares the surveillance camera footage with the stored facial data of the parents to identify the lost child.

[0711] The surveillance camera video frames are analyzed to detect matches between the facial data of the parent and child.

[0712] Specifically, the server uses a video analysis system to compare surveillance camera footage with the parent's facial data to confirm the location of the lost child.

[0713] Input: Surveillance camera footage, digitized parental facial data

[0714] Output: Location information of the lost child as a result of matching with surveillance camera footage

[0715] Step 9:

[0716] If the location of the lost child is determined, the server notifies the lost child center of this information.

[0717] The notification will include detailed location information and the latest camera footage.

[0718] Specifically, the server generates a notification message based on the identified location information and sends it to the lost child center.

[0719] Input: Matched location of lost child

[0720] Output: Location notification sent to the lost child center

[0721] Step 10:

[0722] The server is equipped with an emotion engine that recognizes the emotional state of the parent and the lost child in real time.

[0723] The emotion engine analyzes facial expression and voice data to estimate the emotional state.

[0724] Specifically, the server uses an emotion engine to analyze the emotional state of the parent and the lost child in real time.

[0725] Input: facial expression data, voice data

[0726] Output: Parsed emotional state data

[0727] Step 11:

[0728] The server provides appropriate support based on the emotion data acquired by the emotion engine.

[0729] For example, if a parent is in a very unstable state, the server will notify facility staff and instruct them on psychological support and appropriate measures.

[0730] As a specific operation, the server analyzes the emotion data and notifies the facility staff of the specific support required.

[0731] Input: Emotional state data

[0732] Output: Support instruction notification to facility staff

[0733] (Application example 2)

[0734] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0735] When a child gets lost in a large facility, there is a need to quickly find the child and reunite them with their parents safely and securely. However, current systems take time to locate a lost child, making it difficult to alleviate the anxiety of the parent and child. Furthermore, simply providing location information without considering the emotional state of the parent and child does not provide sufficient support. Therefore, there is a need for a system that can quickly locate a lost child while also providing appropriate support based on the emotional state of the parent and child.

[0736] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0737] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a facial photo of a parent when a child gets lost, means for comparing the facial photo data of the parent with the facial data in the database, means for acquiring footage from multiple surveillance cameras within the facility, means for comparing the surveillance camera footage with the facial photo data of the parent to identify the location of the lost child, means for notifying the location information of the identified lost child, emotion analysis means for recognizing the emotional states of the parent and the lost child, and means for providing support based on the emotional states, thereby enabling the quick discovery of a lost child and support according to the emotional states of the parent and child.

[0738] The "means for acquiring facial image data" refers to a device and method for capturing facial images of visitors using a facial recognition camera installed at the entrance of the facility or in each area.

[0739] The "means for digitizing the facial image data" refers to a process and software for analyzing the acquired facial image and converting facial feature points into digital values.

[0740] The "means for storing the digitized facial data in a database" refers to a system and method for storing facial data converted into digital values ​​in a database and adding metadata such as date, time, and camera position.

[0741] The "means for inputting a parent's facial photograph when a child gets lost" refers to a device and method for taking a parent's facial photograph using a dedicated terminal or smart device and inputting it into the system.

[0742] The "means for matching parent's facial photograph data with the facial data in the database" refers to the process and software for using artificial intelligence to compare and match parent's facial photograph data with the facial data obtained upon entry into the facility.

[0743] The "means for acquiring images from multiple surveillance cameras within a facility" refers to a system and method for transmitting images from surveillance cameras installed in each area within a facility to a server in real time.

[0744] The "means for locating a lost child by comparing the surveillance camera footage with the facial photograph data of the parent" refers to the process and software for comparing and matching the surveillance camera footage with the facial photograph data of the parent to locate a lost child.

[0745] The "means for notifying the location information of the identified lost child" refers to a system and method for notifying the location information of the identified lost child to a lost child center or a smart device.

[0746] The "emotion analysis means for recognizing the emotional states of parents and lost children" is a device and method that uses an emotion engine to analyze facial expression data and voice data acquired and estimate the emotional states of parents and children.

[0747] The "means for providing support based on the emotional state" refers to a system and method for providing appropriate psychological support and responses based on the emotional states of the parent and the lost child.

[0748] The present invention is a system that, when a child gets lost in a large facility, quickly finds the child by utilizing facial recognition and video data from a surveillance camera, and further recognizes the emotional states of the parent and child using an emotion engine, and provides appropriate support. Specific embodiments of the present invention will be described in detail below.

[0749] Acquiring facial image data

[0750] The server captures visitors' facial images in real time through a facial recognition camera installed at the entrance of the facility. This camera has high resolution and can analyze facial feature points in detail. The facial images are converted into digital values ​​and stored in a database.

[0751] Storing digitized facial data

[0752] The server sends the acquired facial image data to a facial recognition system (e.g., Amazon Rekognition, Microsoft Azure Face API) and converts it into digital values. This conversion digitizes facial feature points, facilitating subsequent matching. The digitized data is stored in a database using cloud services such as AWS or Google Cloud. When the data is saved, metadata such as the date, time, and camera position is also added.

[0753] Parents' photos can be entered when a child gets lost

[0754] When a child gets lost, the user (facility staff) uses a dedicated terminal or smart device to input a parent's facial photo into the system. The parent's facial photo is taken with a high-resolution camera and sent to the server. The server digitizes the parent's facial photo and converts it into encoded data.

[0755] Verifying parent-child relationships

[0756] The server uses artificial intelligence (AI) to compare the facial data stored in the database at the entrance with the facial data of the parent entered, making it possible to confirm with high accuracy whether the parent and child entered the facility together.

[0757] Matching and tracking with surveillance camera footage

[0758] The server captures video in real time from multiple surveillance cameras installed within the facility. Each surveillance camera periodically transmits video frames to the server. The server analyzes the video frames and matches them with the facial data of the parents to identify the lost child.

[0759] Location notification

[0760] If the server locates a lost child, the information is sent to a dedicated terminal or smart device, along with detailed location information and the latest camera footage, helping parents and facility staff quickly reunite the child.

[0761] Emotion recognition by emotion engine

[0762] The server recognizes the emotional state using an emotion engine (e.g., Affectiva Emotion AI, Microsoft Azure Text Analytics). This engine analyzes facial expression and voice data acquired from a facial recognition camera and sensors on the dedicated device to estimate the emotional state of the parent and lost child.

[0763] Supported by an emotional engine

[0764] The server provides appropriate support based on the emotional data acquired by the emotion engine. For example, if a parent is extremely unstable, the server notifies facility staff of this information and instructs them on how to provide psychological support and take appropriate measures.

[0765] Specific examples and prompts for generative AI models

[0766] As a concrete example, consider a large shopping mall. A facial recognition camera is installed at the entrance, and facial data of visitors is collected. When a child gets lost, the parent uses smart glasses to go to the lost child center and input a photo of their face. The server matches the parent and child's faces, and by analyzing the surveillance camera footage, locates the child's location and notifies the smart glasses.

[0767] Example prompts for generative AI models:

[0768] ---

[0769] Please provide a detailed description of a child loss prevention system that uses facial recognition and security cameras. Explain the functionality of the smart glasses application, which compares the facial data of parents and children to quickly identify a child if they get lost and notify the parent of their location and emotional state. Include the specific processing steps and technologies used.

[0770] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0771] Step 1: Acquire facial image data

[0772] The server captures visitors' facial images through high-resolution facial recognition cameras installed at the entrances of the facilities. The input is facial image data, which is then converted into a digital format. Specifically, the server takes the facial image captured by the camera sensor as initial data, and processes the high-quality image through an AI-based facial recognition algorithm.

[0773] Step 2: Digitizing the face data

[0774] The server sends the acquired facial image data to the facial recognition system, where an AI algorithm analyzes the facial feature points and converts them into digital values. The input is the captured facial image, and the output is the digitized facial feature point data. This process uses a high-speed image analysis algorithm and is performed in real time.

[0775] Step 3: Save the digitized data

[0776] The server stores the digitized facial feature data in a cloud database, along with metadata such as date, time, and camera position. The input is the digitized facial feature data and metadata, and the output is a record in the database.

[0777] Step 4: Enter a photo of the parents' faces in case the child gets lost

[0778] The user (facility staff) uses a dedicated terminal or smart device to input a facial photo of the parent of the lost child into the system. The parent's facial photo is taken with a high-resolution camera and sent to the server. The input is the parent's facial image data, and the output is digitized facial feature point data of the parent.

[0779] Step 5: Verify parent-child relationships

[0780] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database at the entrance. The input is the parent's digitized facial data and the facial data in the database, and the output is the parent-child relationship identification result with a high degree of match. In this process, the AI ​​compares facial feature points and determines the parent-child relationship with a high degree of accuracy.

[0781] Step 6: Matching with CCTV footage and tracking

[0782] The server receives real-time video footage from multiple surveillance cameras installed within the facility. It analyzes these videos and matches them with the facial data of the parents to identify lost children. The input is the video frames from each surveillance camera and the parents' digitized facial data, and the output is the identified location information of the lost child.

[0783] Step 7: Location Notification

[0784] When the server identifies the location of a lost child, it notifies the dedicated terminal or smart device of that information. The notification includes the latest camera footage and detailed location information. The input is the location information and video data of the lost child, and the output is a notification message.

[0785] Step 8: Emotion Recognition with the Emotion Engine

[0786] The server uses an emotion engine to recognize the emotional state of the parent and the lost child in real time. The input is facial expression and voice data acquired from a facial recognition camera and sensors on a dedicated device, and the output is an estimated emotional state. This allows for a detailed understanding of the emotional states of the parent and child.

[0787] Step 9: Support from the Emotion Engine

[0788] The server provides appropriate support based on the emotional data acquired by the emotion engine. For example, if a parent is in a very unstable state, the server notifies facility staff of this information and instructs them on how to provide psychological support and appropriate measures for the parent. The input is emotional data, and the output is instructions and provision of support measures.

[0789] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0790] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0791] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0792] [Third embodiment]

[0793] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0794] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0795] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0796] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0797] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0798] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0799] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0800] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0801] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0802] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0803] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0804] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0805] The present invention provides a system for quickly finding a child who has become lost in a large facility by using facial recognition and video data from a surveillance camera. A specific embodiment of the present invention will be described in detail below.

[0806] Acquiring facial image data

[0807] The server captures visitors' facial images in real time from facial recognition cameras installed at the facility entrances. The facial recognition cameras provide high-resolution images, which are necessary for generating accurate digital values ​​in the facial recognition system. The captured facial image data is then digitized and encoded on the spot.

[0808] Storing digitized facial data

[0809] The server stores the encoded facial data in a database, which is used to safely and efficiently manage existing visitors' facial data, allowing for data storage and rapid retrieval.

[0810] Parents' photos can be entered when a child gets lost

[0811] When a child gets lost, the user (facility staff) enters a photo of the parent's face into a dedicated terminal at the lost child center. This photo is then digitized and sent to the server. The server encodes the parent's face data and compares it with the facial data in the database.

[0812] Verifying parent-child relationships

[0813] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database. This process verifies whether the parent and child entered the facility together. The AI ​​checks the degree of similarity of facial features and identifies the parent-child relationship with high accuracy.

[0814] Matching and tracking with surveillance camera footage

[0815] The server acquires video footage from security cameras installed in each area of ​​the facility. The server analyzes each frame of the security camera footage in real time and compares it with the matched parent's facial data. By matching the security camera video data with the parent's facial data, the lost child can be quickly identified.

[0816] Location notification

[0817] If a lost child's location is identified, the server notifies the lost child center, which then provides the specific location information to the parents and facility staff, enabling the child and parent to be reunited quickly and providing peace of mind.

[0818] As a concrete example, consider a large theme park. A facial recognition camera is installed at the entrance, and facial data of visitors is collected when they enter the park. When a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the surveillance camera footage within the park, and if a specific camera captures the lost child, it notifies the child of the child's location. Using this information, the parent and facility staff can quickly reunite the child.

[0819] The above is a specific embodiment of the present invention. This system allows lost children to be quickly reunited with their parents, and increases safety and security for facility users.

[0820] The processing flow will be explained below.

[0821] Step 1:

[0822] The server acquires visitors' facial images in real time from a facial recognition camera installed at the entrance of the facility. The facial images are captured by a high-resolution camera.

[0823] Step 2:

[0824] The server sends the acquired facial image data to the facial recognition system and converts it into digital values, which digitize facial feature points and enable subsequent matching.

[0825] Step 3:

[0826] The server stores the digitized facial data in a database, along with the date, time, and camera position at the time of storage.

[0827] Step 4:

[0828] When a child is lost, the user (facility staff) enters a parent's face photo into the system using a dedicated terminal at the lost child center. The photo is taken using a high-resolution camera.

[0829] Step 5:

[0830] The server digitizes the parent's facial photo and converts it into encoded data, which serves as the basis for matching the lost child.

[0831] Step 6:

[0832] The server uses AI to compare the facial data stored at the facility entrance with the parent's facial data. The AI ​​analyzes the degree of similarity of facial feature points, and if there is a match, it recognizes the parent-child relationship.

[0833] Step 7:

[0834] The server acquires video in real time from surveillance cameras installed in each area of ​​the facility, and each surveillance camera periodically transmits video frames to the server.

[0835] Step 8:

[0836] The server analyzes the video frames from the security cameras and compares them with the stored facial data of the parents, checking whether the facial data matches the specific camera footage.

[0837] Step 9:

[0838] The server identifies the child's location from the surveillance camera footage that matches the parent's facial data, and records the identified location as specific area information.

[0839] Step 10:

[0840] The server then notifies the lost child center of the child's location, including detailed location information and the latest camera footage.

[0841] Step 11:

[0842] The user (facility staff) will then communicate the location information received at the lost child center to the parents, who can then head to the designated area based on the specific location information and quickly reunite with their child.

[0843] This series of processes enables lost children to be quickly reunited with their parents, increasing the safety and security of the facility.

[0844] Example 1

[0845] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0846] When a child gets lost in a large facility, it poses a serious problem for the safety and security of parents and children. With the current system, it is difficult to quickly and accurately find a lost child, which increases the burden on facility staff. In addition, the process of collecting and confirming information when a child gets lost is complicated and time-consuming, so a quick response is required.

[0847] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0848] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a parent's facial photo when a child gets lost, means for digitizing the parent's facial photo data and transmitting it to the server, means for comparing the parent's facial photo data with the facial data in the database using artificial intelligence, means for acquiring footage from multiple surveillance cameras in the facility in real time, means for comparing each frame of the surveillance camera footage with the parent's facial data to identify the location of the lost child, and means for sending a push notification of the identified location of the lost child to the lost child center.This makes it possible to quickly and accurately find a lost child and quickly reunite the parent and child.

[0849] "Facial image data" is digital data that captures the facial features of a visitor.

[0850] "Digitization" is the process of electronically processing analog facial image data to convert it into digital form.

[0851] "Database" means a computer system for storing and managing captured and encoded facial data.

[0852] "Method for inputting a photograph of the parents' faces when a child becomes lost" refers to a method in which facility staff digitally input a photograph of the parents' faces using a dedicated terminal.

[0853] The "means for digitizing facial photograph data and transmitting it to a server" refers to a means for converting a photograph of a face into a digital format and transmitting it to a server.

[0854] "Artificial intelligence" refers to machine learning and deep learning models that perform complex processes such as facial recognition and data matching.

[0855] "Means for acquiring surveillance camera footage in real time" refers to means for continuously and immediately acquiring footage from multiple surveillance cameras installed within a facility.

[0856] The "means for collating each frame of surveillance camera video with parent's facial data" refers to a means for comparing each frame of surveillance camera video acquired in real time with parent's facial data.

[0857] The "means for identifying the location of a lost child" is a means for identifying the location of a lost child based on the location information of surveillance camera footage that matches the facial data of the parent.

[0858] "Push notification" is a communication method for proactively sending the location information of a identified lost child to the lost child center.

[0859] This invention is a system that uses facial recognition and video data from surveillance cameras to quickly find a child who has become lost in a large facility. Specific embodiments of this system will be described in detail below, including the hardware and software used to process and calculate data.

[0860] The server captures visitors' facial images in real time using a high-resolution facial recognition camera (e.g., a typical high-performance camera) installed at the entrance to the facility. The facial recognition camera analyzes facial features and encodes them as digital data. This data is temporarily stored in the server's memory. The server then stores this encoded facial data in a database. This database can be built using MySQL, for example.

[0861] When a child gets lost, the user (facility staff) uses a dedicated terminal at the lost child center to input a parent's facial photo. This terminal digitizes the parent's facial photo and sends the image data to a server. In this transmission process, the image captured by the terminal is transferred to the server via a network. The server then encodes the received parent's facial photo and stores it in a database.

[0862] The server uses artificial intelligence (AI models, such as Amazon Rekognition) to compare the facial data stored in the database with the parent's facial photo data. The AI ​​model evaluates the degree of match of facial features, and if a high match is detected, the parent-child relationship is identified.

[0863] The server captures and analyzes video footage from surveillance cameras (e.g., standard surveillance cameras) installed in each area of ​​the facility in real time. The server then applies a facial recognition algorithm to each frame of video footage from the security cameras and compares it with the parent's facial data. If a match is confirmed, the server records the location information of that frame and identifies the lost child.

[0864] Once the location of a lost child is identified, the server sends the information to the lost child center as a push notification. The lost child center's device receives this information and displays the location information to facility staff and parents. For example, by notifying them that the lost child is in a specific area, it helps to quickly reunite the child.

[0865] As a concrete example, imagine a large theme park. When visitors enter the park, a facial recognition camera at the entrance captures and stores their facial data. If a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the footage from the facility's security cameras, and if a specific security camera captures the lost child, it notifies them of the child's location. Based on this information, the parents and facility staff can quickly reunite the child.

[0866] This system will enable lost children to be quickly reunited with their parents, improving safety and peace of mind for facility users.

[0867] Example prompts to input to a generative AI model:

[0868] "Please explain how the system works by inputting a photo of the parent's face when a child goes missing and matching it with the security camera footage within the facility to locate the child."

[0869] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0870] Step 1: Acquire facial image data

[0871] The server captures visitors' facial images in real time using a high-resolution facial recognition camera installed at the entrance of the facility. The input data is the video stream from the camera. The server extracts individual facial images from this stream and encodes them into digital data. The converted output data is temporarily stored in the server's memory.

[0872] Specific behavior:

[0873] The server receives the video stream transmitted from the camera in real time.

[0874] The server extracts facial images from the video stream using a facial recognition algorithm.

[0875] The server encodes the extracted facial image and converts it into digital data.

[0876] Step 2: Save the digitized face data

[0877] The server stores the encoded facial data in a database. The input data for this step is the digitized facial data acquired in step 1. The server performs error checks before storing to ensure data integrity. Once integrity is confirmed, the data is stored in the database.

[0878] Specific behavior:

[0879] The server performs error checking on the encoded face data.

[0880] The server generates SQL queries to store the digitized face data in a database.

[0881] The server sends an SQL query to the database and receives a response confirming the save.

[0882] Step 3: Enter a photo of the parents' faces in case the child gets lost

[0883] When a child gets lost, the user (facility staff) uses a dedicated terminal at the lost child center to input a parent's facial photo. The input data for this step is the parent's facial photo. The terminal digitizes the input facial photo and sends the image data to the server. The server then encodes the received facial photo and stores it in a database.

[0884] Specific behavior:

[0885] The user (facility staff) takes a photo of the parent's face using a dedicated terminal.

[0886] The terminal digitizes the captured facial photograph.

[0887] The terminal transmits the digitized facial photograph to the server.

[0888] The server encodes the received parent's facial photograph and stores it in a database.

[0889] Step 4: Verify parent-child relationships

[0890] The server uses an artificial intelligence model to match the facial data of the parent entered with the facial data stored in the database at the entrance. The input data for this step is the facial data of the parent and the facial data acquired at the entrance. The server provides the input data to the AI ​​model and evaluates the degree of match of facial features. If a high match is detected, the server identifies the parent-child relationship.

[0891] Specific behavior:

[0892] The server inputs the parents' facial data into the AI ​​model.

[0893] The AI ​​model compares the parent's facial data with multiple facial data stored in the entrance database.

[0894] The server receives the match assessment from the AI ​​model and identifies parent-child relationships.

[0895] Step 5: Matching with CCTV footage and tracking

[0896] The server captures and analyzes real-time video from security cameras installed in each area of ​​the facility. The input data for this step are the video stream from the security cameras and the parent's facial data. The server applies a facial recognition algorithm to each frame from the security cameras and compares it with the parent's facial data. If a match is confirmed, the server records the location information of that frame.

[0897] Specific behavior:

[0898] The server receives the video streams from each surveillance camera.

[0899] The server applies a facial recognition algorithm to each video frame.

[0900] The server records the position information of the frame where a match is confirmed by comparing it with the parent's face data.

[0901] Step 6: Location Notification

[0902] Once the location of the lost child is identified, the server pushes the information to the lost child center. The input data for this step is the location information of the identified lost child. The server converts the location information into an explainable format and sends it to the lost child center's terminal. The lost child center's terminal receives the notification and displays the location information to facility staff and parents.

[0903] Specific behavior:

[0904] The server converts the identified location information of the lost child into an explainable format.

[0905] The server sends the location information to the lost child center's terminal as a push notification.

[0906] The terminal at the lost child center displays the location information to facility staff and parents based on the received location information.

[0907] (Application example 1)

[0908] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0909] When a child gets lost in a large facility, a system that can quickly and accurately locate the child is extremely important. However, with existing systems, it takes time to identify the lost child, causing anxiety and stress for parents and facility staff. Furthermore, there is a lack of a way for parents to directly obtain information about the lost child's location, further extending the time until the child is reunited. Furthermore, information sharing with lost child centers and security guards is often not smooth. A system that can solve these problems and respond quickly and efficiently when a child gets lost is needed.

[0910] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0911] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a parent's facial photograph when a child gets lost, means for comparing the parent's facial photograph data with the facial data in the database, means for acquiring footage from multiple surveillance cameras within the facility, means for comparing the surveillance camera footage with the parent's facial photograph data to identify the location of the lost child, means for notifying the parent's mobile device of the identified location information of the lost child, and means for sharing the location information notified to the parent's mobile device with the facility's lost child center and security guard terminals in real time. This allows parents to quickly determine the child's location and share the information with the lost child center and security guard, enabling a prompt reunion.

[0912] "Facial image data" refers to facial image information captured by a facial recognition camera or other camera device.

[0913] "Digitalized facial data" refers to facial image data that has been digitally digitized and converted into a form that is easy for a computer to handle.

[0914] A "database" refers to a system that efficiently manages accumulated digital data and quickly searches and obtains necessary information.

[0915] "Parent's face photo" refers to a photo of the face provided by the parent when the child gets lost. This photo is digitized and sent to the server.

[0916] "Matching" refers to the process of comparing one piece of data with another to see if they match.

[0917] A "surveillance camera" refers to a camera installed within a facility to monitor and record the movements of people and objects in real time.

[0918] "Location information of lost children" refers to information showing the current location of a lost child. This information is obtained by comparing video data from surveillance cameras with facial data.

[0919] "Parent's mobile device" refers to a mobile information device such as a smartphone or tablet owned by the parent. This device will be notified of the lost child's location.

[0920] A "lost child center" is a department located within a large facility that specializes in dealing with lost children.

[0921] "Security guard devices" refer to the mobile information devices owned by security guards at the facility. The location information of lost children is also shared with these devices.

[0922] This invention is a system for quickly reuniting parents and children when they get lost in a large facility. This system is composed of a facial recognition camera, a surveillance camera, a parent's mobile device, a lost child center, and a security guard's terminal.

[0923] The server first uses a facial recognition camera installed at the entrance of the facility to obtain facial image data. The facial recognition camera provides high-resolution images that allow the facial recognition system to generate accurate digital values. The facial image data is digitized and transmitted to the server in real time.

[0924] The server stores the captured digitized facial data in a database designed to securely and efficiently manage visitor facial data and enable rapid search and analysis.

[0925] When a child gets lost, the parent enters a facial photo via a dedicated terminal at the lost child center. This photo is then digitized and sent to a server. The server encodes the parent's facial photo data and compares it with facial data in a database. This identifies the parent-child relationship and confirms that the child has entered the facility.

[0926] The server captures and analyzes video footage from multiple surveillance cameras installed within the facility in real time. By comparing the video data from the surveillance cameras with facial photos of the parents, the location of the lost child can be identified. AI technology is used to confirm the degree of match of facial features with high accuracy.

[0927] When a lost child's location is identified, the server notifies the parent's mobile device of the location information. The location information is also shared in real time with the lost child center and security guards' devices. This function allows parents to quickly determine the child's location, and the lost child center and security guards can take action at the same time.

[0928] As an example of how this system can be put into practical use, consider a large shopping mall. As a parent and child pass through the entrance, facial image data is captured by a facial recognition camera. If a child gets lost, the parent provides a facial photo to the lost child center, and the data is collated on a server. After that, an analysis of the surveillance camera identifies the lost child as being in the food court, and the parent's smartphone is notified of the child's location. At the same time, this information is shared with the lost child center and security guards, allowing the parent and child to be quickly reunited.

[0929] An example of an input prompt for a generative AI model is:

[0930] "Please create a system to quickly find children who get lost in a shopping mall. The system will use facial recognition cameras and surveillance cameras to capture and match the facial data of parents and children. Please also implement a function to notify the parents' smartphones of the child's location when the child gets lost."

[0931] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0932] Step 1:

[0933] The server acquires the visitor's facial image data in real time using a facial recognition camera installed at the facility entrance. The facial recognition camera provides high-resolution images and generates accurate digital values ​​in the facial recognition system. The server acquires the facial image data as input and obtains the digitized facial data as output.

[0934] Step 2:

[0935] The server digitizes and encodes the acquired facial image data, converting it into a digital format that is easy for a computer to handle. It receives facial image data as input and obtains encoded facial data as output.

[0936] Step 3:

[0937] The server stores the encoded facial data in a database designed to efficiently manage visitor facial data and enable rapid search and analysis. The server receives digitized facial data as input and stores it in the database.

[0938] Step 4:

[0939] When a child gets lost, the user (parent) enters a facial photo through a dedicated terminal at the lost child center. The parent's facial photo is digitized on the spot and sent to the server. The parent's facial photo is obtained as input, and the digitized parent's facial photo data is obtained as output.

[0940] Step 5:

[0941] The server encodes the parent's facial photo data and matches it with facial data in the database. This process identifies the parent-child relationship and verifies whether the parent and child entered the facility together. The server receives digitized parent facial data as input and provides information on the degree of match between the facial data as output.

[0942] Step 6:

[0943] The server acquires video footage from multiple surveillance cameras installed within the facility in real time. The video data from the surveillance cameras is used to match the parent's facial data. The surveillance camera video is received as input, and the video data for analysis is obtained as output.

[0944] Step 7:

[0945] The server analyzes the surveillance camera footage and compares it with the parent's facial data. This analysis is performed using AI technology to confirm the degree of match of facial features with high accuracy. The server receives the surveillance camera footage and the parent's facial data as input, and obtains specific information about the matching parts as output.

[0946] Step 8:

[0947] The server identifies the location of the lost child and notifies the parent's mobile device of the location information. At the same time, the server also shares the location information with the lost child center and security guards' devices in real time. The server receives the matching part of the location information as input and outputs the location information notification.

[0948] Step 9:

[0949] The parents check the location information of their lost child that is notified to their mobile device, and then use that information to work with facility staff to find the lost child.The location information notification is received as input, and the actual on-site confirmation and response action are output.

[0950] The above are the specific processing steps of the system that realizes the application example.

[0951] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0952] This invention is a system that, when a child gets lost in a large facility, quickly finds the child using facial recognition and video data from a surveillance camera, and further recognizes the emotional states of the parent and child using an emotion engine, and provides appropriate support. Specific embodiments of the present invention will be described in detail below.

[0953] Acquiring facial image data

[0954] The server acquires visitors' facial images in real time from a facial recognition camera installed at the entrance of the facility. The facial images are captured by a high-resolution camera, which allows for detailed analysis of facial feature points.

[0955] Storing digitized facial data

[0956] The server sends the acquired facial image data to the facial recognition system and converts it into digital values. This conversion quantifies facial feature points, enabling subsequent matching. The digitized data is stored in a database and managed along with metadata such as date, time, and camera position.

[0957] Parents' photos can be entered when a child gets lost

[0958] When a child gets lost, the user (facility staff) enters a parent's facial photo into the system via a dedicated terminal at the lost child center. The photo is taken using a high-resolution camera and sent to the server. The server digitizes the parent's facial photo and converts it into encoded data.

[0959] Verifying parent-child relationships

[0960] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database. This process verifies whether the parent and child entered the facility together. The AI ​​checks the degree of similarity of facial features and identifies the parent-child relationship with high accuracy.

[0961] Matching and tracking with surveillance camera footage

[0962] The server acquires video in real time from surveillance cameras installed in each area of ​​the facility. Each surveillance camera periodically transmits video frames to the server. The server analyzes the video frames from the surveillance cameras and compares them with the stored facial data of the parents. By comparing the video data from the surveillance cameras with the facial data of the parents, lost children can be quickly identified.

[0963] Location notification

[0964] If a lost child's location is identified, the server notifies the lost child center with the location information, including the child's detailed location and the latest camera footage, providing parents and facility staff with specific location information and facilitating a speedy reunion.

[0965] Emotion recognition by emotion engine

[0966] The server is equipped with an emotion engine that recognizes the emotional state of parents and lost children in real time. The emotion engine analyzes facial expression and voice data acquired through a facial recognition camera and sensors on the dedicated device to estimate the emotional state. For example, if a parent is feeling stressed, the emotion engine will notify the server based on that information.

[0967] Supported by an emotional engine

[0968] The server provides appropriate support based on the emotional data acquired by the emotion engine. For example, if a parent is in a very unstable state, the server notifies the facility staff of this information and instructs them on how to provide psychological support and take appropriate measures. This reduces the anxiety of both parents and children and supports rapid reunification.

[0969] As a concrete example, consider a large theme park. A facial recognition camera is installed at the entrance, and facial data of visitors is collected when they enter the park. When a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the surveillance camera footage within the park, and if a specific camera captures the lost child, it notifies the location information and the parent's emotional state. Based on this information, the parent and facility staff can quickly reunite, and at the same time, the parent can receive appropriate support according to their emotional state.

[0970] The above is a specific embodiment of the present invention. This system allows lost children to be quickly reunited with their parents, enhancing safety and a sense of security for facility users. Furthermore, the use of an emotion engine reduces anxiety for parents and children, supporting a safe and secure facility experience.

[0971] The processing flow will be explained below.

[0972] Step 1:

[0973] The server acquires visitors' facial images in real time from a facial recognition camera installed at the entrance of the facility. The facial images are captured by a high-resolution camera.

[0974] Step 2:

[0975] The server sends the acquired facial image data to the facial recognition system and converts it into digital values. By quantifying facial features in detail, subsequent matching can be performed.

[0976] Step 3:

[0977] The server stores the digitized facial data in a database, along with metadata such as date, time, and camera position.

[0978] Step 4:

[0979] When a child gets lost, the user (facility staff) enters a photo of the parent's face into the system via a dedicated terminal at the lost child center. The photo is taken using a high-resolution camera.

[0980] Step 5:

[0981] The server digitizes the parent's facial photo data and converts it into encoded data, which is used as the basis for matching lost children.

[0982] Step 6:

[0983] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database. The AI ​​analyzes the degree of similarity of facial features and, if there is a match, recognizes the parent-child relationship.

[0984] Step 7:

[0985] The server acquires video in real time from surveillance cameras installed in each area of ​​the facility, and each surveillance camera periodically transmits video frames to the server.

[0986] Step 8:

[0987] The server analyzes the video frames from the security cameras and compares them with the stored facial data of the parents, checking whether the facial data matches the specific camera footage.

[0988] Step 9:

[0989] The server identifies the child's location from the surveillance camera footage that matches the parent's facial data, and records the identified location as specific area information.

[0990] Step 10:

[0991] The server then notifies the lost child center of the location of the identified lost child, including detailed location information and the latest camera footage.

[0992] Step 11:

[0993] The user (facility staff) will then communicate the location information received at the lost child center to the parents, who can then head to the designated area based on the specific location information and quickly reunite with their child.

[0994] Step 12:

[0995] The server activates an emotion engine that recognizes the emotional state of the parent and the lost child in real time. The emotion engine analyzes facial expression and voice data acquired through a facial recognition camera and sensors on the dedicated device to estimate the emotional state.

[0996] Step 13:

[0997] The server provides appropriate support based on the parent's emotional data analyzed by the emotion engine. For example, if the parent is feeling stressed, the server notifies facility staff of this information and instructs them on how to provide psychological support and take appropriate measures.

[0998] Step 14:

[0999] Based on the analysis results of the emotion engine, the user (facility staff) communicates with the parents in a way that gives them a sense of security, thereby reducing their anxiety and supporting a speedy reunion.

[1000] This series of processes not only enables lost children to be quickly reunited with their parents, but also reduces the anxiety of both parents and children and increases the sense of safety and security for facility users.

[1001] Example 2

[1002] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1003] Having a child get lost in a large facility is a major source of anxiety for parents. It is also extremely important for facility staff to quickly and accurately locate lost children, but conventional methods often require a lot of time and effort. Furthermore, understanding the emotional state of lost children and their parents and providing appropriate support has also been a challenge. Therefore, the objective of this invention is to provide a system that uses facial recognition technology and surveillance camera footage to quickly identify lost children and recognize and respond to the emotional state of parents and children.

[1004] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1005] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a parent's facial photo when a child gets lost, means for using artificial intelligence to compare the parent's facial photo data with the facial data in the database, means for acquiring footage from multiple surveillance cameras within the facility, means for locating the lost child by comparing the surveillance camera footage with the parent's facial photo data, means for notifying the location of the located lost child, means including an emotion engine for recognizing the emotional states of the parent and the lost child in real time, and means for providing appropriate support based on the emotion data acquired by the emotion engine. This enables rapid location identification of the lost child and appropriate support taking into account the emotional states of the parent and child.

[1006] "Means for acquiring facial image data" refers to devices or technologies for capturing high-resolution images of visitors' faces, and specifically includes facial recognition cameras.

[1007] "Means for digitizing facial image data" refers to software or systems for analyzing acquired facial images and converting them into numerical data.

[1008] "Means for storing digitized facial data in a database" refers to technologies or systems that securely and effectively store facial data converted into a numerical data format in a database along with certain metadata.

[1009] "Means for inputting a parent's facial photograph when a child gets lost" refers to a device or method for inputting and transmitting a parent's facial photograph into the system via a dedicated terminal.

[1010] "Means using artificial intelligence to match parent facial photograph data with facial data in the database" refers to a matching system that uses AI technology to compare parent facial photographs with facial data in the database and determine the degree of match with a high degree of accuracy.

[1011] "Means for acquiring footage from multiple surveillance cameras within a facility" refers to systems and technologies for collecting real-time video data from multiple surveillance cameras installed within a facility.

[1012] "Means for locating a lost child by comparing surveillance camera footage with facial photographic data of parents" refers to technology or systems for locating a lost child by comparing surveillance camera footage with facial photographic data of parents.

[1013] "Means for notifying the location of identified lost children" refers to a notification system that quickly notifies facility personnel of the precise location of identified lost children.

[1014] "Means including an emotion engine for recognizing the emotional states of the parent and the lost child in real time" refers to an emotion recognition engine for analyzing facial expressions and voice data and detecting the emotional states of the parent and the lost child in real time.

[1015] "Means for providing appropriate support based on emotional data obtained by the emotion engine" refers to a support system that uses data from the emotion recognition engine to give instructions to facility staff and suggest specific countermeasures.

[1016] This invention is a system that, when a child gets lost in a large facility, quickly finds the child using facial recognition and video data from a surveillance camera, and further recognizes the emotional states of the parent and child using an emotion engine, and provides appropriate support. Specific embodiments are described below.

[1017] System configuration

[1018] The system includes the following hardware and software:

[1019] Hardware

[1020] High-resolution facial recognition camera (captures facial image data)

[1021] Surveillance cameras (capture footage within the facility)

[1022] Dedicated terminal (parent's photo input)

[1023] software

[1024] Facial recognition system (analysis and digitization of facial image data)

[1025] Database (storage of digitized facial data)

[1026] Artificial intelligence (matching parental facial data with facial data in the database)

[1027] Emotion engine (recognition and analysis of emotional states)

[1028] Notification system (notifying the location of identified lost children)

[1029] Process Overview

[1030] 1. Acquiring facial image data

[1031] The server captures visitors' facial images in real time from a high-resolution facial recognition camera, allowing detailed facial feature analysis.

[1032] 2. Storage of digitized facial data

[1033] The server sends the acquired facial image data to a facial recognition system and converts it into numerical data, which is then stored in a database along with metadata.

[1034] 3. Parental photo input in case of lost child

[1035] When a child gets lost, the user (facility staff) enters a parent's facial photo into the system using a dedicated terminal at the lost child center. The entered facial photo is sent to the server and digitized.

[1036] 4. Confirming parent-child relationships

[1037] The server uses artificial intelligence to match the facial data stored in the database with the facial data of the parent entered, making it possible to verify whether the parent and child entered the facility together.

[1038] 5. Matching with surveillance camera footage and tracking

[1039] The server receives real-time footage from surveillance cameras installed in each area and matches it with stored facial data of parents to identify lost children.

[1040] 6. Location Notifications

[1041] If a lost child's location is identified, the server notifies the lost child center with the information, including detailed location information and the latest camera footage.

[1042] 7. Emotion Recognition with Emotion Engine

[1043] The server uses an emotion engine to analyze facial expression and voice data acquired through a facial recognition camera and dedicated terminals, and recognizes the emotional state of parents and lost children in real time.

[1044] 8. Supported by Emotion Engine

[1045] The server provides appropriate support based on the analysis results of the emotion engine. For example, if the server determines that a parent is in an extremely unstable state, it will notify facility staff and instruct them on how to provide psychological support and take appropriate action.

[1046] Specific examples

[1047] Let's take a large theme park as an example. Facial recognition cameras installed at the entrance collect facial data of visitors. When a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the surveillance camera footage within the park, and if a specific camera captures a lost child, it notifies the child's location and the parent's emotional state. Using this information, the parent and facility staff can quickly reunite, and the parent can also receive psychological support.

[1048] Example prompts for generative AI models

[1049] "Please explain the specific process flow of a system that uses video data from facial recognition cameras and security cameras to quickly locate children who become lost in large facilities. Please also provide details on the hardware and software used, the data processing method, and the emotional state recognition by the emotion engine."

[1050] This system allows lost children to be quickly reunited with their parents, increasing the safety and sense of security of facility users.The use of an emotion engine also reduces anxiety for parents and children, allowing them to use the facility with peace of mind.

[1051] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1052] Step 1:

[1053] The server acquires face image data in real time from the face recognition camera.

[1054] Specifically, the facial recognition camera photographs the face of a visitor as they pass through the entrance and immediately sends the image data to a server.

[1055] Input: Real-time facial images of visitors passing through the entrance

[1056] Output: Facial image data sent to the server

[1057] Step 2:

[1058] The server passes the received facial image data to a facial authentication system and digitizes it.

[1059] A facial recognition system analyzes facial feature points and encodes them as numerical data.

[1060] Specifically, the facial recognition system analyzes the position and shape of the eyes, nose, and mouth and converts this into numerical data.

[1061] Input: Facial image data stored on the server

[1062] Output: Digitized face data

[1063] Step 3:

[1064] The server stores the digitized facial data in a database.

[1065] When saving, metadata such as date, time, and camera location are also recorded.

[1066] Specifically, the server stores the numerical data in a database and manages the data efficiently.

[1067] Input: Digitized face data, metadata

[1068] Output: Face data stored in a database

[1069] Step 4:

[1070] When a child gets lost, the user (facility staff) takes a photo of the parent's face using a dedicated terminal and enters it into the system.

[1071] The terminal transmits the captured facial photograph data to the server.

[1072] Specifically, the user takes a photo of the parent's face using a dedicated terminal, and the terminal then sends the data to the server.

[1073] Input: Parent's photo taken by facility staff

[1074] Output: Parent face photo data sent to the server

[1075] Step 5:

[1076] The server receives and digitizes the facial photograph data of the parents.

[1077] The facial recognition system encodes the parent's facial data as numerical data.

[1078] Specifically, the server passes the parent's facial photo data to the facial recognition system, which then analyzes it and converts it into numerical data.

[1079] Input: Parent's facial photo data received by the server

[1080] Output: Digitized parent face data

[1081] Step 6:

[1082] The server uses artificial intelligence to compare the facial data of the parent stored in the database with the facial data of the person at the entrance.

[1083] The AI ​​calculates the degree of similarity of facial features and confirms the parent-child relationship.

[1084] Specifically, the server uses an AI matching system to compare the parent's facial data with the facial data in the database and determine the degree of match.

[1085] Input: Digitized parent face data

[1086] Output: Confirmation of parent-child relationship based on the degree of matching of facial feature points

[1087] Step 7:

[1088] The server acquires real-time images from surveillance cameras installed in each area of ​​the facility.

[1089] Each surveillance camera transmits video frames to the server at regular intervals.

[1090] Specifically, the server continuously receives video data transmitted from multiple surveillance cameras.

[1091] Input: Security camera footage from inside the facility

[1092] Output: Real-time video of each area sent to the server

[1093] Step 8:

[1094] The server compares the surveillance camera footage with the stored facial data of the parents to identify the lost child.

[1095] The surveillance camera video frames are analyzed to detect matches between the facial data of the parent and child.

[1096] Specifically, the server uses a video analysis system to compare surveillance camera footage with the parent's facial data to confirm the location of the lost child.

[1097] Input: Surveillance camera footage, digitized parental facial data

[1098] Output: Location information of the lost child as a result of matching with surveillance camera footage

[1099] Step 9:

[1100] If the location of the lost child is determined, the server notifies the lost child center of this information.

[1101] The notification will include detailed location information and the latest camera footage.

[1102] Specifically, the server generates a notification message based on the identified location information and sends it to the lost child center.

[1103] Input: Matched location of lost child

[1104] Output: Location notification sent to the lost child center

[1105] Step 10:

[1106] The server is equipped with an emotion engine that recognizes the emotional state of the parent and the lost child in real time.

[1107] The emotion engine analyzes facial expression and voice data to estimate the emotional state.

[1108] Specifically, the server uses an emotion engine to analyze the emotional state of the parent and the lost child in real time.

[1109] Input: facial expression data, voice data

[1110] Output: Parsed emotional state data

[1111] Step 11:

[1112] The server provides appropriate support based on the emotion data acquired by the emotion engine.

[1113] For example, if a parent is in a very unstable state, the server will notify facility staff and instruct them on psychological support and appropriate measures.

[1114] As a specific operation, the server analyzes the emotion data and notifies the facility staff of the specific support required.

[1115] Input: Emotional state data

[1116] Output: Support instruction notification to facility staff

[1117] (Application example 2)

[1118] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1119] When a child gets lost in a large facility, there is a need to quickly find the child and reunite them with their parents safely and securely. However, current systems take time to locate a lost child, making it difficult to alleviate the anxiety of the parent and child. Furthermore, simply providing location information without considering the emotional state of the parent and child does not provide sufficient support. Therefore, there is a need for a system that can quickly locate a lost child while also providing appropriate support based on the emotional state of the parent and child.

[1120] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1121] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a facial photo of a parent when a child gets lost, means for comparing the facial photo data of the parent with the facial data in the database, means for acquiring footage from multiple surveillance cameras within the facility, means for comparing the surveillance camera footage with the facial photo data of the parent to identify the location of the lost child, means for notifying the location information of the identified lost child, emotion analysis means for recognizing the emotional states of the parent and the lost child, and means for providing support based on the emotional states, thereby enabling the quick discovery of a lost child and support according to the emotional states of the parent and child.

[1122] The "means for acquiring facial image data" refers to a device and method for capturing facial images of visitors using a facial recognition camera installed at the entrance of the facility or in each area.

[1123] The "means for digitizing the facial image data" refers to a process and software for analyzing the acquired facial image and converting facial feature points into digital values.

[1124] The "means for storing the digitized facial data in a database" refers to a system and method for storing facial data converted into digital values ​​in a database and adding metadata such as date, time, and camera position.

[1125] The "means for inputting a parent's facial photograph when a child gets lost" refers to a device and method for taking a parent's facial photograph using a dedicated terminal or smart device and inputting it into the system.

[1126] The "means for matching parent's facial photograph data with the facial data in the database" refers to the process and software for using artificial intelligence to compare and match parent's facial photograph data with the facial data obtained upon entry into the facility.

[1127] The "means for acquiring images from multiple surveillance cameras within a facility" refers to a system and method for transmitting images from surveillance cameras installed in each area within a facility to a server in real time.

[1128] The "means for locating a lost child by comparing the surveillance camera footage with the facial photograph data of the parent" refers to the process and software for comparing and matching the surveillance camera footage with the facial photograph data of the parent to locate a lost child.

[1129] The "means for notifying the location information of the identified lost child" refers to a system and method for notifying the location information of the identified lost child to a lost child center or a smart device.

[1130] The "emotion analysis means for recognizing the emotional states of parents and lost children" is a device and method that uses an emotion engine to analyze facial expression data and voice data acquired and estimate the emotional states of parents and children.

[1131] The "means for providing support based on the emotional state" refers to a system and method for providing appropriate psychological support and responses based on the emotional states of the parent and the lost child.

[1132] The present invention is a system that, when a child gets lost in a large facility, quickly finds the child by utilizing facial recognition and video data from a surveillance camera, and further recognizes the emotional states of the parent and child using an emotion engine, and provides appropriate support. Specific embodiments of the present invention will be described in detail below.

[1133] Acquiring facial image data

[1134] The server captures visitors' facial images in real time through a facial recognition camera installed at the entrance of the facility. This camera has high resolution and can analyze facial feature points in detail. The facial images are converted into digital values ​​and stored in a database.

[1135] Storing digitized facial data

[1136] The server sends the acquired facial image data to a facial recognition system (e.g., Amazon Rekognition, Microsoft Azure Face API) and converts it into digital values. This conversion digitizes facial feature points, facilitating subsequent matching. The digitized data is stored in a database using cloud services such as AWS or Google Cloud. When the data is saved, metadata such as the date, time, and camera position is also added.

[1137] Parents' photos can be entered when a child gets lost

[1138] When a child gets lost, the user (facility staff) uses a dedicated terminal or smart device to input a parent's facial photo into the system. The parent's facial photo is taken with a high-resolution camera and sent to the server. The server digitizes the parent's facial photo and converts it into encoded data.

[1139] Verifying parent-child relationships

[1140] The server uses artificial intelligence (AI) to compare the facial data stored in the database at the entrance with the facial data of the parent entered, making it possible to confirm with high accuracy whether the parent and child entered the facility together.

[1141] Matching and tracking with surveillance camera footage

[1142] The server captures video in real time from multiple surveillance cameras installed within the facility. Each surveillance camera periodically transmits video frames to the server. The server analyzes the video frames and matches them with the facial data of the parents to identify the lost child.

[1143] Location notification

[1144] If the server locates a lost child, the information is sent to a dedicated terminal or smart device, along with detailed location information and the latest camera footage, helping parents and facility staff quickly reunite the child.

[1145] Emotion recognition by emotion engine

[1146] The server recognizes the emotional state using an emotion engine (e.g., Affectiva Emotion AI, Microsoft Azure Text Analytics). This engine analyzes facial expression and voice data acquired from a facial recognition camera and sensors on the dedicated device to estimate the emotional state of the parent and lost child.

[1147] Supported by an emotional engine

[1148] The server provides appropriate support based on the emotional data acquired by the emotion engine. For example, if a parent is extremely unstable, the server notifies facility staff of this information and instructs them on how to provide psychological support and take appropriate measures.

[1149] Specific examples and prompts for generative AI models

[1150] As a concrete example, consider a large shopping mall. A facial recognition camera is installed at the entrance, and facial data of visitors is collected. When a child gets lost, the parent uses smart glasses to go to the lost child center and input a photo of their face. The server matches the parent and child's faces, and by analyzing the surveillance camera footage, locates the child's location and notifies the smart glasses.

[1151] Example prompts for generative AI models:

[1152] ---

[1153] Please provide a detailed description of a child loss prevention system that uses facial recognition and security cameras. Explain the functionality of the smart glasses application, which compares the facial data of parents and children to quickly identify a child if they get lost and notify the parent of their location and emotional state. Include the specific processing steps and technologies used.

[1154] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1155] Step 1: Acquire facial image data

[1156] The server captures visitors' facial images through high-resolution facial recognition cameras installed at the entrances of the facilities. The input is facial image data, which is then converted into a digital format. Specifically, the server takes the facial image captured by the camera sensor as initial data, and processes the high-quality image through an AI-based facial recognition algorithm.

[1157] Step 2: Digitizing the face data

[1158] The server sends the acquired facial image data to the facial recognition system, where an AI algorithm analyzes the facial feature points and converts them into digital values. The input is the captured facial image, and the output is the digitized facial feature point data. This process uses a high-speed image analysis algorithm and is performed in real time.

[1159] Step 3: Save the digitized data

[1160] The server stores the digitized facial feature data in a cloud database, along with metadata such as date, time, and camera position. The input is the digitized facial feature data and metadata, and the output is a record in the database.

[1161] Step 4: Enter a photo of the parents' faces in case the child gets lost

[1162] The user (facility staff) uses a dedicated terminal or smart device to input a facial photo of the parent of the lost child into the system. The parent's facial photo is taken with a high-resolution camera and sent to the server. The input is the parent's facial image data, and the output is digitized facial feature point data of the parent.

[1163] Step 5: Verify parent-child relationships

[1164] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database at the entrance. The input is the parent's digitized facial data and the facial data in the database, and the output is the parent-child relationship identification result with a high degree of match. In this process, the AI ​​compares facial feature points and determines the parent-child relationship with a high degree of accuracy.

[1165] Step 6: Matching with CCTV footage and tracking

[1166] The server receives real-time video footage from multiple surveillance cameras installed within the facility. It analyzes these videos and matches them with the facial data of the parents to identify lost children. The input is the video frames from each surveillance camera and the parents' digitized facial data, and the output is the identified location information of the lost child.

[1167] Step 7: Location Notification

[1168] When the server identifies the location of a lost child, it notifies the dedicated terminal or smart device of that information. The notification includes the latest camera footage and detailed location information. The input is the location information and video data of the lost child, and the output is a notification message.

[1169] Step 8: Emotion Recognition with the Emotion Engine

[1170] The server uses an emotion engine to recognize the emotional state of the parent and the lost child in real time. The input is facial expression and voice data acquired from a facial recognition camera and sensors on a dedicated device, and the output is an estimated emotional state. This allows for a detailed understanding of the emotional states of the parent and child.

[1171] Step 9: Support from the Emotion Engine

[1172] The server provides appropriate support based on the emotional data acquired by the emotion engine. For example, if a parent is in a very unstable state, the server notifies facility staff of this information and instructs them on how to provide psychological support and appropriate measures for the parent. The input is emotional data, and the output is instructions and provision of support measures.

[1173] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1174] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1175] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1176] [Fourth embodiment]

[1177] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1178] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1179] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1180] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1181] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1182] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1183] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1184] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1185] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1186] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1187] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1188] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1189] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1190] The present invention provides a system for quickly finding a child who has become lost in a large facility by using facial recognition and video data from a surveillance camera. A specific embodiment of the present invention will be described in detail below.

[1191] Acquiring facial image data

[1192] The server captures visitors' facial images in real time from facial recognition cameras installed at the facility entrances. The facial recognition cameras provide high-resolution images, which are necessary for generating accurate digital values ​​in the facial recognition system. The captured facial image data is then digitized and encoded on the spot.

[1193] Storing digitized facial data

[1194] The server stores the encoded facial data in a database, which is used to safely and efficiently manage existing visitors' facial data, allowing for data storage and rapid retrieval.

[1195] Parents' photos can be entered when a child gets lost

[1196] When a child gets lost, the user (facility staff) enters a photo of the parent's face into a dedicated terminal at the lost child center. This photo is then digitized and sent to the server. The server encodes the parent's face data and compares it with the facial data in the database.

[1197] Verifying parent-child relationships

[1198] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database. This process verifies whether the parent and child entered the facility together. The AI ​​checks the degree of similarity of facial features and identifies the parent-child relationship with high accuracy.

[1199] Matching and tracking with surveillance camera footage

[1200] The server acquires video footage from security cameras installed in each area of ​​the facility. The server analyzes each frame of the security camera footage in real time and compares it with the matched parent's facial data. By matching the security camera video data with the parent's facial data, the lost child can be quickly identified.

[1201] Location notification

[1202] If a lost child's location is identified, the server notifies the lost child center, which then provides the specific location information to the parents and facility staff, enabling the child and parent to be reunited quickly and providing peace of mind.

[1203] As a concrete example, consider a large theme park. A facial recognition camera is installed at the entrance, and facial data of visitors is collected when they enter the park. When a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the surveillance camera footage within the park, and if a specific camera captures the lost child, it notifies the child of the child's location. Using this information, the parent and facility staff can quickly reunite the child.

[1204] The above is a specific embodiment of the present invention. This system allows lost children to be quickly reunited with their parents, and increases safety and security for facility users.

[1205] The processing flow will be explained below.

[1206] Step 1:

[1207] The server acquires visitors' facial images in real time from a facial recognition camera installed at the entrance of the facility. The facial images are captured by a high-resolution camera.

[1208] Step 2:

[1209] The server sends the acquired facial image data to the facial recognition system and converts it into digital values, which digitize facial feature points and enable subsequent matching.

[1210] Step 3:

[1211] The server stores the digitized facial data in a database, along with the date, time, and camera position at the time of storage.

[1212] Step 4:

[1213] When a child is lost, the user (facility staff) enters a parent's face photo into the system using a dedicated terminal at the lost child center. The photo is taken using a high-resolution camera.

[1214] Step 5:

[1215] The server digitizes the parent's facial photo and converts it into encoded data, which serves as the basis for matching the lost child.

[1216] Step 6:

[1217] The server uses AI to compare the facial data stored at the facility entrance with the parent's facial data. The AI ​​analyzes the degree of similarity of facial feature points, and if there is a match, it recognizes the parent-child relationship.

[1218] Step 7:

[1219] The server acquires video in real time from surveillance cameras installed in each area of ​​the facility, and each surveillance camera periodically transmits video frames to the server.

[1220] Step 8:

[1221] The server analyzes the video frames from the security cameras and compares them with the stored facial data of the parents, checking whether the facial data matches the specific camera footage.

[1222] Step 9:

[1223] The server identifies the child's location from the surveillance camera footage that matches the parent's facial data, and records the identified location as specific area information.

[1224] Step 10:

[1225] The server then notifies the lost child center of the child's location, including detailed location information and the latest camera footage.

[1226] Step 11:

[1227] The user (facility staff) will then communicate the location information received at the lost child center to the parents, who can then head to the designated area based on the specific location information and quickly reunite with their child.

[1228] This series of processes enables lost children to be quickly reunited with their parents, increasing the safety and security of the facility.

[1229] Example 1

[1230] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1231] When a child gets lost in a large facility, it poses a serious problem for the safety and security of parents and children. With the current system, it is difficult to quickly and accurately find a lost child, which increases the burden on facility staff. In addition, the process of collecting and confirming information when a child gets lost is complicated and time-consuming, so a quick response is required.

[1232] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1233] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a parent's facial photo when a child gets lost, means for digitizing the parent's facial photo data and transmitting it to the server, means for comparing the parent's facial photo data with the facial data in the database using artificial intelligence, means for acquiring footage from multiple surveillance cameras in the facility in real time, means for comparing each frame of the surveillance camera footage with the parent's facial data to identify the location of the lost child, and means for sending a push notification of the identified location of the lost child to the lost child center.This makes it possible to quickly and accurately find a lost child and quickly reunite the parent and child.

[1234] "Facial image data" is digital data that captures the facial features of a visitor.

[1235] "Digitization" is the process of electronically processing analog facial image data to convert it into digital form.

[1236] "Database" means a computer system for storing and managing captured and encoded facial data.

[1237] "Method for inputting a photograph of the parents' faces when a child becomes lost" refers to a method in which facility staff digitally input a photograph of the parents' faces using a dedicated terminal.

[1238] The "means for digitizing facial photograph data and transmitting it to a server" refers to a means for converting a photograph of a face into a digital format and transmitting it to a server.

[1239] "Artificial intelligence" refers to machine learning and deep learning models that perform complex processes such as facial recognition and data matching.

[1240] "Means for acquiring surveillance camera footage in real time" refers to means for continuously and immediately acquiring footage from multiple surveillance cameras installed within a facility.

[1241] The "means for collating each frame of surveillance camera video with parent's facial data" refers to a means for comparing each frame of surveillance camera video acquired in real time with parent's facial data.

[1242] The "means for identifying the location of a lost child" is a means for identifying the location of a lost child based on the location information of surveillance camera footage that matches the facial data of the parent.

[1243] "Push notification" is a communication method for proactively sending the location information of a identified lost child to the lost child center.

[1244] This invention is a system that uses facial recognition and video data from surveillance cameras to quickly find a child who has become lost in a large facility. Specific embodiments of this system will be described in detail below, including the hardware and software used to process and calculate data.

[1245] The server captures visitors' facial images in real time using a high-resolution facial recognition camera (e.g., a typical high-performance camera) installed at the entrance to the facility. The facial recognition camera analyzes facial features and encodes them as digital data. This data is temporarily stored in the server's memory. The server then stores this encoded facial data in a database. This database can be built using MySQL, for example.

[1246] When a child gets lost, the user (facility staff) uses a dedicated terminal at the lost child center to input a parent's facial photo. This terminal digitizes the parent's facial photo and sends the image data to a server. In this transmission process, the image captured by the terminal is transferred to the server via a network. The server then encodes the received parent's facial photo and stores it in a database.

[1247] The server uses artificial intelligence (AI models, such as Amazon Rekognition) to compare the facial data stored in the database with the parent's facial photo data. The AI ​​model evaluates the degree of match of facial features, and if a high match is detected, the parent-child relationship is identified.

[1248] The server captures and analyzes video footage from surveillance cameras (e.g., standard surveillance cameras) installed in each area of ​​the facility in real time. The server then applies a facial recognition algorithm to each frame of video footage from the security cameras and compares it with the parent's facial data. If a match is confirmed, the server records the location information of that frame and identifies the lost child.

[1249] Once the location of a lost child is identified, the server sends the information to the lost child center as a push notification. The lost child center's device receives this information and displays the location information to facility staff and parents. For example, by notifying them that the lost child is in a specific area, it helps to quickly reunite the child.

[1250] As a concrete example, imagine a large theme park. When visitors enter the park, a facial recognition camera at the entrance captures and stores their facial data. If a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the footage from the facility's security cameras, and if a specific security camera captures the lost child, it notifies them of the child's location. Based on this information, the parents and facility staff can quickly reunite the child.

[1251] This system will enable lost children to be quickly reunited with their parents, improving safety and peace of mind for facility users.

[1252] Example prompts to input to a generative AI model:

[1253] "Please explain how the system works by inputting a photo of the parent's face when a child goes missing and matching it with the security camera footage within the facility to locate the child."

[1254] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1255] Step 1: Acquire facial image data

[1256] The server captures visitors' facial images in real time using a high-resolution facial recognition camera installed at the entrance of the facility. The input data is the video stream from the camera. The server extracts individual facial images from this stream and encodes them into digital data. The converted output data is temporarily stored in the server's memory.

[1257] Specific behavior:

[1258] The server receives the video stream transmitted from the camera in real time.

[1259] The server extracts facial images from the video stream using a facial recognition algorithm.

[1260] The server encodes the extracted facial image and converts it into digital data.

[1261] Step 2: Save the digitized face data

[1262] The server stores the encoded facial data in a database. The input data for this step is the digitized facial data acquired in step 1. The server performs error checks before storing to ensure data integrity. Once integrity is confirmed, the data is stored in the database.

[1263] Specific behavior:

[1264] The server performs error checking on the encoded face data.

[1265] The server generates SQL queries to store the digitized face data in a database.

[1266] The server sends an SQL query to the database and receives a response confirming the save.

[1267] Step 3: Enter a photo of the parents' faces in case the child gets lost

[1268] When a child gets lost, the user (facility staff) uses a dedicated terminal at the lost child center to input a parent's facial photo. The input data for this step is the parent's facial photo. The terminal digitizes the input facial photo and sends the image data to the server. The server then encodes the received facial photo and stores it in a database.

[1269] Specific behavior:

[1270] The user (facility staff) takes a photo of the parent's face using a dedicated terminal.

[1271] The terminal digitizes the captured facial photograph.

[1272] The terminal transmits the digitized facial photograph to the server.

[1273] The server encodes the received parent's facial photograph and stores it in a database.

[1274] Step 4: Verify parent-child relationships

[1275] The server uses an artificial intelligence model to match the facial data of the parent entered with the facial data stored in the database at the entrance. The input data for this step is the facial data of the parent and the facial data acquired at the entrance. The server provides the input data to the AI ​​model and evaluates the degree of match of facial features. If a high match is detected, the server identifies the parent-child relationship.

[1276] Specific behavior:

[1277] The server inputs the parents' facial data into the AI ​​model.

[1278] The AI ​​model compares the parent's facial data with multiple facial data stored in the entrance database.

[1279] The server receives the match assessment from the AI ​​model and identifies parent-child relationships.

[1280] Step 5: Matching with CCTV footage and tracking

[1281] The server captures and analyzes real-time video from security cameras installed in each area of ​​the facility. The input data for this step are the video stream from the security cameras and the parent's facial data. The server applies a facial recognition algorithm to each frame from the security cameras and compares it with the parent's facial data. If a match is confirmed, the server records the location information of that frame.

[1282] Specific behavior:

[1283] The server receives the video streams from each surveillance camera.

[1284] The server applies a facial recognition algorithm to each video frame.

[1285] The server records the position information of the frame where a match is confirmed by comparing it with the parent's face data.

[1286] Step 6: Location Notification

[1287] Once the location of the lost child is identified, the server pushes the information to the lost child center. The input data for this step is the location information of the identified lost child. The server converts the location information into an explainable format and sends it to the lost child center's terminal. The lost child center's terminal receives the notification and displays the location information to facility staff and parents.

[1288] Specific behavior:

[1289] The server converts the identified location information of the lost child into an explainable format.

[1290] The server sends the location information to the lost child center's terminal as a push notification.

[1291] The terminal at the lost child center displays the location information to facility staff and parents based on the received location information.

[1292] (Application example 1)

[1293] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1294] When a child gets lost in a large facility, a system that can quickly and accurately locate the child is extremely important. However, with existing systems, it takes time to identify the lost child, causing anxiety and stress for parents and facility staff. Furthermore, there is a lack of a way for parents to directly obtain information about the lost child's location, further extending the time until the child is reunited. Furthermore, information sharing with lost child centers and security guards is often not smooth. A system that can solve these problems and respond quickly and efficiently when a child gets lost is needed.

[1295] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1296] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a parent's facial photograph when a child gets lost, means for comparing the parent's facial photograph data with the facial data in the database, means for acquiring footage from multiple surveillance cameras within the facility, means for comparing the surveillance camera footage with the parent's facial photograph data to identify the location of the lost child, means for notifying the parent's mobile device of the identified location information of the lost child, and means for sharing the location information notified to the parent's mobile device with the facility's lost child center and security guard terminals in real time. This allows parents to quickly determine the child's location and share the information with the lost child center and security guard, enabling a prompt reunion.

[1297] "Facial image data" refers to facial image information captured by a facial recognition camera or other camera device.

[1298] "Digitalized facial data" refers to facial image data that has been digitally digitized and converted into a form that is easy for a computer to handle.

[1299] A "database" refers to a system that efficiently manages accumulated digital data and quickly searches and obtains necessary information.

[1300] "Parent's face photo" refers to a photo of the face provided by the parent when the child gets lost. This photo is digitized and sent to the server.

[1301] "Matching" refers to the process of comparing one piece of data with another to see if they match.

[1302] A "surveillance camera" refers to a camera installed within a facility to monitor and record the movements of people and objects in real time.

[1303] "Location information of lost children" refers to information showing the current location of a lost child. This information is obtained by comparing video data from surveillance cameras with facial data.

[1304] "Parent's mobile device" refers to a mobile information device such as a smartphone or tablet owned by the parent. This device will be notified of the lost child's location.

[1305] A "lost child center" is a department located within a large facility that specializes in dealing with lost children.

[1306] "Security guard devices" refer to the mobile information devices owned by security guards at the facility. The location information of lost children is also shared with these devices.

[1307] This invention is a system for quickly reuniting parents and children when they get lost in a large facility. This system is composed of a facial recognition camera, a surveillance camera, a parent's mobile device, a lost child center, and a security guard's terminal.

[1308] The server first uses a facial recognition camera installed at the entrance of the facility to obtain facial image data. The facial recognition camera provides high-resolution images that allow the facial recognition system to generate accurate digital values. The facial image data is digitized and transmitted to the server in real time.

[1309] The server stores the captured digitized facial data in a database designed to securely and efficiently manage visitor facial data and enable rapid search and analysis.

[1310] When a child gets lost, the parent enters a facial photo via a dedicated terminal at the lost child center. This photo is then digitized and sent to a server. The server encodes the parent's facial photo data and compares it with facial data in a database. This identifies the parent-child relationship and confirms that the child has entered the facility.

[1311] The server captures and analyzes video footage from multiple surveillance cameras installed within the facility in real time. By comparing the video data from the surveillance cameras with facial photos of the parents, the location of the lost child can be identified. AI technology is used to confirm the degree of match of facial features with high accuracy.

[1312] When a lost child's location is identified, the server notifies the parent's mobile device of the location information. The location information is also shared in real time with the lost child center and security guards' devices. This function allows parents to quickly determine the child's location, and the lost child center and security guards can take action at the same time.

[1313] As an example of how this system can be put into practical use, consider a large shopping mall. As a parent and child pass through the entrance, facial image data is captured by a facial recognition camera. If a child gets lost, the parent provides a facial photo to the lost child center, and the data is collated on a server. After that, an analysis of the surveillance camera identifies the lost child as being in the food court, and the parent's smartphone is notified of the child's location. At the same time, this information is shared with the lost child center and security guards, allowing the parent and child to be quickly reunited.

[1314] An example of an input prompt for a generative AI model is:

[1315] "Please create a system to quickly find children who get lost in a shopping mall. The system will use facial recognition cameras and surveillance cameras to capture and match the facial data of parents and children. Please also implement a function to notify the parents' smartphones of the child's location when the child gets lost."

[1316] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1317] Step 1:

[1318] The server acquires the visitor's facial image data in real time using a facial recognition camera installed at the facility entrance. The facial recognition camera provides high-resolution images and generates accurate digital values ​​in the facial recognition system. The server acquires the facial image data as input and obtains the digitized facial data as output.

[1319] Step 2:

[1320] The server digitizes and encodes the acquired facial image data, converting it into a digital format that is easy for a computer to handle. It receives facial image data as input and obtains encoded facial data as output.

[1321] Step 3:

[1322] The server stores the encoded facial data in a database designed to efficiently manage visitor facial data and enable rapid search and analysis. The server receives digitized facial data as input and stores it in the database.

[1323] Step 4:

[1324] When a child gets lost, the user (parent) enters a facial photo through a dedicated terminal at the lost child center. The parent's facial photo is digitized on the spot and sent to the server. The parent's facial photo is obtained as input, and the digitized parent's facial photo data is obtained as output.

[1325] Step 5:

[1326] The server encodes the parent's facial photo data and matches it with facial data in the database. This process identifies the parent-child relationship and verifies whether the parent and child entered the facility together. The server receives digitized parent facial data as input and provides information on the degree of match between the facial data as output.

[1327] Step 6:

[1328] The server acquires video footage from multiple surveillance cameras installed within the facility in real time. The video data from the surveillance cameras is used to match the parent's facial data. The surveillance camera video is received as input, and the video data for analysis is obtained as output.

[1329] Step 7:

[1330] The server analyzes the surveillance camera footage and compares it with the parent's facial data. This analysis is performed using AI technology to confirm the degree of match of facial features with high accuracy. The server receives the surveillance camera footage and the parent's facial data as input, and obtains specific information about the matching parts as output.

[1331] Step 8:

[1332] The server identifies the location of the lost child and notifies the parent's mobile device of the location information. At the same time, the server also shares the location information with the lost child center and security guards' devices in real time. The server receives the matching part of the location information as input and outputs the location information notification.

[1333] Step 9:

[1334] The parents check the location information of their lost child that is notified to their mobile device, and then use that information to work with facility staff to find the lost child.The location information notification is received as input, and the actual on-site confirmation and response action are output.

[1335] The above are the specific processing steps of the system that realizes the application example.

[1336] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1337] This invention is a system that, when a child gets lost in a large facility, quickly finds the child using facial recognition and video data from a surveillance camera, and further recognizes the emotional states of the parent and child using an emotion engine, and provides appropriate support. Specific embodiments of the present invention will be described in detail below.

[1338] Acquiring facial image data

[1339] The server acquires visitors' facial images in real time from a facial recognition camera installed at the entrance of the facility. The facial images are captured by a high-resolution camera, which allows for detailed analysis of facial feature points.

[1340] Storing digitized facial data

[1341] The server sends the acquired facial image data to the facial recognition system and converts it into digital values. This conversion quantifies facial feature points, enabling subsequent matching. The digitized data is stored in a database and managed along with metadata such as date, time, and camera position.

[1342] Parents' photos can be entered when a child gets lost

[1343] When a child gets lost, the user (facility staff) enters a parent's facial photo into the system via a dedicated terminal at the lost child center. The photo is taken using a high-resolution camera and sent to the server. The server digitizes the parent's facial photo and converts it into encoded data.

[1344] Verifying parent-child relationships

[1345] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database. This process verifies whether the parent and child entered the facility together. The AI ​​checks the degree of similarity of facial features and identifies the parent-child relationship with high accuracy.

[1346] Matching and tracking with surveillance camera footage

[1347] The server acquires video in real time from surveillance cameras installed in each area of ​​the facility. Each surveillance camera periodically transmits video frames to the server. The server analyzes the video frames from the surveillance cameras and compares them with the stored facial data of the parents. By comparing the video data from the surveillance cameras with the facial data of the parents, lost children can be quickly identified.

[1348] Location notification

[1349] If a lost child's location is identified, the server notifies the lost child center with the location information, including the child's detailed location and the latest camera footage, providing parents and facility staff with specific location information and facilitating a speedy reunion.

[1350] Emotion recognition by emotion engine

[1351] The server is equipped with an emotion engine that recognizes the emotional state of parents and lost children in real time. The emotion engine analyzes facial expression and voice data acquired through a facial recognition camera and sensors on the dedicated device to estimate the emotional state. For example, if a parent is feeling stressed, the emotion engine will notify the server based on that information.

[1352] Supported by an emotional engine

[1353] The server provides appropriate support based on the emotional data acquired by the emotion engine. For example, if a parent is in a very unstable state, the server notifies the facility staff of this information and instructs them on how to provide psychological support and take appropriate measures. This reduces the anxiety of both parents and children and supports rapid reunification.

[1354] As a concrete example, consider a large theme park. A facial recognition camera is installed at the entrance, and facial data of visitors is collected when they enter the park. When a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the surveillance camera footage within the park, and if a specific camera captures the lost child, it notifies the location information and the parent's emotional state. Based on this information, the parent and facility staff can quickly reunite, and at the same time, the parent can receive appropriate support according to their emotional state.

[1355] The above is a specific embodiment of the present invention. This system allows lost children to be quickly reunited with their parents, enhancing safety and a sense of security for facility users. Furthermore, the use of an emotion engine reduces anxiety for parents and children, supporting a safe and secure facility experience.

[1356] The processing flow will be explained below.

[1357] Step 1:

[1358] The server acquires visitors' facial images in real time from a facial recognition camera installed at the entrance of the facility. The facial images are captured by a high-resolution camera.

[1359] Step 2:

[1360] The server sends the acquired facial image data to the facial recognition system and converts it into digital values. By quantifying facial features in detail, subsequent matching can be performed.

[1361] Step 3:

[1362] The server stores the digitized facial data in a database, along with metadata such as date, time, and camera position.

[1363] Step 4:

[1364] When a child gets lost, the user (facility staff) enters a photo of the parent's face into the system via a dedicated terminal at the lost child center. The photo is taken using a high-resolution camera.

[1365] Step 5:

[1366] The server digitizes the parent's facial photo data and converts it into encoded data, which is used as the basis for matching lost children.

[1367] Step 6:

[1368] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database. The AI ​​analyzes the degree of similarity of facial features and, if there is a match, recognizes the parent-child relationship.

[1369] Step 7:

[1370] The server acquires video in real time from surveillance cameras installed in each area of ​​the facility, and each surveillance camera periodically transmits video frames to the server.

[1371] Step 8:

[1372] The server analyzes the video frames from the security cameras and compares them with the stored facial data of the parents, checking whether the facial data matches the specific camera footage.

[1373] Step 9:

[1374] The server identifies the child's location from the surveillance camera footage that matches the parent's facial data, and records the identified location as specific area information.

[1375] Step 10:

[1376] The server then notifies the lost child center of the location of the identified lost child, including detailed location information and the latest camera footage.

[1377] Step 11:

[1378] The user (facility staff) will then communicate the location information received at the lost child center to the parents, who can then head to the designated area based on the specific location information and quickly reunite with their child.

[1379] Step 12:

[1380] The server activates an emotion engine that recognizes the emotional state of the parent and the lost child in real time. The emotion engine analyzes facial expression and voice data acquired through a facial recognition camera and sensors on the dedicated device to estimate the emotional state.

[1381] Step 13:

[1382] The server provides appropriate support based on the parent's emotional data analyzed by the emotion engine. For example, if the parent is feeling stressed, the server notifies facility staff of this information and instructs them on how to provide psychological support and take appropriate measures.

[1383] Step 14:

[1384] Based on the analysis results of the emotion engine, the user (facility staff) communicates with the parents in a way that gives them a sense of security, thereby reducing their anxiety and supporting a speedy reunion.

[1385] This series of processes not only enables lost children to be quickly reunited with their parents, but also reduces the anxiety of both parents and children and increases the sense of safety and security for facility users.

[1386] Example 2

[1387] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1388] Having a child get lost in a large facility is a major source of anxiety for parents. It is also extremely important for facility staff to quickly and accurately locate lost children, but conventional methods often require a lot of time and effort. Furthermore, understanding the emotional state of lost children and their parents and providing appropriate support has also been a challenge. Therefore, the objective of this invention is to provide a system that uses facial recognition technology and surveillance camera footage to quickly identify lost children and recognize and respond to the emotional state of parents and children.

[1389] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1390] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a parent's facial photo when a child gets lost, means for using artificial intelligence to compare the parent's facial photo data with the facial data in the database, means for acquiring footage from multiple surveillance cameras within the facility, means for locating the lost child by comparing the surveillance camera footage with the parent's facial photo data, means for notifying the location of the located lost child, means including an emotion engine for recognizing the emotional states of the parent and the lost child in real time, and means for providing appropriate support based on the emotion data acquired by the emotion engine. This enables rapid location identification of the lost child and appropriate support taking into account the emotional states of the parent and child.

[1391] "Means for acquiring facial image data" refers to devices or technologies for capturing high-resolution images of visitors' faces, and specifically includes facial recognition cameras.

[1392] "Means for digitizing facial image data" refers to software or systems for analyzing acquired facial images and converting them into numerical data.

[1393] "Means for storing digitized facial data in a database" refers to technologies or systems that securely and effectively store facial data converted into a numerical data format in a database along with certain metadata.

[1394] "Means for inputting a parent's facial photograph when a child gets lost" refers to a device or method for inputting and transmitting a parent's facial photograph into the system via a dedicated terminal.

[1395] "Means using artificial intelligence to match parent facial photograph data with facial data in the database" refers to a matching system that uses AI technology to compare parent facial photographs with facial data in the database and determine the degree of match with a high degree of accuracy.

[1396] "Means for acquiring footage from multiple surveillance cameras within a facility" refers to systems and technologies for collecting real-time video data from multiple surveillance cameras installed within a facility.

[1397] "Means for locating a lost child by comparing surveillance camera footage with facial photographic data of parents" refers to technology or systems for locating a lost child by comparing surveillance camera footage with facial photographic data of parents.

[1398] "Means for notifying the location of identified lost children" refers to a notification system that quickly notifies facility personnel of the precise location of identified lost children.

[1399] "Means including an emotion engine for recognizing the emotional states of the parent and the lost child in real time" refers to an emotion recognition engine for analyzing facial expressions and voice data and detecting the emotional states of the parent and the lost child in real time.

[1400] "Means for providing appropriate support based on emotional data obtained by the emotion engine" refers to a support system that uses data from the emotion recognition engine to give instructions to facility staff and suggest specific countermeasures.

[1401] This invention is a system that, when a child gets lost in a large facility, quickly finds the child using facial recognition and video data from a surveillance camera, and further recognizes the emotional states of the parent and child using an emotion engine, and provides appropriate support. Specific embodiments are described below.

[1402] System configuration

[1403] The system includes the following hardware and software:

[1404] Hardware

[1405] High-resolution facial recognition camera (captures facial image data)

[1406] Surveillance cameras (capture footage within the facility)

[1407] Dedicated terminal (parent's photo input)

[1408] software

[1409] Facial recognition system (analysis and digitization of facial image data)

[1410] Database (storage of digitized facial data)

[1411] Artificial intelligence (matching parental facial data with facial data in the database)

[1412] Emotion engine (recognition and analysis of emotional states)

[1413] Notification system (notifying the location of identified lost children)

[1414] Process Overview

[1415] 1. Acquiring facial image data

[1416] The server captures visitors' facial images in real time from a high-resolution facial recognition camera, allowing detailed facial feature analysis.

[1417] 2. Storage of digitized facial data

[1418] The server sends the acquired facial image data to a facial recognition system and converts it into numerical data, which is then stored in a database along with metadata.

[1419] 3. Parental photo input in case of lost child

[1420] When a child gets lost, the user (facility staff) enters a parent's facial photo into the system using a dedicated terminal at the lost child center. The entered facial photo is sent to the server and digitized.

[1421] 4. Confirming parent-child relationships

[1422] The server uses artificial intelligence to match the facial data stored in the database with the facial data of the parent entered, making it possible to verify whether the parent and child entered the facility together.

[1423] 5. Matching with surveillance camera footage and tracking

[1424] The server receives real-time footage from surveillance cameras installed in each area and matches it with stored facial data of parents to identify lost children.

[1425] 6. Location Notifications

[1426] If a lost child's location is identified, the server notifies the lost child center with the information, including detailed location information and the latest camera footage.

[1427] 7. Emotion Recognition with Emotion Engine

[1428] The server uses an emotion engine to analyze facial expression and voice data acquired through a facial recognition camera and dedicated terminals, and recognizes the emotional state of parents and lost children in real time.

[1429] 8. Supported by Emotion Engine

[1430] The server provides appropriate support based on the analysis results of the emotion engine. For example, if the server determines that a parent is in an extremely unstable state, it will notify facility staff and instruct them on how to provide psychological support and take appropriate action.

[1431] Specific examples

[1432] Let's take a large theme park as an example. Facial recognition cameras installed at the entrance collect facial data of visitors. When a child gets lost, the user (facility staff) enters a photo of the parent's face at the lost child center, and the server compares it with a face database. The server analyzes the surveillance camera footage within the park, and if a specific camera captures a lost child, it notifies the child's location and the parent's emotional state. Using this information, the parent and facility staff can quickly reunite, and the parent can also receive psychological support.

[1433] Example prompts for generative AI models

[1434] "Please explain the specific process flow of a system that uses video data from facial recognition cameras and security cameras to quickly locate children who become lost in large facilities. Please also provide details on the hardware and software used, the data processing method, and the emotional state recognition by the emotion engine."

[1435] This system allows lost children to be quickly reunited with their parents, increasing the safety and sense of security of facility users.The use of an emotion engine also reduces anxiety for parents and children, allowing them to use the facility with peace of mind.

[1436] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1437] Step 1:

[1438] The server acquires face image data in real time from the face recognition camera.

[1439] Specifically, the facial recognition camera photographs the face of a visitor as they pass through the entrance and immediately sends the image data to a server.

[1440] Input: Real-time facial images of visitors passing through the entrance

[1441] Output: Facial image data sent to the server

[1442] Step 2:

[1443] The server passes the received facial image data to a facial authentication system and digitizes it.

[1444] A facial recognition system analyzes facial feature points and encodes them as numerical data.

[1445] Specifically, the facial recognition system analyzes the position and shape of the eyes, nose, and mouth and converts this into numerical data.

[1446] Input: Facial image data stored on the server

[1447] Output: Digitized face data

[1448] Step 3:

[1449] The server stores the digitized facial data in a database.

[1450] When saving, metadata such as date, time, and camera location are also recorded.

[1451] Specifically, the server stores the numerical data in a database and manages the data efficiently.

[1452] Input: Digitized face data, metadata

[1453] Output: Face data stored in a database

[1454] Step 4:

[1455] When a child gets lost, the user (facility staff) takes a photo of the parent's face using a dedicated terminal and enters it into the system.

[1456] The terminal transmits the captured facial photograph data to the server.

[1457] Specifically, the user takes a photo of the parent's face using a dedicated terminal, and the terminal then sends the data to the server.

[1458] Input: Parent's photo taken by facility staff

[1459] Output: Parent face photo data sent to the server

[1460] Step 5:

[1461] The server receives and digitizes the facial photograph data of the parents.

[1462] The facial recognition system encodes the parent's facial data as numerical data.

[1463] Specifically, the server passes the parent's facial photo data to the facial recognition system, which then analyzes it and converts it into numerical data.

[1464] Input: Parent's facial photo data received by the server

[1465] Output: Digitized parent face data

[1466] Step 6:

[1467] The server uses artificial intelligence to compare the facial data of the parent stored in the database with the facial data of the person at the entrance.

[1468] The AI ​​calculates the degree of similarity of facial features and confirms the parent-child relationship.

[1469] Specifically, the server uses an AI matching system to compare the parent's facial data with the facial data in the database and determine the degree of match.

[1470] Input: Digitized parent face data

[1471] Output: Confirmation of parent-child relationship based on the degree of matching of facial feature points

[1472] Step 7:

[1473] The server acquires real-time images from surveillance cameras installed in each area of ​​the facility.

[1474] Each surveillance camera transmits video frames to the server at regular intervals.

[1475] Specifically, the server continuously receives video data transmitted from multiple surveillance cameras.

[1476] Input: Security camera footage from inside the facility

[1477] Output: Real-time video of each area sent to the server

[1478] Step 8:

[1479] The server compares the surveillance camera footage with the stored facial data of the parents to identify the lost child.

[1480] The surveillance camera video frames are analyzed to detect matches between the facial data of the parent and child.

[1481] Specifically, the server uses a video analysis system to compare surveillance camera footage with the parent's facial data to confirm the location of the lost child.

[1482] Input: Surveillance camera footage, digitized parental facial data

[1483] Output: Location information of the lost child as a result of matching with surveillance camera footage

[1484] Step 9:

[1485] If the location of the lost child is determined, the server notifies the lost child center of this information.

[1486] The notification will include detailed location information and the latest camera footage.

[1487] Specifically, the server generates a notification message based on the identified location information and sends it to the lost child center.

[1488] Input: Matched location of lost child

[1489] Output: Location notification sent to the lost child center

[1490] Step 10:

[1491] The server is equipped with an emotion engine that recognizes the emotional state of the parent and the lost child in real time.

[1492] The emotion engine analyzes facial expression and voice data to estimate the emotional state.

[1493] Specifically, the server uses an emotion engine to analyze the emotional state of the parent and the lost child in real time.

[1494] Input: facial expression data, voice data

[1495] Output: Parsed emotional state data

[1496] Step 11:

[1497] The server provides appropriate support based on the emotion data acquired by the emotion engine.

[1498] For example, if a parent is in a very unstable state, the server will notify facility staff and instruct them on psychological support and appropriate measures.

[1499] As a specific operation, the server analyzes the emotion data and notifies the facility staff of the specific support required.

[1500] Input: Emotional state data

[1501] Output: Support instruction notification to facility staff

[1502] (Application example 2)

[1503] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1504] When a child gets lost in a large facility, there is a need to quickly find the child and reunite them with their parents safely and securely. However, current systems take time to locate a lost child, making it difficult to alleviate the anxiety of the parent and child. Furthermore, simply providing location information without considering the emotional state of the parent and child does not provide sufficient support. Therefore, there is a need for a system that can quickly locate a lost child while also providing appropriate support based on the emotional state of the parent and child.

[1505] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1506] In this invention, the server includes means for acquiring facial image data, means for digitizing the facial image data, means for storing the digitized facial data in a database, means for inputting a facial photo of a parent when a child gets lost, means for comparing the facial photo data of the parent with the facial data in the database, means for acquiring footage from multiple surveillance cameras within the facility, means for comparing the surveillance camera footage with the facial photo data of the parent to identify the location of the lost child, means for notifying the location information of the identified lost child, emotion analysis means for recognizing the emotional states of the parent and the lost child, and means for providing support based on the emotional states, thereby enabling the quick discovery of a lost child and support according to the emotional states of the parent and child.

[1507] The "means for acquiring facial image data" refers to a device and method for capturing facial images of visitors using a facial recognition camera installed at the entrance of the facility or in each area.

[1508] The "means for digitizing the facial image data" refers to a process and software for analyzing the acquired facial image and converting facial feature points into digital values.

[1509] The "means for storing the digitized facial data in a database" refers to a system and method for storing facial data converted into digital values ​​in a database and adding metadata such as date, time, and camera position.

[1510] The "means for inputting a parent's facial photograph when a child gets lost" refers to a device and method for taking a parent's facial photograph using a dedicated terminal or smart device and inputting it into the system.

[1511] The "means for matching parent's facial photograph data with the facial data in the database" refers to the process and software for using artificial intelligence to compare and match parent's facial photograph data with the facial data obtained upon entry into the facility.

[1512] The "means for acquiring images from multiple surveillance cameras within a facility" refers to a system and method for transmitting images from surveillance cameras installed in each area within a facility to a server in real time.

[1513] The "means for locating a lost child by comparing the surveillance camera footage with the facial photograph data of the parent" refers to the process and software for comparing and matching the surveillance camera footage with the facial photograph data of the parent to locate a lost child.

[1514] The "means for notifying the location information of the identified lost child" refers to a system and method for notifying the location information of the identified lost child to a lost child center or a smart device.

[1515] The "emotion analysis means for recognizing the emotional states of parents and lost children" is a device and method that uses an emotion engine to analyze facial expression data and voice data acquired and estimate the emotional states of parents and children.

[1516] The "means for providing support based on the emotional state" refers to a system and method for providing appropriate psychological support and responses based on the emotional states of the parent and the lost child.

[1517] The present invention is a system that, when a child gets lost in a large facility, quickly finds the child by utilizing facial recognition and video data from a surveillance camera, and further recognizes the emotional states of the parent and child using an emotion engine, and provides appropriate support. Specific embodiments of the present invention will be described in detail below.

[1518] Acquiring facial image data

[1519] The server captures visitors' facial images in real time through a facial recognition camera installed at the entrance of the facility. This camera has high resolution and can analyze facial feature points in detail. The facial images are converted into digital values ​​and stored in a database.

[1520] Storing digitized facial data

[1521] The server sends the acquired facial image data to a facial recognition system (e.g., Amazon Rekognition, Microsoft Azure Face API) and converts it into digital values. This conversion digitizes facial feature points, facilitating subsequent matching. The digitized data is stored in a database using cloud services such as AWS or Google Cloud. When the data is saved, metadata such as the date, time, and camera position is also added.

[1522] Parents' photos can be entered when a child gets lost

[1523] When a child gets lost, the user (facility staff) uses a dedicated terminal or smart device to input a parent's facial photo into the system. The parent's facial photo is taken with a high-resolution camera and sent to the server. The server digitizes the parent's facial photo and converts it into encoded data.

[1524] Verifying parent-child relationships

[1525] The server uses artificial intelligence (AI) to compare the facial data stored in the database at the entrance with the facial data of the parent entered, making it possible to confirm with high accuracy whether the parent and child entered the facility together.

[1526] Matching and tracking with surveillance camera footage

[1527] The server captures video in real time from multiple surveillance cameras installed within the facility. Each surveillance camera periodically transmits video frames to the server. The server analyzes the video frames and matches them with the facial data of the parents to identify the lost child.

[1528] Location notification

[1529] If the server locates a lost child, the information is sent to a dedicated terminal or smart device, along with detailed location information and the latest camera footage, helping parents and facility staff quickly reunite the child.

[1530] Emotion recognition by emotion engine

[1531] The server recognizes the emotional state using an emotion engine (e.g., Affectiva Emotion AI, Microsoft Azure Text Analytics). This engine analyzes facial expression and voice data acquired from a facial recognition camera and sensors on the dedicated device to estimate the emotional state of the parent and lost child.

[1532] Supported by an emotional engine

[1533] The server provides appropriate support based on the emotional data acquired by the emotion engine. For example, if a parent is extremely unstable, the server notifies facility staff of this information and instructs them on how to provide psychological support and take appropriate measures.

[1534] Specific examples and prompts for generative AI models

[1535] As a concrete example, consider a large shopping mall. A facial recognition camera is installed at the entrance, and facial data of visitors is collected. When a child gets lost, the parent uses smart glasses to go to the lost child center and input a photo of their face. The server matches the parent and child's faces, and by analyzing the surveillance camera footage, locates the child's location and notifies the smart glasses.

[1536] Example prompts for generative AI models:

[1537] ---

[1538] Please provide a detailed description of a child loss prevention system that uses facial recognition and security cameras. Explain the functionality of the smart glasses application, which compares the facial data of parents and children to quickly identify a child if they get lost and notify the parent of their location and emotional state. Include the specific processing steps and technologies used.

[1539] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1540] Step 1: Acquire facial image data

[1541] The server captures visitors' facial images through high-resolution facial recognition cameras installed at the entrances of the facilities. The input is facial image data, which is then converted into a digital format. Specifically, the server takes the facial image captured by the camera sensor as initial data, and processes the high-quality image through an AI-based facial recognition algorithm.

[1542] Step 2: Digitizing the face data

[1543] The server sends the acquired facial image data to the facial recognition system, where an AI algorithm analyzes the facial feature points and converts them into digital values. The input is the captured facial image, and the output is the digitized facial feature point data. This process uses a high-speed image analysis algorithm and is performed in real time.

[1544] Step 3: Save the digitized data

[1545] The server stores the digitized facial feature data in a cloud database, along with metadata such as date, time, and camera position. The input is the digitized facial feature data and metadata, and the output is a record in the database.

[1546] Step 4: Enter a photo of the parents' faces in case the child gets lost

[1547] The user (facility staff) uses a dedicated terminal or smart device to input a facial photo of the parent of the lost child into the system. The parent's facial photo is taken with a high-resolution camera and sent to the server. The input is the parent's facial image data, and the output is digitized facial feature point data of the parent.

[1548] Step 5: Verify parent-child relationships

[1549] The server uses AI to compare the facial data of the parent entered with the facial data stored in the database at the entrance. The input is the parent's digitized facial data and the facial data in the database, and the output is the parent-child relationship identification result with a high degree of match. In this process, the AI ​​compares facial feature points and determines the parent-child relationship with a high degree of accuracy.

[1550] Step 6: Matching with CCTV footage and tracking

[1551] The server receives real-time video footage from multiple surveillance cameras installed within the facility. It analyzes these videos and matches them with the facial data of the parents to identify lost children. The input is the video frames from each surveillance camera and the parents' digitized facial data, and the output is the identified location information of the lost child.

[1552] Step 7: Location Notification

[1553] When the server identifies the location of a lost child, it notifies the dedicated terminal or smart device of that information. The notification includes the latest camera footage and detailed location information. The input is the location information and video data of the lost child, and the output is a notification message.

[1554] Step 8: Emotion Recognition with the Emotion Engine

[1555] The server uses an emotion engine to recognize the emotional state of the parent and the lost child in real time. The input is facial expression and voice data acquired from a facial recognition camera and sensors on a dedicated device, and the output is an estimated emotional state. This allows for a detailed understanding of the emotional states of the parent and child.

[1556] Step 9: Support from the Emotion Engine

[1557] The server provides appropriate support based on the emotional data acquired by the emotion engine. For example, if a parent is in a very unstable state, the server notifies facility staff of this information and instructs them on how to provide psychological support and appropriate measures for the parent. The input is emotional data, and the output is instructions and provision of support measures.

[1558] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1559] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1560] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1561] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1562] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1563] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1564] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1565] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1566] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1567] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1568] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1569] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1570] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1571] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1572] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1573] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1574] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or ...

Claims

1. means for acquiring face image data; means for digitizing the facial image data; means for storing the digitized facial data in a database; A way to input a parent's photo when a child gets lost, means for comparing parent facial photograph data with facial data in the database; A means for acquiring images from multiple surveillance cameras within the facility; a means for collating the surveillance camera image with the facial photograph data of the parent to identify the location of the lost child; a means for notifying the location information of the identified lost child; A system including:

2. The means for acquiring face image data includes a face recognition camera. The system of claim 1 .

3. The means for comparing the parent's facial photograph data with the facial data in the database is configured to use artificial intelligence. The system of claim 1 .

Images