System

Abstract

A system is provided.SOLUTION: A system comprising: means for acquiring image data of a missing person; means for operating a generative model to generate predictive facial image data from the image data; means for storing the generated predictive facial image data; means for acquiring video data in real time from a surveillance camera; means for extracting a facial image from the video data; means for matching the extracted facial image with the predictive facial image data and calculating a match rate; and means for generating and notifying an alert if the match rate exceeds a certain threshold.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The number of people who go missing due to dementia and other reasons is increasing year by year, reaching approximately 18,709 cases nationwide in 2022. However, current search methods only have a detection rate of 80%, making it difficult to quickly find missing people and reunite them with their families. To solve this problem, there is a need for technology that can efficiently grasp the facial features of missing people and identify them in real time. [Means for solving the problem]

[0005] The present invention provides a system that uses a generative model to generate predictive facial image data based on facial image data of a missing person, taking into account changes over time, such as changes due to cosmetic surgery, etc. Specifically, the system includes means for acquiring image data of the missing person, means for operating a generative model for generating predictive facial image data from the image data, means for saving the generated predictive facial image data, means for acquiring video data from a surveillance camera in real time, means for extracting a facial image from the video data, means for matching the extracted facial image with the predictive facial image data and calculating the match rate, and means for generating an alert and sending a notification when the match rate exceeds a certain threshold, thereby enabling the system to quickly and efficiently locate the missing person.

[0006] "Missing person image data" refers to a still image or video frame containing the face of a missing person.

[0007] A "generative model" refers to an artificial intelligence model that uses facial image data of a missing person as input, predicts changes in facial features due to aging, cosmetic surgery, etc., and generates new facial image data.

[0008] "Predicted facial image data" refers to future facial image data that is generated from the original facial image data using a generative model, taking into account changes such as aging and cosmetic surgery.

[0009] A "surveillance camera" refers to a video camera that is installed at designated locations such as stations and public institutions and captures footage in real time.

[0010] "Video data" refers to video data captured in real time from a surveillance camera.

[0011] A "facial recognition algorithm" refers to a computational method for extracting facial features from video or image data and identifying the face of a specific person.

[0012] "Matching" refers to the process of comparing and matching common features between different data sets and calculating the degree of agreement.

[0013] The "match rate" is a score indicating the commonality between two pieces of facial image data, and is an index for evaluating the degree of possibility that they are the same person.

[0014] An "alert" is a warning signal that occurs when the match rate exceeds a certain threshold, and refers to a notification that a person believed to be missing has been found.

[0015] "Terminal" refers to any device used to operate the system, including in particular computers and mobile devices used by police and related agencies. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system for improving the rate at which missing persons are found. Its purpose is to efficiently find missing persons by generating predicted facial image data using a generative model based on facial image data of the missing person and then comparing this with video data from surveillance cameras installed in public institutions.

[0038] First, family members or other people involved (hereinafter referred to as "users") upload facial image data of the missing person to the system. This facial image data is a past photo of the person who went missing due to dementia or other factors, and in most cases, multiple photos are provided.

[0039] The server stores the facial image data received from the user in a database and then performs preprocessing, such as resizing the image, removing noise, and adjusting color tones, to convert it into a format suitable for input to the generative model.

[0040] The preprocessed facial image data is input into a generative model on the server. This generative model generates multiple predicted facial image data, taking into account changes such as aging and cosmetic surgery. These predicted facial image data are important for estimating the current appearance of the missing person.

[0041] Next, the generated predicted facial image data is stored on a server and linked to a surveillance camera system. Multiple surveillance cameras continuously capture video data in real time at stations and public places. The surveillance camera terminals then transmit this video data to the server in real time.

[0042] The server applies a facial recognition algorithm to extract facial images from the received video data. The extracted facial images are compared with predicted facial image data stored in a database to calculate a match rate. If the match rate exceeds a certain threshold, the server generates an alert.

[0043] The alert will include the matched facial image, the CCTV camera location, and the time the match was detected. The alert will be sent to a police device. The police device will receive the alert and provide on-site information to response teams. Police will then arrive at the scene to identify and protect the individual. After identification, police will contact the family and arrange for the missing person to be reunited with their family.

[0044] Specific examples

[0045] For example, consider a case where a person with dementia has gone missing. Family members upload past photos of the person to the system, along with basic information such as age and gender. The server preprocesses these images and generates multiple facial images that are input into a generative model to predict their current appearance. Then, facial images are extracted from real-time footage captured by surveillance cameras at station ticket gates and compared with the stored predicted facial images. If there is a high match, the server sends an alert to the police, who respond to the scene and protect the person. The family is then immediately contacted to ensure the reunion.

[0046] The above is an embodiment of the present invention, and it is expected that this process will significantly improve the early detection of missing persons and reunification with their families.

[0047] The processing flow will be explained below.

[0048] Step 1:

[0049] Users upload facial image data and basic information (such as name, age, gender, and last seen location) of missing people to the system, which is used to accurately understand the latest status of missing people.

[0050] Step 2:

[0051] The server stores the facial image data received from the user in a database in an appropriate format to maintain the quality of the image data.

[0052] Step 3:

[0053] The server preprocesses the stored facial image data, specifically by resizing the image, removing noise, adjusting color, etc., and converting it into a format suitable for the generative model.

[0054] Step 4:

[0055] The server inputs the preprocessed facial image data into a generative model. This generative model generates predicted facial image data, taking into account factors such as aging and cosmetic surgery. The generated predicted facial image data has multiple variations, each reflecting a different possibility.

[0056] Step 5:

[0057] The server stores the generated predicted facial image data in a database, which makes it available for subsequent facial image matching processes.

[0058] Step 6:

[0059] The terminal (surveillance camera system) acquires video data in real time. These surveillance cameras are installed in many public institutions and facilities and continuously transmit the video data to a server.

[0060] Step 7:

[0061] The server receives real-time video data transmitted from the surveillance cameras, which is then analyzed using a facial recognition algorithm to extract facial images.

[0062] Step 8:

[0063] The server compares the facial images extracted from the real-time video data with the stored predicted facial image data. A matching algorithm calculates the match rate and evaluates the likelihood of identifying the person.

[0064] Step 9:

[0065] The server generates an alert when the match rate exceeds a set threshold, and the alert includes the matching face image, the surveillance camera's location information, and time information.

[0066] Step 10:

[0067] The server sends the generated alert to the police terminal, which can then respond immediately to the incident.

[0068] Step 11:

[0069] After receiving the alert, police will head to the scene, identify the person, and take them into custody. After that, the police will confirm that the person is a missing person and contact their family.

[0070] Step 12:

[0071] Police will arrange for the missing person to be reunited with their family and return them safely home, a process that will ensure the early detection and reunion of missing people.

[0072] Example 1

[0073] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0074] Locating missing people often becomes more difficult as time passes. This becomes even more difficult when changes in appearance or age occur due to factors such as dementia. Technology is also needed to efficiently analyze video data acquired in real time from multiple surveillance devices. The present invention aims to solve these problems and improve the rate at which missing people are found.

[0075] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0076] In this invention, the server includes a means for preprocessing image data of the missing person and converting it into an appropriate format, a means for operating a generative model for generating predicted facial image data from the preprocessed image data, and a means for saving the generated predicted facial image data. This enables the generation of predicted facial image data that takes into account changes over time, such as changes in the patient's appearance or surgical procedures. The server also includes a means for acquiring video data from multiple monitoring devices in real time, extracting facial images from the video data, comparing them with the predicted facial image data, and calculating the match rate. This allows for the prompt generation of an alert and notification when the match rate exceeds a certain threshold.

[0077] "Missing person" refers to a person whose whereabouts are unknown.

[0078] "Image data" refers to data that digitally represents visual information, including the face of a missing person.

[0079] "Preprocessing" refers to processes such as resizing, noise removal, and color adjustment that are performed to convert image data into a format suitable for the generative model.

[0080] A "generative model" refers to artificial intelligence that generates predicted facial image data based on input past image data, taking into account changes over time, such as changes in appearance or surgical procedures.

[0081] "Predicted facial image data" refers to image data that predicts the current appearance of a missing person, generated by a generative model.

[0082] "Monitoring equipment" refers to devices installed at stations and public places that capture video data in real time.

[0083] "Video data" refers to real-time image and video data captured by a monitoring device.

[0084] "Facial image" refers to image data that includes the facial portion of a person extracted from video data.

[0085] "Matching" refers to the process of comparing the extracted facial image with predicted facial image data and calculating the match rate.

[0086] The "match rate" refers to the ratio indicating the degree to which the matched face image and predicted face image data match.

[0087] An "alert" refers to warning information that is generated when the match rate exceeds a certain threshold.

[0088] "Notification" refers to the process of communicating generated alerts to police and other relevant parties.

[0089] The present invention is a system for improving the rate at which missing persons are found. Specifically, the system generates predicted facial image data using a generative model based on facial image data of the missing person, and then compares this with video data from surveillance equipment installed in public institutions, thereby efficiently finding the missing person. The overall flow of the system will be explained in detail below, with the roles of the user, server, and terminal.

[0090] First, the user uploads facial image data of the missing person to the system. This facial image data is a past photograph of the missing person, and usually multiple photographs are provided. The user accesses a dedicated page of the system using a web browser, uploads any image of the missing person they have, and enters basic information such as the missing person's name, age, and gender.

[0091] Next, the server stores the facial image data received from the user in a database and performs preprocessing, including resizing the image, removing noise, and adjusting color tone, before converting it into a format suitable for input to the generative model. The hardware used is a server with high-performance computing power and large storage capacity (e.g., the Dell PowerEdge series), and the software used is a facial recognition library such as OpenCV or Dlib.

[0092] The facial image data after preprocessing is input into a generative model on the server. This generative model generates multiple predicted facial image data to estimate the missing person's current appearance, taking into account changes over time, such as changes due to surgery, etc. A generative AI model such as StyleGAN is used as this generative model. The generated predicted facial image data is stored on the server.

[0093] The server then connects to multiple monitoring devices. These devices continuously capture video data in real time at stations and public places and send it to the server. The terminals (monitoring devices) use a surveillance system that includes high-resolution cameras (e.g., cameras manufactured by Axis Communications).

[0094] The server applies a facial recognition algorithm to extract facial images from the received video data. The extracted facial images are compared with predicted facial image data stored in a database to calculate the match rate. Facial recognition algorithms used include OpenCV and Dlib. If the match rate exceeds a certain threshold, the server generates an alert. The alert includes the matched facial image, the location of the surveillance device, and the time the match was detected.

[0095] The alert is sent to the police terminal, which is a sophisticated computer system that receives the alert in real time. Once the police receive the alert, they will dispatch a response team to the scene to identify and protect the individual. After identification, the police will contact the family and arrange for the missing person to be reunited with their family.

[0096] Specific examples

[0097] For example, consider a case where a person with dementia goes missing. Family members upload past photos of the person to the system and enter basic information such as age and gender. The server preprocesses these images and generates multiple facial images that are input into a generative model to predict their current appearance. Then, a surveillance device extracts facial images from real-time footage taken at station ticket gates and compares them with the stored predicted facial images. If there is a high match, the server sends an alert to the police, who respond to the scene and protect the person. The family is then immediately contacted to ensure the reunion.

[0098] Example prompt sentence:

[0099] 1. "Generate facial image data to predict current appearance based on a facial photo taken 40 years ago."

[0100] 2. "Generate a current facial image of a missing person based on past photographs, taking into account age-related changes."

[0101] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0102] Step 1:

[0103] The user uploads facial image data of the missing person.

[0104] Input: Past facial images of the missing person, basic information such as name, age, and gender

[0105] Output: Sending face image data and basic information to the server

[0106] Specific operations: The user opens a web browser and accesses the system's dedicated page. They click the "Upload Images" button, select and upload multiple past photos of the missing person they have, and then enter basic information about the missing person into the form.

[0107] Step 2:

[0108] The server stores the facial image data received from the user in a database and performs preprocessing.

[0109] Input: Facial image data and basic information sent to the server

[0110] Output: Preprocessed image data stored in a database

[0111] Specific operation: The server records the received facial images in a database. Then, it performs preprocessing such as resizing, noise removal, and color adjustment to convert the images into a format suitable for input to the generative model.

[0112] Step 3:

[0113] The server inputs the preprocessed facial image data into a generative model to generate predicted facial image data.

[0114] Input: Preprocessed face image data

[0115] Output: Generated predicted face image data

[0116] Specific operation: The server inputs the preprocessed facial image into the generative model. The generative model generates multiple predicted facial image data, taking into account changes over time, such as aging and surgical procedures. The prompts used are, "Based on a facial photo from 40 years ago, please generate facial image data to predict current appearance," and "Based on past photographs of a missing person, please generate a current facial image that takes into account changes due to age."

[0117] Step 4:

[0118] The server stores the generated predicted face image data in a database.

[0119] Input: Generated predicted face image data

[0120] Output: Predicted face image data stored in a database

[0121] Specific operation: The server stores the generated predicted face image data in a database.

[0122] Step 5:

[0123] The terminal (monitoring device) transmits video data to the server in real time.

[0124] Input: Real-time video data captured by a surveillance camera

[0125] Output: Real-time video data sent to the server

[0126] Specific operation: The terminal (monitoring device) streams video footage taken at station ticket gates and public places to a server in real time.

[0127] Step 6:

[0128] The server extracts a facial image from the video data and compares it with predicted facial image data.

[0129] Input: Real-time video data

[0130] Output: Matching result (match rate)

[0131] Specific operation: The server applies a facial recognition algorithm to the received video data to extract facial images. The extracted facial images are compared with the stored predicted facial image data and the match rate is calculated. OpenCV and Dlib are used here.

[0132] Step 7:

[0133] The server generates an alert and notifies you if the match rate exceeds a certain threshold.

[0134] Input: Matching result (match rate)

[0135] Output: Alert information

[0136] Specific operation: If the calculated match rate exceeds a threshold, the server generates an alert. The alert includes the matched face image, the location information of the monitoring device, and the time when the match was detected. The generated alert is sent to the police terminal.

[0137] Step 8:

[0138] Police receive an alert, provide on-site information to response units, and initiate action.

[0139] Input: Alert information sent from the server

[0140] Output: On-site confirmation and protective action

[0141] Specific operations: The police terminal receives the alert and provides the relevant scene information to the response team. The police then go to the scene to identify and protect the person in question, and after confirmation, contact the family. Finally, they arrange for the missing person to be reunited with their family.

[0142] This concludes the detailed explanation of each processing step in the system program, which will enable efficient detection of missing persons and prompt response.

[0143] (Application example 1)

[0144] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0145] There is a need for an efficient and rapid method to improve the rate at which missing persons are found. However, current systems only use public surveillance cameras, and efficient searches using mobile information devices owned by individuals or at home are not performed. Furthermore, there is a lack of reliable predictive methods for responding to changes in facial images. As a result, searches for missing persons are delayed and the rate at which they are found decreases. Therefore, there is a need for a rapid and widespread method for finding missing persons.

[0146] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0147] In this invention, the server includes means for acquiring image data of a missing person, means for operating a generative model for generating predicted facial image data from the image data, means for saving the generated predicted facial image data, means for acquiring video data in real time from a surveillance camera or a camera of a mobile information terminal, means for extracting a facial image from the video data, means for comparing the extracted facial image with the predicted facial image data and calculating a match rate, means for generating and notifying an alert when the match rate exceeds a certain threshold, and means for sharing location information based on the alert, thereby enabling a wide-area search for missing people in real time.

[0148] A "missing person" is someone whose current whereabouts are unknown and who requires a search.

[0149] "Image data" refers to digital data that includes past photographs of faces and other visual information about missing persons.

[0150] A "generative model" is an artificial intelligence model that predicts and generates current facial image data from input past facial image data.

[0151] A "personal digital assistant" is a portable device such as a smartphone or tablet.

[0152] A "surveillance camera" is a camera device installed in a public place that continuously records and monitors footage.

[0153] "Acquiring video data in real time" means that the video obtained from the camera is immediately sent to the system for processing.

[0154] "Extracting a face image" means cutting out and acquiring only the face portion of a person from video data.

[0155] "Matching" means comparing and verifying a match against different data sets.

[0156] The "match rate" is a ratio indicating the degree to which the extracted face image and the predicted face image data are identical.

[0157] An "alert" is a warning or notification that is generated when the match rate exceeds a certain threshold.

[0158] "Sharing location information" means sending and receiving geographical data of the location where a missing person was found between multiple devices.

[0159] The present invention is a system for efficiently locating missing persons. This system consists of three main components: a server, a terminal, and a user.

[0160] First, the user uploads past facial image data of the missing person to the system. This can be done using a device such as a smartphone or PC. The image data provided by the user is sent to the server. The server receives this data and performs preprocessing. This preprocessing involves resizing the image, removing noise, and adjusting the color tone. This process converts the data into a format suitable for the generative model.

[0161] Next, a generative model generates predicted facial image data from the preprocessed facial image data. This generative model outputs multiple predicted facial images, taking into account facial changes such as aging and cosmetic surgery. This predicted facial image data is stored in a database on the server.

[0162] The server receives video data in real time from surveillance cameras and mobile devices. For example, when a smartphone camera captures video in a public place, the video data is sent to the server. The server then applies a facial recognition algorithm to extract facial images from the video data. This algorithm can use models such as InceptionResnetV1 or MTCNN.

[0163] The extracted facial image is compared with predicted facial image data stored in the server's database. At this time, the match rate is calculated. If the match rate exceeds a certain threshold, the server generates an alert. This alert includes the matched facial image, detected location information, and time information. The alert is sent to the devices of the police and relevant parties.

[0164] Additionally, the alert notification includes a location sharing function, enabling a rapid response if a missing person is found. For example, the smartphone app can send geographical data of the detected location to a server, allowing police or family members to receive the data and take action.

[0165] Examples:

[0166] Below are some example prompts for the generative AI model:

[0167] "Generate a current image of a 50-year-old woman's face from an image taken 20 years ago."

[0168] "Please output an image of a 20-year-old man's face converted to look like a 30-year-old man."

[0169] This enables wide-area and real-time searches for missing people. The server calculates the facial image match rate and can quickly generate an alert and notify relevant parties if a match exceeds a certain threshold. This enables the early detection and protection of missing people and promotes reunification with their families.

[0170] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0171] Step 1:

[0172] The user uploads the missing person's past facial image data to the system. The user uses a device such as a smartphone or PC to send the image data to the server. The input at this time is the missing person's past facial image data, and the output is the data transferred to the server.

[0173] Step 2:

[0174] The server preprocesses the facial image data received from the user. It resizes the image, removes noise, adjusts color tone, and converts the data format into a format that can be input to the generative model. The input of this process is the uploaded facial image data, and the output is the preprocessed facial image data.

[0175] Step 3:

[0176] The server inputs the preprocessed facial image data into a generative model, which generates multiple predicted facial image data by taking into account factors such as aging and cosmetic surgery. The input is the preprocessed facial image data, and the output is predicted facial image data.

[0177] Step 4:

[0178] The generated predicted facial image data is stored in a database on the server. This database is used in subsequent facial recognition processes. The input is the predicted facial image data, and the output is the data stored in the database.

[0179] Step 5:

[0180] A camera on a device (e.g., a smartphone) or a public surveillance camera captures video data in real time. The captured video data is sent to a server. The input is the real-time video data, and the output is the data sent to the server.

[0181] Step 6:

[0182] The server applies a facial recognition algorithm to extract facial images from the transmitted video data, such as InceptionResnetV1 or MTCNN. The input of this process is real-time video data, and the output is extracted facial image data.

[0183] Step 7:

[0184] The extracted facial image data is compared with predicted facial image data stored in the server's database, and a match rate is calculated. This calculation confirms whether the image matches the missing person. The input is the extracted facial image data and predicted facial image data, and the output is the calculated match rate.

[0185] Step 8:

[0186] If the match rate exceeds a certain threshold, the server generates an alert, which includes the matched face image, location information, and time information. The input is the situation where the match rate exceeds a certain threshold, and the output is the generated alert.

[0187] Step 9:

[0188] The generated alert is sent to the devices of the police and other relevant parties, and a response is initiated at the scene. The input is the generated alert, and the output is the notified information. The police and other relevant parties then proceed to the scene based on the location information to identify and protect the missing person.

[0189] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0190] The present invention is a system for improving the rate at which missing persons are found, and aims to efficiently find missing persons by generating predicted facial image data using a generative model based on facial image data of the missing person and comparing this with video data from surveillance cameras installed in public institutions.Furthermore, the system is characterized by improving the responsiveness of the system by combining it with an emotion engine that recognizes the user's emotions.

[0191] First, family members and other people involved (hereafter referred to as "users") upload facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. The emotion engine then analyzes the user's emotions in real time and optimizes the operation of the entire system based on that information.

[0192] The server stores the facial image data received from the user in a database and then preprocesses it. This involves resizing the image, removing noise, adjusting color, and converting it into a format suitable for input to the generative model. The emotion engine also obtains the user's emotional data during this process, which can be used as additional information to help search for missing people.

[0193] The facial image data after preprocessing is input into a generative model on the server. This generative model generates multiple predicted facial image data, taking into account changes such as aging and cosmetic surgery. The generated predicted facial image data is important for estimating what the missing person currently looks like.

[0194] Next, the generated predicted facial image data is stored on a server and linked to a surveillance camera system. Multiple surveillance cameras continuously capture video data in real time at stations and public places. The surveillance camera terminals then transmit this video data to the server in real time.

[0195] The server applies a facial recognition algorithm to extract facial images from the received video data. An emotion engine then analyzes the user's emotional data and sets optimal parameters for searching for the missing person. The extracted facial images are compared with predicted facial image data stored in a database to calculate the match rate. If the match rate exceeds a certain threshold, the server generates an alert.

[0196] The alert includes the matched facial image, the surveillance camera's location, and the time the match was detected. The alert is sent to the police device. The emotion engine takes the user's emotional state into account to customize the alert content and notification method. The police device receives the alert and provides on-site information to response teams. The police then arrive at the scene to identify and protect the person. After identification, the police contact the family and arrange for the missing person to be reunited with their family.

[0197] Specific examples

[0198] For example, consider a case where a person with dementia has gone missing. Family members upload past photos of the person to the system, along with basic information such as age and gender. The server preprocesses these images and inputs them into a generative model to generate multiple facial images for predicting their current appearance. The emotion engine also analyzes the family's emotional state of concern and adjusts the system's overall responsiveness and alertness. Next, facial images are extracted from real-time footage captured by surveillance cameras at station ticket gates and compared with the stored predicted facial images. If the match rate is high, the server sends an alert to the police, who then dispatch to the scene to protect the person. The family is then immediately contacted, enabling the reunion. The emotion engine's analysis of the user's emotions also allows for efficient care and support after the reunion.

[0199] The above is an embodiment of the present invention, and by combining this process with an emotion engine, it is expected that the early discovery and reunion of missing persons will proceed even more smoothly.

[0200] The processing flow will be explained below.

[0201] Step 1:

[0202] The user uploads facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. At the same time, the user inputs emotional data (such as worry, anxiety, and impatience). The emotional data can be in text format or voice input.

[0203] Step 2:

[0204] The server stores the facial image data and emotion data received from the user in a database, with the facial image data stored in high quality for subsequent processing.

[0205] Step 3:

[0206] The server preprocesses the facial image data, specifically resizing the image, removing noise, adjusting color, etc. The preprocessed facial image data is then converted into a suitable format for input to the generative model.

[0207] Step 4:

[0208] The server inputs the preprocessed facial image data into a generative model, which then generates multiple predicted facial image data, taking into account factors such as aging and cosmetic surgery.

[0209] Step 5:

[0210] The server stores the generated predicted facial image data in a database. At this stage, the system has multiple facial image variations that can be used to more easily identify missing people.

[0211] Step 6:

[0212] The server uses an emotion engine to analyze the user's emotion data. Based on the analysis results, the system adjusts its responsiveness and alertness. For example, if the user shows a high level of impatience, the system will use a high-precision model to generate predicted facial images more quickly.

[0213] Step 7:

[0214] The terminals (surveillance camera systems) acquire video data in real time. These surveillance cameras are installed in stations and public places, and continuously transmit the video data to a server.

[0215] Step 8:

[0216] The server receives real-time video data transmitted from the surveillance cameras, which is analyzed using a facial recognition algorithm to extract facial images.

[0217] Step 9:

[0218] The server compares the extracted facial image with predicted facial image data stored in a database, calculates the match rate, and generates an alert if the match rate exceeds a certain threshold.

[0219] Step 10:

[0220] The server then sends the generated alert to the police terminal, which includes the matching face image, the location information of the surveillance camera, and the time when the match was detected.

[0221] Step 11:

[0222] Police devices receive the alert and provide on-site information to responding teams, who then arrive on-site to identify and protect the individual.

[0223] Step 12:

[0224] Once the police confirm that the person is missing, they will contact the family. Based on the user's emotional state, the emotion engine will customize the notification content and contact the user in the most appropriate way to reassure them.

[0225] Step 13:

[0226] Police will arrange for missing people to be reunited with their families and return home safely, and the emotion engine will analyze the user's emotions and coordinate support and care after the reunion.

[0227] These steps will ensure that missing persons are quickly identified and reunited with their families.

[0228] Example 2

[0229] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0230] The present invention aims to improve the rate at which missing people are found, and to solve the problems of cases where finding a missing person is difficult using conventional search methods, or where searches take a long time. Furthermore, since a flexible response that takes into account the user's emotions is required, a system that can optimize search activities and provide care after a person is found is also needed.

[0231] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring image data of a missing person, means for operating a generative model for generating predicted face image data from the image data, means for saving the generated predicted face image data, means for acquiring video data from a surveillance camera in real time, means for extracting a face image from the video data, means for matching the extracted face image with the predicted face image data and calculating a match rate, means for generating and notifying an alert when the match rate exceeds a certain threshold, means for analyzing emotional data acquired from a user, and means for optimizing the operation of the entire system based on the emotional data. This enables the missing person to be found quickly and efficiently, and also allows for flexible responses that take user emotions into consideration.

[0232] "Image data of a missing person" is digital data that shows the facial image and physical characteristics of a missing person.

[0233] A "generative model" refers to an algorithm or program that uses artificial intelligence to generate predicted facial image data from input image data.

[0234] "Predicted facial image data" is image data that predicts the future appearance of a missing person, taking into account changes such as aging and cosmetic surgery.

[0235] A "surveillance camera" is an electronic device installed in a public place to capture and record video in real time.

[0236] "Means of acquiring video data in real time" refers to the technology and equipment used to collect ongoing video footage from surveillance cameras and send it to a server.

[0237] "Means for extracting facial images" refers to an algorithm or technology for identifying a person's face from video data and extracting it as a facial image.

[0238] The "means for calculating the matching rate" refers to an algorithm or program that compares the extracted facial image with predicted facial image data and calculates the degree of matching as a numerical value.

[0239] "Means for generating and notifying alerts" refers to a system for creating a warning message and notifying relevant organizations and users when the match rate exceeds a certain threshold.

[0240] "Emotion data" is data that expresses the user's emotional state numerically or by category.

[0241] "Means for analyzing emotional data" refers to algorithms or technologies that analyze the emotional state of a user from input, facial expressions, and voice and obtain the data.

[0242] "Means for optimizing the operation of the entire system" refers to techniques and methods for adjusting the operation and response of each part of the system based on the acquired emotional data, allowing it to operate in the most effective manner.

[0243] The present invention is a system for improving the rate at which missing persons are found, and aims to efficiently find missing persons by generating predicted facial image data based on facial image data of the missing person and comparing this with video data from surveillance cameras installed in public institutions.Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system's responsiveness is improved.

[0244] First, family members or people involved (hereinafter referred to as "users") upload facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. The uploaded data is then sent to the server, which stores it in a database and begins pre-processing the image data. This pre-processing includes processes such as image resizing, noise removal, and color adjustment.

[0245] The server inputs the preprocessed facial image data into a generative AI model (e.g., StyleGAN) to generate multiple predicted facial image data that take into account changes over time, such as cosmetic surgery, etc. The generated predicted facial image data is then stored in a database by the server.

[0246] Meanwhile, surveillance cameras are installed in stations and public places and capture video data in real time. The surveillance camera terminals send the captured video data to a server. The server uses facial recognition algorithms such as OpenCV and Dlib to extract facial images from the video data and compare them with predicted facial image data stored in a database. The server calculates the match rate and generates an alert if this match rate exceeds a certain threshold.

[0247] The generated alert includes the matched facial image, the location of the surveillance camera, and the time the match was detected. The alert is sent to the device of the responding agency, such as the police, who will then go to the site to identify and protect the person and arrange for them to be reunited with their family.

[0248] The emotion engine also analyzes users' emotional data and optimizes the operation of the entire system based on that data. Specifically, when a user uploads data, the emotion engine analyzes the emotions in real time from the user's face and sentences. The server obtains the analyzed emotional data and adjusts the search priority and alert content.

[0249] Specific examples

[0250] For example, consider a case where a person with dementia has gone missing. Family members upload past photos of the person to the system, along with basic information such as age and gender. The server preprocesses these images and inputs them into a generative model to generate multiple facial images for predicting their current appearance. The emotion engine also analyzes the family's emotional state of concern and adjusts the system's overall responsiveness and alertness. Next, facial images are extracted from real-time footage captured by surveillance cameras at station ticket gates and compared with the stored predicted facial images. If the match rate is high, the server sends an alert to the police, who then dispatch to the scene to protect the person. The family is then immediately contacted, enabling the reunion. The emotion engine's analysis of the user's emotions also allows for efficient care and support after the reunion.

[0251] Prompt Sentence Examples

[0252] "Predict the current appearance of a person who has gone missing due to dementia based on past facial photographs. Age: 70, gender: male, last seen in: XX City, XX Prefecture."

[0253] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0254] Step 1:

[0255] The user uploads facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. This data is then sent to the server. At this time, the facial image data and basic information are prepared as input, and the raw facial image data and basic information stored on the server are obtained as output. Specifically, the user selects an image file from a dedicated web form, enters basic information in the text box, and presses the send button.

[0256] Step 2:

[0257] The server stores the received facial image data in a database and begins preprocessing. Preprocessing includes image resizing, noise removal, and color adjustment. The input here is the raw facial image data stored on the server, and the output is preprocessed facial image data converted into a format suitable for the generative AI model. Specifically, the server uses an image processing library (e.g., OpenCV) to resize the facial image (e.g., to 256x256 pixels), apply a noise removal filter, and equalize the color tone.

[0258] Step 3:

[0259] The server inputs the preprocessed facial image data into a generative AI model (e.g., StyleGAN) to generate multiple predicted facial image data that take into account changes such as aging and cosmetic surgery. The input here is preprocessed facial image data, and the output is predicted facial image data. Specifically, the server sends an API request to the generative model and stores the generated results in a database.

[0260] Step 4:

[0261] The server stores the generated predicted face image data in a database. The input of this step is the predicted face image data output from the generative AI model, and the output is the predicted face image data stored in the database. Specifically, the server uses a database management system (e.g., MySQL) to store the predicted face image data in an appropriate format.

[0262] Step 5:

[0263] Surveillance camera terminals continuously capture video data in real time at stations and public places. The captured video data is sent to a server. The input of this step is the real-time video data, and the output is the video data sent to the server. Specifically, the surveillance camera terminals have the function of sending video streams to the server over the network (e.g., using the RTSP protocol).

[0264] Step 6:

[0265] The server applies a facial recognition algorithm to extract facial images from the received video data. The input to this step is the video data sent to the server, and the output is the facial images extracted from the video data. Specifically, the server uses libraries such as OpenCV and Dlib to detect facial areas from the video frames and extract facial images.

[0266] Step 7:

[0267] The server matches the extracted facial image with predicted facial image data stored in a database. The inputs to this step are the extracted facial image and predicted facial image data, and the output is a match rate. Specifically, the server uses a facial recognition algorithm to compare the features of each facial image and calculate the match rate (e.g., cosine similarity).

[0268] Step 8:

[0269] The server generates an alert and notifies if the match rate exceeds a certain threshold. The input of this step is the calculated match rate, and the output is the generated alert. Specifically, the server generates an alert message when the match rate exceeds a set threshold (e.g., 90%) and sends an alert to the police or related organizations through the notification system.

[0270] Step 9:

[0271] The emotion engine analyzes the user's emotional data and optimizes the overall system behavior based on that data. The input to this step is the user's emotional data, and the output is optimized system behavior. Specifically, the emotion engine analyzes the facial expressions and sentences used when the user uploads data to the system, and sends the estimated emotional state as numerical data to the server. The server then adjusts responsiveness and alert content based on this data.

[0272] (Application example 2)

[0273] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0274] Conventional missing person search systems have had problems with low accuracy in handling facial image data and matching it with actual facial images, resulting in low rates of finding missing persons. In particular, as time passes, the appearance of a missing person changes, further reducing the accuracy of matching. Furthermore, because searches are performed without taking the user's emotional state into consideration, there are also problems with low responsiveness and efficiency of the entire system. The purpose of the present invention is to solve these problems and improve the rate of finding missing persons and the responsiveness of the system.

[0275] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0276] In this invention, the server includes means for acquiring image data of a missing person, means for operating a generative model for generating predicted facial image data from the image data, means for saving the generated predicted facial image data, means for acquiring video data from a surveillance camera in real time, means for extracting a facial image from the video data, means for comparing the extracted facial image with the predicted facial image data and calculating a match rate, means for generating and notifying an alert when the match rate exceeds a certain threshold, means for analyzing user emotional data and optimizing system operation, and means for inputting and managing information about the missing person using an application installed on a smart device. This enables the generation of predicted facial images that take into account changes in the missing person's appearance and optimal system operation based on the user's emotional state, improving the rate at which missing people are found and the responsiveness of the entire system.

[0277] "Image data of a missing person" is data that includes a facial image of a missing person and related personal information.

[0278] A "generative model" is a machine learning algorithm for generating predicted facial image data from input facial image data of a missing person.

[0279] "Predicted facial image data" is facial image data obtained by a generative model that predicts the current appearance of a missing person.

[0280] A "surveillance camera" is a device that is installed in public facilities or public places and captures video data in real time.

[0281] "Video data" refers to real-time video information acquired from a surveillance camera.

[0282] "Means for extracting facial images" refers to techniques or algorithms for extracting only the facial portion from video data.

[0283] The "means for matching and calculating the degree of match" is a technique for comparing the extracted face image with predicted face image data and quantifying the degree of similarity between the two.

[0284] "Means for generating and notifying alerts" refers to technology that generates an alert when the matching rate exceeds a certain threshold and notifies relevant institutions and parties of that information.

[0285] "Means for analyzing user emotional data and optimizing system operation" refers to technology that analyzes the user's emotional state and adjusts the system's responsiveness and alertness.

[0286] An "application installed on a smart device" is application software installed on a mobile device such as a smartphone or smart glasses for inputting and managing information about missing persons.

[0287] "Preprocessing means" refers to a technique for performing processes such as noise removal and resizing before inputting facial image data into a generative model.

[0288] This invention is a system for improving the rate at which missing people are found. The system is mainly composed of a server, a surveillance camera, an application, and an emotion analysis engine.

[0289] 1. System Configuration

[0290] server:

[0291] The server is the central data processing unit and performs the following main processes:

[0292] Data Acquisition: Acquires image data and basic information of the missing person provided by the user.

[0293] Generative model: Operate a generative AI model to generate predicted facial image data from the provided image data. Here, TensorFlow or PyTorch are used as the generative AI model.

[0294] Data storage: The generated predicted face image data is stored in a database.

[0295] Video data acquisition: Acquire video data from surveillance cameras in real time.

[0296] Facial image extraction: Facial images are extracted from video data using OpenCV.

[0297] Matching and match rate calculation: Using a face recognition library such as Dlib, the extracted face image is matched with the predicted face image data and the match rate is calculated.

[0298] Alert generation: Provides a function to generate an alert and notify when the match rate exceeds a certain threshold. Firebase Cloud Messaging is used for alert notification.

[0299] Emotion analysis: Analyzes user emotional data using emotion analysis engines such as Affectiva. Optimize system operation based on the results of this analysis.

[0300] Surveillance cameras:

[0301] It is installed in public places and is responsible for transmitting video data to a server in real time. Video data is managed via a network video recorder (NVR).

[0302] Smart device applications:

[0303] An application that allows users to input and manage information about missing persons through smart devices such as smartphones or smart glasses, and provides specific prompts for:

[0304] "Please upload a photo of the subject's face. Enter basic information (name, age, gender, last seen location)."

[0305] "We will optimize search parameters based on the user's emotional state. We are analyzing emotional data in real time."

[0306] 2. Working Example

[0307] Examples:

[0308] For example, if someone with dementia goes missing, a family member can use a smartphone application to upload a past photo of the person to the system, along with basic information such as name, age, gender, and last known location.

[0309] Preprocessing of facial image data:

[0310] The server performs preprocessing such as resizing the images, removing noise, and adjusting color tones, and converts them into a format that can be input to the generative model. This processing is done using OpenCV.

[0311] Generate predicted face images:

[0312] The preprocessed facial image data is input into the server's generative model (TensorFlow or PyTorch), and multiple predicted facial image data are generated, taking into account changes such as aging and cosmetic surgery.

[0313] Real-time video data collection and analysis:

[0314] Surveillance cameras capture video data in real time at stations and public places and send it to a server, which then extracts facial images from the video data and compares them with predicted facial image data stored in a database.

[0315] Alert Notifications:

[0316] If the match rate of the matched facial images exceeds a certain threshold, the server generates an alert and notifies police and other relevant parties in real time using Firebase Cloud Messaging, allowing relevant authorities to respond to the scene and protect the individual.

[0317] The above process enables the early detection and protection of missing persons, and also makes it possible to optimize the system's responsiveness by reflecting the results of user emotion analysis.

[0318] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0319] Step 1:

[0320] User Input

[0321] Using a smart device application, users input a facial image and basic information (name, age, gender, last seen location) of the missing person, which is then sent to a server via the application.

[0322] Input: Facial image data and basic information of the missing person

[0323] Output: Facial image data and basic information sent to the server

[0324] Step 2:

[0325] Image data preprocessing

[0326] The server processes the received facial image data. This preprocessing involves resizing the image, removing noise, adjusting color tone, etc., and converting it into a format that can be input to the generative model. OpenCV is used here.

[0327] Input: Acquired facial image data

[0328] Output: Preprocessed face image data

[0329] Specific operations: resize image (e.g., change to 128x128 pixels), remove noise (apply Gaussian filter), adjust color tone (convert to grayscale)

[0330] Step 3:

[0331] Generating predicted face images

[0332] The preprocessed facial image data is input into a generative model on the server to generate a predicted facial image that reflects the missing person's current appearance. This generation is done using TensorFlow and PyTorch. The generated predicted facial image data is stored in a database.

[0333] Input: Preprocessed face image data

[0334] Output: Predicted face image data

[0335] Specific operation: An image is input into the generative model, and multiple predicted facial images are generated, taking into account changes such as aging and cosmetic surgery.

[0336] Step 4:

[0337] Collecting video data from surveillance cameras

[0338] The server receives video data in real time from surveillance cameras installed in stations and public places, and manages the video data via a network video recorder (NVR).

[0339] Input: Surveillance camera video data

[0340] Output: Real-time video data sent to the server

[0341] Specific operation: Video data from a surveillance camera is sent to a server via a network.

[0342] Step 5:

[0343] Facial image extraction from video data

[0344] The server uses OpenCV to extract facial images from the video data received in real time, resulting in image data with only the facial area extracted.

[0345] Input: Real-time video data

[0346] Output: Extracted face image

[0347] Specific operation: Detects faces from video frames and extracts only the facial parts

[0348] Step 6:

[0349] Facial image matching and matching rate calculation

[0350] The server uses a facial recognition library such as Dlib to match the extracted facial image with stored predicted facial image data and calculate the match rate.

[0351] Input: Extracted face image, predicted face image data

[0352] Output: Match rate

[0353] Specific operation: Calculate the similarity between the two using a face recognition algorithm

[0354] Step 7:

[0355] Alerting and Notifications

[0356] If the match rate exceeds a certain threshold, the server generates an alert using Firebase Cloud Messaging to notify relevant authorities and relatives, including information such as the matched face image, the detection location, and the detection time.

[0357] Input: Match rate, face image, location information, time information

[0358] Output: Alert notification

[0359] What it does: Sends alerts to mobile devices and computers using Firebase Cloud Messaging

[0360] Step 8:

[0361] Emotional data analysis and system optimization

[0362] The server uses an emotion analysis engine such as Affectiva to analyze user emotion data and optimize the responsiveness and alertness of the entire system.

[0363] Input: User emotion data

[0364] Output: Optimized system response

[0365] Specific operation: Drives the emotion analysis engine and adjusts the system's operating parameters based on the analysis results.

[0366] In this way, the entire system goes through a series of processing steps to help find missing people early and realize optimal search activities that take into account the user's emotional state.

[0367] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0368] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0369] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0370] [Second embodiment]

[0371] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0372] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0373] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0374] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0375] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0376] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0377] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0378] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0379] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0380] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0381] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0382] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0383] The present invention is a system for improving the rate at which missing persons are found. Its purpose is to efficiently find missing persons by generating predicted facial image data using a generative model based on facial image data of the missing person and then comparing this with video data from surveillance cameras installed in public institutions.

[0384] First, family members or other people involved (hereinafter referred to as "users") upload facial image data of the missing person to the system. This facial image data is a past photo of the person who went missing due to dementia or other factors, and in most cases, multiple photos are provided.

[0385] The server stores the facial image data received from the user in a database and then performs preprocessing, such as resizing the image, removing noise, and adjusting color tones, to convert it into a format suitable for input to the generative model.

[0386] The preprocessed facial image data is input into a generative model on the server. This generative model generates multiple predicted facial image data, taking into account changes such as aging and cosmetic surgery. These predicted facial image data are important for estimating the current appearance of the missing person.

[0387] Next, the generated predicted facial image data is stored on a server and linked to a surveillance camera system. Multiple surveillance cameras continuously capture video data in real time at stations and public places. The surveillance camera terminals then transmit this video data to the server in real time.

[0388] The server applies a facial recognition algorithm to extract facial images from the received video data. The extracted facial images are compared with predicted facial image data stored in a database to calculate a match rate. If the match rate exceeds a certain threshold, the server generates an alert.

[0389] The alert will include the matched facial image, the CCTV camera location, and the time the match was detected. The alert will be sent to a police device. The police device will receive the alert and provide on-site information to response teams. Police will then arrive at the scene to identify and protect the individual. After identification, police will contact the family and arrange for the missing person to be reunited with their family.

[0390] Specific examples

[0391] For example, consider a case where a person with dementia has gone missing. Family members upload past photos of the person to the system, along with basic information such as age and gender. The server preprocesses these images and generates multiple facial images that are input into a generative model to predict their current appearance. Then, facial images are extracted from real-time footage captured by surveillance cameras at station ticket gates and compared with the stored predicted facial images. If there is a high match, the server sends an alert to the police, who respond to the scene and protect the person. The family is then immediately contacted to ensure the reunion.

[0392] The above is an embodiment of the present invention, and it is expected that this process will significantly improve the early detection of missing persons and reunification with their families.

[0393] The processing flow will be explained below.

[0394] Step 1:

[0395] Users upload facial image data and basic information (such as name, age, gender, and last seen location) of missing people to the system, which is used to accurately understand the latest status of missing people.

[0396] Step 2:

[0397] The server stores the facial image data received from the user in a database in an appropriate format to maintain the quality of the image data.

[0398] Step 3:

[0399] The server preprocesses the stored facial image data, specifically by resizing the image, removing noise, adjusting color, etc., and converting it into a format suitable for the generative model.

[0400] Step 4:

[0401] The server inputs the preprocessed facial image data into a generative model. This generative model generates predicted facial image data, taking into account factors such as aging and cosmetic surgery. The generated predicted facial image data has multiple variations, each reflecting a different possibility.

[0402] Step 5:

[0403] The server stores the generated predicted facial image data in a database, which makes it available for subsequent facial image matching processes.

[0404] Step 6:

[0405] The terminal (surveillance camera system) acquires video data in real time. These surveillance cameras are installed in many public institutions and facilities and continuously transmit the video data to a server.

[0406] Step 7:

[0407] The server receives real-time video data transmitted from the surveillance cameras, which is then analyzed using a facial recognition algorithm to extract facial images.

[0408] Step 8:

[0409] The server compares the facial images extracted from the real-time video data with the stored predicted facial image data. A matching algorithm calculates the match rate and evaluates the likelihood of identifying the person.

[0410] Step 9:

[0411] The server generates an alert when the match rate exceeds a set threshold, and the alert includes the matching face image, the surveillance camera's location information, and time information.

[0412] Step 10:

[0413] The server sends the generated alert to the police terminal, which can then respond immediately to the incident.

[0414] Step 11:

[0415] After receiving the alert, police will head to the scene, identify the person, and take them into custody. After that, the police will confirm that the person is a missing person and contact their family.

[0416] Step 12:

[0417] Police will arrange for the missing person to be reunited with their family and return them safely home, a process that will ensure the early detection and reunion of missing people.

[0418] Example 1

[0419] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0420] Locating missing people often becomes more difficult as time passes. This becomes even more difficult when changes in appearance or age occur due to factors such as dementia. Technology is also needed to efficiently analyze video data acquired in real time from multiple surveillance devices. The present invention aims to solve these problems and improve the rate at which missing people are found.

[0421] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0422] In this invention, the server includes a means for preprocessing image data of the missing person and converting it into an appropriate format, a means for operating a generative model for generating predicted facial image data from the preprocessed image data, and a means for saving the generated predicted facial image data. This enables the generation of predicted facial image data that takes into account changes over time, such as changes in the patient's appearance or surgical procedures. The server also includes a means for acquiring video data from multiple monitoring devices in real time, extracting facial images from the video data, comparing them with the predicted facial image data, and calculating the match rate. This allows for the prompt generation of an alert and notification when the match rate exceeds a certain threshold.

[0423] "Missing person" refers to a person whose whereabouts are unknown.

[0424] "Image data" refers to data that digitally represents visual information, including the face of a missing person.

[0425] "Preprocessing" refers to processes such as resizing, noise removal, and color adjustment that are performed to convert image data into a format suitable for the generative model.

[0426] A "generative model" refers to artificial intelligence that generates predicted facial image data based on input past image data, taking into account changes over time, such as changes in appearance or surgical procedures.

[0427] "Predicted facial image data" refers to image data that predicts the current appearance of a missing person, generated by a generative model.

[0428] "Monitoring equipment" refers to devices installed at stations and public places that capture video data in real time.

[0429] "Video data" refers to real-time image and video data captured by a monitoring device.

[0430] "Facial image" refers to image data that includes the facial portion of a person extracted from video data.

[0431] "Matching" refers to the process of comparing the extracted facial image with predicted facial image data and calculating the match rate.

[0432] The "match rate" refers to the ratio indicating the degree to which the matched face image and predicted face image data match.

[0433] An "alert" refers to warning information that is generated when the match rate exceeds a certain threshold.

[0434] "Notification" refers to the process of communicating generated alerts to police and other relevant parties.

[0435] The present invention is a system for improving the rate at which missing persons are found. Specifically, the system generates predicted facial image data using a generative model based on facial image data of the missing person, and then compares this with video data from surveillance equipment installed in public institutions, thereby efficiently finding the missing person. The overall flow of the system will be explained in detail below, with the roles of the user, server, and terminal.

[0436] First, the user uploads facial image data of the missing person to the system. This facial image data is a past photograph of the missing person, and usually multiple photographs are provided. The user accesses a dedicated page of the system using a web browser, uploads any image of the missing person they have, and enters basic information such as the missing person's name, age, and gender.

[0437] Next, the server stores the facial image data received from the user in a database and performs preprocessing, including resizing the image, removing noise, and adjusting color tone, before converting it into a format suitable for input to the generative model. The hardware used is a server with high-performance computing power and large storage capacity (e.g., the Dell PowerEdge series), and the software used is a facial recognition library such as OpenCV or Dlib.

[0438] The facial image data after preprocessing is input into a generative model on the server. This generative model generates multiple predicted facial image data to estimate the missing person's current appearance, taking into account changes over time, such as changes due to surgery, etc. A generative AI model such as StyleGAN is used as this generative model. The generated predicted facial image data is stored on the server.

[0439] The server then connects to multiple monitoring devices. These devices continuously capture video data in real time at stations and public places and send it to the server. The terminals (monitoring devices) use a surveillance system that includes high-resolution cameras (e.g., cameras manufactured by Axis Communications).

[0440] The server applies a facial recognition algorithm to extract facial images from the received video data. The extracted facial images are compared with predicted facial image data stored in a database to calculate the match rate. Facial recognition algorithms used include OpenCV and Dlib. If the match rate exceeds a certain threshold, the server generates an alert. The alert includes the matched facial image, the location of the surveillance device, and the time the match was detected.

[0441] The alert is sent to the police terminal, which is a sophisticated computer system that receives the alert in real time. Once the police receive the alert, they will dispatch a response team to the scene to identify and protect the individual. After identification, the police will contact the family and arrange for the missing person to be reunited with their family.

[0442] Specific examples

[0443] For example, consider a case where a person with dementia goes missing. Family members upload past photos of the person to the system and enter basic information such as age and gender. The server preprocesses these images and generates multiple facial images that are input into a generative model to predict their current appearance. Then, a surveillance device extracts facial images from real-time footage taken at station ticket gates and compares them with the stored predicted facial images. If there is a high match, the server sends an alert to the police, who respond to the scene and protect the person. The family is then immediately contacted to ensure the reunion.

[0444] Example prompt sentence:

[0445] 1. "Generate facial image data to predict current appearance based on a facial photo taken 40 years ago."

[0446] 2. "Generate a current facial image of a missing person based on past photographs, taking into account age-related changes."

[0447] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0448] Step 1:

[0449] The user uploads facial image data of the missing person.

[0450] Input: Past facial images of the missing person, basic information such as name, age, and gender

[0451] Output: Sending face image data and basic information to the server

[0452] Specific operations: The user opens a web browser and accesses the system's dedicated page. They click the "Upload Images" button, select and upload multiple past photos of the missing person they have, and then enter basic information about the missing person into the form.

[0453] Step 2:

[0454] The server stores the facial image data received from the user in a database and performs preprocessing.

[0455] Input: Facial image data and basic information sent to the server

[0456] Output: Preprocessed image data stored in a database

[0457] Specific operation: The server records the received facial images in a database. Then, it performs preprocessing such as resizing, noise removal, and color adjustment to convert the images into a format suitable for input to the generative model.

[0458] Step 3:

[0459] The server inputs the preprocessed facial image data into a generative model to generate predicted facial image data.

[0460] Input: Preprocessed face image data

[0461] Output: Generated predicted face image data

[0462] Specific operation: The server inputs the preprocessed facial image into the generative model. The generative model generates multiple predicted facial image data, taking into account changes over time, such as aging and surgical procedures. The prompts used are, "Based on a facial photo from 40 years ago, please generate facial image data to predict current appearance," and "Based on past photographs of a missing person, please generate a current facial image that takes into account changes due to age."

[0463] Step 4:

[0464] The server stores the generated predicted face image data in a database.

[0465] Input: Generated predicted face image data

[0466] Output: Predicted face image data stored in a database

[0467] Specific operation: The server stores the generated predicted face image data in a database.

[0468] Step 5:

[0469] The terminal (monitoring device) transmits video data to the server in real time.

[0470] Input: Real-time video data captured by a surveillance camera

[0471] Output: Real-time video data sent to the server

[0472] Specific operation: The terminal (monitoring device) streams video footage taken at station ticket gates and public places to a server in real time.

[0473] Step 6:

[0474] The server extracts a facial image from the video data and compares it with predicted facial image data.

[0475] Input: Real-time video data

[0476] Output: Matching result (match rate)

[0477] Specific operation: The server applies a facial recognition algorithm to the received video data to extract facial images. The extracted facial images are compared with the stored predicted facial image data and the match rate is calculated. OpenCV and Dlib are used here.

[0478] Step 7:

[0479] The server generates an alert and notifies you if the match rate exceeds a certain threshold.

[0480] Input: Matching result (match rate)

[0481] Output: Alert information

[0482] Specific operation: If the calculated match rate exceeds a threshold, the server generates an alert. The alert includes the matched face image, the location information of the monitoring device, and the time when the match was detected. The generated alert is sent to the police terminal.

[0483] Step 8:

[0484] Police receive an alert, provide on-site information to response units, and initiate action.

[0485] Input: Alert information sent from the server

[0486] Output: On-site confirmation and protective action

[0487] Specific operations: The police terminal receives the alert and provides the relevant scene information to the response team. The police then go to the scene to identify and protect the person in question, and after confirmation, contact the family. Finally, they arrange for the missing person to be reunited with their family.

[0488] This concludes the detailed explanation of each processing step in the system program, which will enable efficient detection of missing persons and prompt response.

[0489] (Application example 1)

[0490] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0491] There is a need for an efficient and rapid method to improve the rate at which missing persons are found. However, current systems only use public surveillance cameras, and efficient searches using mobile information devices owned by individuals or at home are not performed. Furthermore, there is a lack of reliable predictive methods for responding to changes in facial images. As a result, searches for missing persons are delayed and the rate at which they are found decreases. Therefore, there is a need for a rapid and widespread method for finding missing persons.

[0492] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0493] In this invention, the server includes means for acquiring image data of a missing person, means for operating a generative model for generating predicted facial image data from the image data, means for saving the generated predicted facial image data, means for acquiring video data in real time from a surveillance camera or a camera of a mobile information terminal, means for extracting a facial image from the video data, means for comparing the extracted facial image with the predicted facial image data and calculating a match rate, means for generating and notifying an alert when the match rate exceeds a certain threshold, and means for sharing location information based on the alert, thereby enabling a wide-area search for missing people in real time.

[0494] A "missing person" is someone whose current whereabouts are unknown and who requires a search.

[0495] "Image data" refers to digital data that includes past photographs of faces and other visual information about missing persons.

[0496] A "generative model" is an artificial intelligence model that predicts and generates current facial image data from input past facial image data.

[0497] A "personal digital assistant" is a portable device such as a smartphone or tablet.

[0498] A "surveillance camera" is a camera device installed in a public place that continuously records and monitors footage.

[0499] "Acquiring video data in real time" means that the video obtained from the camera is immediately sent to the system for processing.

[0500] "Extracting a face image" means cutting out and acquiring only the face portion of a person from video data.

[0501] "Matching" means comparing and verifying a match against different data sets.

[0502] The "match rate" is a ratio indicating the degree to which the extracted face image and the predicted face image data are identical.

[0503] An "alert" is a warning or notification that is generated when the match rate exceeds a certain threshold.

[0504] "Sharing location information" means sending and receiving geographical data of the location where a missing person was found between multiple devices.

[0505] The present invention is a system for efficiently locating missing persons. This system consists of three main components: a server, a terminal, and a user.

[0506] First, the user uploads past facial image data of the missing person to the system. This can be done using a device such as a smartphone or PC. The image data provided by the user is sent to the server. The server receives this data and performs preprocessing. This preprocessing involves resizing the image, removing noise, and adjusting the color tone. This process converts the data into a format suitable for the generative model.

[0507] Next, a generative model generates predicted facial image data from the preprocessed facial image data. This generative model outputs multiple predicted facial images, taking into account facial changes such as aging and cosmetic surgery. This predicted facial image data is stored in a database on the server.

[0508] The server receives video data in real time from surveillance cameras and mobile devices. For example, when a smartphone camera captures video in a public place, the video data is sent to the server. The server then applies a facial recognition algorithm to extract facial images from the video data. This algorithm can use models such as InceptionResnetV1 or MTCNN.

[0509] The extracted facial image is compared with predicted facial image data stored in the server's database. At this time, the match rate is calculated. If the match rate exceeds a certain threshold, the server generates an alert. This alert includes the matched facial image, detected location information, and time information. The alert is sent to the devices of the police and relevant parties.

[0510] Additionally, the alert notification includes a location sharing function, enabling a rapid response if a missing person is found. For example, the smartphone app can send geographical data of the detected location to a server, allowing police or family members to receive the data and take action.

[0511] Examples:

[0512] Below are some example prompts for the generative AI model:

[0513] "Generate a current image of a 50-year-old woman's face from an image taken 20 years ago."

[0514] "Please output an image of a 20-year-old man's face converted to look like a 30-year-old man."

[0515] This enables wide-area and real-time searches for missing people. The server calculates the facial image match rate and can quickly generate an alert and notify relevant parties if a match exceeds a certain threshold. This enables the early detection and protection of missing people and promotes reunification with their families.

[0516] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0517] Step 1:

[0518] The user uploads the missing person's past facial image data to the system. The user uses a device such as a smartphone or PC to send the image data to the server. The input at this time is the missing person's past facial image data, and the output is the data transferred to the server.

[0519] Step 2:

[0520] The server preprocesses the facial image data received from the user. It resizes the image, removes noise, adjusts color tone, and converts the data format into a format that can be input to the generative model. The input of this process is the uploaded facial image data, and the output is the preprocessed facial image data.

[0521] Step 3:

[0522] The server inputs the preprocessed facial image data into a generative model, which generates multiple predicted facial image data by taking into account factors such as aging and cosmetic surgery. The input is the preprocessed facial image data, and the output is predicted facial image data.

[0523] Step 4:

[0524] The generated predicted facial image data is stored in a database on the server. This database is used in subsequent facial recognition processes. The input is the predicted facial image data, and the output is the data stored in the database.

[0525] Step 5:

[0526] A camera on a device (e.g., a smartphone) or a public surveillance camera captures video data in real time. The captured video data is sent to a server. The input is the real-time video data, and the output is the data sent to the server.

[0527] Step 6:

[0528] The server applies a facial recognition algorithm to extract facial images from the transmitted video data, such as InceptionResnetV1 or MTCNN. The input of this process is real-time video data, and the output is extracted facial image data.

[0529] Step 7:

[0530] The extracted facial image data is compared with predicted facial image data stored in the server's database, and a match rate is calculated. This calculation confirms whether the image matches the missing person. The input is the extracted facial image data and predicted facial image data, and the output is the calculated match rate.

[0531] Step 8:

[0532] If the match rate exceeds a certain threshold, the server generates an alert, which includes the matched face image, location information, and time information. The input is the situation where the match rate exceeds a certain threshold, and the output is the generated alert.

[0533] Step 9:

[0534] The generated alert is sent to the devices of the police and other relevant parties, and a response is initiated at the scene. The input is the generated alert, and the output is the notified information. The police and other relevant parties then proceed to the scene based on the location information to identify and protect the missing person.

[0535] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0536] The present invention is a system for improving the rate at which missing persons are found, and aims to efficiently find missing persons by generating predicted facial image data using a generative model based on facial image data of the missing person and comparing this with video data from surveillance cameras installed in public institutions.Furthermore, the system is characterized by improving the responsiveness of the system by combining it with an emotion engine that recognizes the user's emotions.

[0537] First, family members and other people involved (hereafter referred to as "users") upload facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. The emotion engine then analyzes the user's emotions in real time and optimizes the operation of the entire system based on that information.

[0538] The server stores the facial image data received from the user in a database and then preprocesses it. This involves resizing the image, removing noise, adjusting color, and converting it into a format suitable for input to the generative model. The emotion engine also obtains the user's emotional data during this process, which can be used as additional information to help search for missing people.

[0539] The facial image data after preprocessing is input into a generative model on the server. This generative model generates multiple predicted facial image data, taking into account changes such as aging and cosmetic surgery. The generated predicted facial image data is important for estimating what the missing person currently looks like.

[0540] Next, the generated predicted facial image data is stored on a server and linked to a surveillance camera system. Multiple surveillance cameras continuously capture video data in real time at stations and public places. The surveillance camera terminals then transmit this video data to the server in real time.

[0541] The server applies a facial recognition algorithm to extract facial images from the received video data. An emotion engine then analyzes the user's emotional data and sets optimal parameters for searching for the missing person. The extracted facial images are compared with predicted facial image data stored in a database to calculate the match rate. If the match rate exceeds a certain threshold, the server generates an alert.

[0542] The alert includes the matched facial image, the surveillance camera's location, and the time the match was detected. The alert is sent to the police device. The emotion engine takes the user's emotional state into account to customize the alert content and notification method. The police device receives the alert and provides on-site information to response teams. The police then arrive at the scene to identify and protect the person. After identification, the police contact the family and arrange for the missing person to be reunited with their family.

[0543] Specific examples

[0544] For example, consider a case where a person with dementia has gone missing. Family members upload past photos of the person to the system, along with basic information such as age and gender. The server preprocesses these images and inputs them into a generative model to generate multiple facial images for predicting their current appearance. The emotion engine also analyzes the family's emotional state of concern and adjusts the system's overall responsiveness and alertness. Next, facial images are extracted from real-time footage captured by surveillance cameras at station ticket gates and compared with the stored predicted facial images. If the match rate is high, the server sends an alert to the police, who then dispatch to the scene to protect the person. The family is then immediately contacted, enabling the reunion. The emotion engine's analysis of the user's emotions also allows for efficient care and support after the reunion.

[0545] The above is an embodiment of the present invention, and by combining this process with an emotion engine, it is expected that the early discovery and reunion of missing persons will proceed even more smoothly.

[0546] The processing flow will be explained below.

[0547] Step 1:

[0548] The user uploads facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. At the same time, the user inputs emotional data (such as worry, anxiety, and impatience). The emotional data can be in text format or voice input.

[0549] Step 2:

[0550] The server stores the facial image data and emotion data received from the user in a database, with the facial image data stored in high quality for subsequent processing.

[0551] Step 3:

[0552] The server preprocesses the facial image data, specifically resizing the image, removing noise, adjusting color, etc. The preprocessed facial image data is then converted into a suitable format for input to the generative model.

[0553] Step 4:

[0554] The server inputs the preprocessed facial image data into a generative model, which then generates multiple predicted facial image data, taking into account factors such as aging and cosmetic surgery.

[0555] Step 5:

[0556] The server stores the generated predicted facial image data in a database. At this stage, the system has multiple facial image variations that can be used to more easily identify missing people.

[0557] Step 6:

[0558] The server uses an emotion engine to analyze the user's emotion data. Based on the analysis results, the system adjusts its responsiveness and alertness. For example, if the user shows a high level of impatience, the system will use a high-precision model to generate predicted facial images more quickly.

[0559] Step 7:

[0560] The terminals (surveillance camera systems) acquire video data in real time. These surveillance cameras are installed in stations and public places, and continuously transmit the video data to a server.

[0561] Step 8:

[0562] The server receives real-time video data transmitted from the surveillance cameras, which is analyzed using a facial recognition algorithm to extract facial images.

[0563] Step 9:

[0564] The server compares the extracted facial image with predicted facial image data stored in a database, calculates the match rate, and generates an alert if the match rate exceeds a certain threshold.

[0565] Step 10:

[0566] The server then sends the generated alert to the police terminal, which includes the matching face image, the location information of the surveillance camera, and the time when the match was detected.

[0567] Step 11:

[0568] Police devices receive the alert and provide on-site information to responding teams, who then arrive on-site to identify and protect the individual.

[0569] Step 12:

[0570] Once the police confirm that the person is missing, they will contact the family. Based on the user's emotional state, the emotion engine will customize the notification content and contact the user in the most appropriate way to reassure them.

[0571] Step 13:

[0572] Police will arrange for missing people to be reunited with their families and return home safely, and the emotion engine will analyze the user's emotions and coordinate support and care after the reunion.

[0573] These steps will ensure that missing persons are quickly identified and reunited with their families.

[0574] Example 2

[0575] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0576] The present invention aims to improve the rate at which missing people are found, and to solve the problems of cases where finding a missing person is difficult using conventional search methods, or where searches take a long time. Furthermore, since a flexible response that takes into account the user's emotions is required, a system that can optimize search activities and provide care after a person is found is also needed.

[0577] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring image data of a missing person, means for operating a generative model for generating predicted face image data from the image data, means for saving the generated predicted face image data, means for acquiring video data from a surveillance camera in real time, means for extracting a face image from the video data, means for matching the extracted face image with the predicted face image data and calculating a match rate, means for generating and notifying an alert when the match rate exceeds a certain threshold, means for analyzing emotional data acquired from a user, and means for optimizing the operation of the entire system based on the emotional data. This enables the missing person to be found quickly and efficiently, and also allows for flexible responses that take user emotions into consideration.

[0578] "Image data of a missing person" is digital data that shows the facial image and physical characteristics of a missing person.

[0579] A "generative model" refers to an algorithm or program that uses artificial intelligence to generate predicted facial image data from input image data.

[0580] "Predicted facial image data" is image data that predicts the future appearance of a missing person, taking into account changes such as aging and cosmetic surgery.

[0581] A "surveillance camera" is an electronic device installed in a public place to capture and record video in real time.

[0582] "Means of acquiring video data in real time" refers to the technology and equipment used to collect ongoing video footage from surveillance cameras and send it to a server.

[0583] "Means for extracting facial images" refers to an algorithm or technology for identifying a person's face from video data and extracting it as a facial image.

[0584] The "means for calculating the matching rate" refers to an algorithm or program that compares the extracted facial image with predicted facial image data and calculates the degree of matching as a numerical value.

[0585] "Means for generating and notifying alerts" refers to a system for creating a warning message and notifying relevant organizations and users when the match rate exceeds a certain threshold.

[0586] "Emotion data" is data that expresses the user's emotional state numerically or by category.

[0587] "Means for analyzing emotional data" refers to algorithms or technologies that analyze the emotional state of a user from input, facial expressions, and voice and obtain the data.

[0588] "Means for optimizing the operation of the entire system" refers to techniques and methods for adjusting the operation and response of each part of the system based on the acquired emotional data, allowing it to operate in the most effective manner.

[0589] The present invention is a system for improving the rate at which missing persons are found, and aims to efficiently find missing persons by generating predicted facial image data based on facial image data of the missing person and comparing this with video data from surveillance cameras installed in public institutions.Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system's responsiveness is improved.

[0590] First, family members or people involved (hereinafter referred to as "users") upload facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. The uploaded data is then sent to the server, which stores it in a database and begins pre-processing the image data. This pre-processing includes processes such as image resizing, noise removal, and color adjustment.

[0591] The server inputs the preprocessed facial image data into a generative AI model (e.g., StyleGAN) to generate multiple predicted facial image data that take into account changes over time, such as cosmetic surgery, etc. The generated predicted facial image data is then stored in a database by the server.

[0592] Meanwhile, surveillance cameras are installed in stations and public places and capture video data in real time. The surveillance camera terminals send the captured video data to a server. The server uses facial recognition algorithms such as OpenCV and Dlib to extract facial images from the video data and compare them with predicted facial image data stored in a database. The server calculates the match rate and generates an alert if this match rate exceeds a certain threshold.

[0593] The generated alert includes the matched facial image, the location of the surveillance camera, and the time the match was detected. The alert is sent to the device of the responding agency, such as the police, who will then go to the site to identify and protect the person and arrange for them to be reunited with their family.

[0594] The emotion engine also analyzes users' emotional data and optimizes the operation of the entire system based on that data. Specifically, when a user uploads data, the emotion engine analyzes the emotions in real time from the user's face and sentences. The server obtains the analyzed emotional data and adjusts the search priority and alert content.

[0595] Specific examples

[0596] For example, consider a case where a person with dementia has gone missing. Family members upload past photos of the person to the system, along with basic information such as age and gender. The server preprocesses these images and inputs them into a generative model to generate multiple facial images for predicting their current appearance. The emotion engine also analyzes the family's emotional state of concern and adjusts the system's overall responsiveness and alertness. Next, facial images are extracted from real-time footage captured by surveillance cameras at station ticket gates and compared with the stored predicted facial images. If the match rate is high, the server sends an alert to the police, who then dispatch to the scene to protect the person. The family is then immediately contacted, enabling the reunion. The emotion engine's analysis of the user's emotions also allows for efficient care and support after the reunion.

[0597] Prompt Sentence Examples

[0598] "Predict the current appearance of a person who has gone missing due to dementia based on past facial photographs. Age: 70, gender: male, last seen in: XX City, XX Prefecture."

[0599] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0600] Step 1:

[0601] The user uploads facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. This data is then sent to the server. At this time, the facial image data and basic information are prepared as input, and the raw facial image data and basic information stored on the server are obtained as output. Specifically, the user selects an image file from a dedicated web form, enters basic information in the text box, and presses the send button.

[0602] Step 2:

[0603] The server stores the received facial image data in a database and begins preprocessing. Preprocessing includes image resizing, noise removal, and color adjustment. The input here is the raw facial image data stored on the server, and the output is preprocessed facial image data converted into a format suitable for the generative AI model. Specifically, the server uses an image processing library (e.g., OpenCV) to resize the facial image (e.g., to 256x256 pixels), apply a noise removal filter, and equalize the color tone.

[0604] Step 3:

[0605] The server inputs the preprocessed facial image data into a generative AI model (e.g., StyleGAN) to generate multiple predicted facial image data that take into account changes such as aging and cosmetic surgery. The input here is preprocessed facial image data, and the output is predicted facial image data. Specifically, the server sends an API request to the generative model and stores the generated results in a database.

[0606] Step 4:

[0607] The server stores the generated predicted face image data in a database. The input of this step is the predicted face image data output from the generative AI model, and the output is the predicted face image data stored in the database. Specifically, the server uses a database management system (e.g., MySQL) to store the predicted face image data in an appropriate format.

[0608] Step 5:

[0609] Surveillance camera terminals continuously capture video data in real time at stations and public places. The captured video data is sent to a server. The input of this step is the real-time video data, and the output is the video data sent to the server. Specifically, the surveillance camera terminals have the function of sending video streams to the server over the network (e.g., using the RTSP protocol).

[0610] Step 6:

[0611] The server applies a facial recognition algorithm to extract facial images from the received video data. The input to this step is the video data sent to the server, and the output is the facial images extracted from the video data. Specifically, the server uses libraries such as OpenCV and Dlib to detect facial areas from the video frames and extract facial images.

[0612] Step 7:

[0613] The server matches the extracted facial image with predicted facial image data stored in a database. The inputs to this step are the extracted facial image and predicted facial image data, and the output is a match rate. Specifically, the server uses a facial recognition algorithm to compare the features of each facial image and calculate the match rate (e.g., cosine similarity).

[0614] Step 8:

[0615] The server generates an alert and notifies if the match rate exceeds a certain threshold. The input of this step is the calculated match rate, and the output is the generated alert. Specifically, the server generates an alert message when the match rate exceeds a set threshold (e.g., 90%) and sends an alert to the police or related organizations through the notification system.

[0616] Step 9:

[0617] The emotion engine analyzes the user's emotional data and optimizes the overall system behavior based on that data. The input to this step is the user's emotional data, and the output is optimized system behavior. Specifically, the emotion engine analyzes the facial expressions and sentences used when the user uploads data to the system, and sends the estimated emotional state as numerical data to the server. The server then adjusts responsiveness and alert content based on this data.

[0618] (Application example 2)

[0619] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0620] Conventional missing person search systems have had problems with low accuracy in handling facial image data and matching it with actual facial images, resulting in low rates of finding missing persons. In particular, as time passes, the appearance of a missing person changes, further reducing the accuracy of matching. Furthermore, because searches are performed without taking the user's emotional state into consideration, there are also problems with low responsiveness and efficiency of the entire system. The purpose of the present invention is to solve these problems and improve the rate of finding missing persons and the responsiveness of the system.

[0621] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0622] In this invention, the server includes means for acquiring image data of a missing person, means for operating a generative model for generating predicted facial image data from the image data, means for saving the generated predicted facial image data, means for acquiring video data from a surveillance camera in real time, means for extracting a facial image from the video data, means for comparing the extracted facial image with the predicted facial image data and calculating a match rate, means for generating and notifying an alert when the match rate exceeds a certain threshold, means for analyzing user emotional data and optimizing system operation, and means for inputting and managing information about the missing person using an application installed on a smart device. This enables the generation of predicted facial images that take into account changes in the missing person's appearance and optimal system operation based on the user's emotional state, improving the rate at which missing people are found and the responsiveness of the entire system.

[0623] "Image data of a missing person" is data that includes a facial image of a missing person and related personal information.

[0624] A "generative model" is a machine learning algorithm for generating predicted facial image data from input facial image data of a missing person.

[0625] "Predicted facial image data" is facial image data obtained by a generative model that predicts the current appearance of a missing person.

[0626] A "surveillance camera" is a device that is installed in public facilities or public places and captures video data in real time.

[0627] "Video data" refers to real-time video information acquired from a surveillance camera.

[0628] "Means for extracting facial images" refers to techniques or algorithms for extracting only the facial portion from video data.

[0629] The "means for matching and calculating the degree of match" is a technique for comparing the extracted face image with predicted face image data and quantifying the degree of similarity between the two.

[0630] "Means for generating and notifying alerts" refers to technology that generates an alert when the matching rate exceeds a certain threshold and notifies relevant institutions and parties of that information.

[0631] "Means for analyzing user emotional data and optimizing system operation" refers to technology that analyzes the user's emotional state and adjusts the system's responsiveness and alertness.

[0632] An "application installed on a smart device" is application software installed on a mobile device such as a smartphone or smart glasses for inputting and managing information about missing persons.

[0633] "Preprocessing means" refers to a technique for performing processes such as noise removal and resizing before inputting facial image data into a generative model.

[0634] This invention is a system for improving the rate at which missing people are found. The system is mainly composed of a server, a surveillance camera, an application, and an emotion analysis engine.

[0635] 1. System Configuration

[0636] server:

[0637] The server is the central data processing unit and performs the following main processes:

[0638] Data Acquisition: Acquires image data and basic information of the missing person provided by the user.

[0639] Generative model: Operate a generative AI model to generate predicted facial image data from the provided image data. Here, TensorFlow or PyTorch are used as the generative AI model.

[0640] Data storage: The generated predicted face image data is stored in a database.

[0641] Video data acquisition: Acquire video data from surveillance cameras in real time.

[0642] Facial image extraction: Facial images are extracted from video data using OpenCV.

[0643] Matching and match rate calculation: Using a face recognition library such as Dlib, the extracted face image is matched with the predicted face image data and the match rate is calculated.

[0644] Alert generation: Provides a function to generate an alert and notify when the match rate exceeds a certain threshold. Firebase Cloud Messaging is used for alert notification.

[0645] Emotion analysis: Analyzes user emotional data using emotion analysis engines such as Affectiva. Optimize system operation based on the results of this analysis.

[0646] Surveillance cameras:

[0647] It is installed in public places and is responsible for transmitting video data to a server in real time. Video data is managed via a network video recorder (NVR).

[0648] Smart device applications:

[0649] An application that allows users to input and manage information about missing persons through smart devices such as smartphones or smart glasses, and provides specific prompts for:

[0650] "Please upload a photo of the subject's face. Enter basic information (name, age, gender, last seen location)."

[0651] "We will optimize search parameters based on the user's emotional state. We are analyzing emotional data in real time."

[0652] 2. Working Example

[0653] Examples:

[0654] For example, if someone with dementia goes missing, a family member can use a smartphone application to upload a past photo of the person to the system, along with basic information such as name, age, gender, and last known location.

[0655] Preprocessing of facial image data:

[0656] The server performs preprocessing such as resizing the images, removing noise, and adjusting color tones, and converts them into a format that can be input to the generative model. This processing is done using OpenCV.

[0657] Generate predicted face images:

[0658] The preprocessed facial image data is input into the server's generative model (TensorFlow or PyTorch), and multiple predicted facial image data are generated, taking into account changes such as aging and cosmetic surgery.

[0659] Real-time video data collection and analysis:

[0660] Surveillance cameras capture video data in real time at stations and public places and send it to a server, which then extracts facial images from the video data and compares them with predicted facial image data stored in a database.

[0661] Alert Notifications:

[0662] If the match rate of the matched facial images exceeds a certain threshold, the server generates an alert and notifies police and other relevant parties in real time using Firebase Cloud Messaging, allowing relevant authorities to respond to the scene and protect the individual.

[0663] The above process enables the early detection and protection of missing persons, and also makes it possible to optimize the system's responsiveness by reflecting the results of user emotion analysis.

[0664] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0665] Step 1:

[0666] User Input

[0667] Using a smart device application, users input a facial image and basic information (name, age, gender, last seen location) of the missing person, which is then sent to a server via the application.

[0668] Input: Facial image data and basic information of the missing person

[0669] Output: Facial image data and basic information sent to the server

[0670] Step 2:

[0671] Image data preprocessing

[0672] The server processes the received facial image data. This preprocessing involves resizing the image, removing noise, adjusting color tone, etc., and converting it into a format that can be input to the generative model. OpenCV is used here.

[0673] Input: Acquired facial image data

[0674] Output: Preprocessed face image data

[0675] Specific operations: resize image (e.g., change to 128x128 pixels), remove noise (apply Gaussian filter), adjust color tone (convert to grayscale)

[0676] Step 3:

[0677] Generating predicted face images

[0678] The preprocessed facial image data is input into a generative model on the server to generate a predicted facial image that reflects the missing person's current appearance. This generation is done using TensorFlow and PyTorch. The generated predicted facial image data is stored in a database.

[0679] Input: Preprocessed face image data

[0680] Output: Predicted face image data

[0681] Specific operation: An image is input into the generative model, and multiple predicted facial images are generated, taking into account changes such as aging and cosmetic surgery.

[0682] Step 4:

[0683] Collecting video data from surveillance cameras

[0684] The server receives video data in real time from surveillance cameras installed in stations and public places, and manages the video data via a network video recorder (NVR).

[0685] Input: Surveillance camera video data

[0686] Output: Real-time video data sent to the server

[0687] Specific operation: Video data from a surveillance camera is sent to a server via a network.

[0688] Step 5:

[0689] Facial image extraction from video data

[0690] The server uses OpenCV to extract facial images from the video data received in real time, resulting in image data with only the facial area extracted.

[0691] Input: Real-time video data

[0692] Output: Extracted face image

[0693] Specific operation: Detects faces from video frames and extracts only the facial parts

[0694] Step 6:

[0695] Facial image matching and matching rate calculation

[0696] The server uses a facial recognition library such as Dlib to match the extracted facial image with stored predicted facial image data and calculate the match rate.

[0697] Input: Extracted face image, predicted face image data

[0698] Output: Match rate

[0699] Specific operation: Calculate the similarity between the two using a face recognition algorithm

[0700] Step 7:

[0701] Alerting and Notifications

[0702] If the match rate exceeds a certain threshold, the server generates an alert using Firebase Cloud Messaging to notify relevant authorities and relatives, including information such as the matched face image, the detection location, and the detection time.

[0703] Input: Match rate, face image, location information, time information

[0704] Output: Alert notification

[0705] What it does: Sends alerts to mobile devices and computers using Firebase Cloud Messaging

[0706] Step 8:

[0707] Emotional data analysis and system optimization

[0708] The server uses an emotion analysis engine such as Affectiva to analyze user emotion data and optimize the responsiveness and alertness of the entire system.

[0709] Input: User emotion data

[0710] Output: Optimized system response

[0711] Specific operation: Drives the emotion analysis engine and adjusts the system's operating parameters based on the analysis results.

[0712] In this way, the entire system goes through a series of processing steps to help find missing people early and realize optimal search activities that take into account the user's emotional state.

[0713] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0714] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0715] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0716] [Third embodiment]

[0717] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0718] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0719] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0720] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0721] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0722] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0723] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0724] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0725] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0726] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0727] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0728] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0729] The present invention is a system for improving the rate at which missing persons are found. Its purpose is to efficiently find missing persons by generating predicted facial image data using a generative model based on facial image data of the missing person and then comparing this with video data from surveillance cameras installed in public institutions.

[0730] First, family members or other people involved (hereinafter referred to as "users") upload facial image data of the missing person to the system. This facial image data is a past photo of the person who went missing due to dementia or other factors, and in most cases, multiple photos are provided.

[0731] The server stores the facial image data received from the user in a database and then performs preprocessing, such as resizing the image, removing noise, and adjusting color tones, to convert it into a format suitable for input to the generative model.

[0732] The preprocessed facial image data is input into a generative model on the server. This generative model generates multiple predicted facial image data, taking into account changes such as aging and cosmetic surgery. These predicted facial image data are important for estimating the current appearance of the missing person.

[0733] Next, the generated predicted facial image data is stored on a server and linked to a surveillance camera system. Multiple surveillance cameras continuously capture video data in real time at stations and public places. The surveillance camera terminals then transmit this video data to the server in real time.

[0734] The server applies a facial recognition algorithm to extract facial images from the received video data. The extracted facial images are compared with predicted facial image data stored in a database to calculate a match rate. If the match rate exceeds a certain threshold, the server generates an alert.

[0735] The alert will include the matched facial image, the CCTV camera location, and the time the match was detected. The alert will be sent to a police device. The police device will receive the alert and provide on-site information to response teams. Police will then arrive at the scene to identify and protect the individual. After identification, police will contact the family and arrange for the missing person to be reunited with their family.

[0736] Specific examples

[0737] For example, consider a case where a person with dementia has gone missing. Family members upload past photos of the person to the system, along with basic information such as age and gender. The server preprocesses these images and generates multiple facial images that are input into a generative model to predict their current appearance. Then, facial images are extracted from real-time footage captured by surveillance cameras at station ticket gates and compared with the stored predicted facial images. If there is a high match, the server sends an alert to the police, who respond to the scene and protect the person. The family is then immediately contacted to ensure the reunion.

[0738] The above is an embodiment of the present invention, and it is expected that this process will significantly improve the early detection of missing persons and reunification with their families.

[0739] The processing flow will be explained below.

[0740] Step 1:

[0741] Users upload facial image data and basic information (such as name, age, gender, and last seen location) of missing people to the system, which is used to accurately understand the latest status of missing people.

[0742] Step 2:

[0743] The server stores the facial image data received from the user in a database in an appropriate format to maintain the quality of the image data.

[0744] Step 3:

[0745] The server preprocesses the stored facial image data, specifically by resizing the image, removing noise, adjusting color, etc., and converting it into a format suitable for the generative model.

[0746] Step 4:

[0747] The server inputs the preprocessed facial image data into a generative model. This generative model generates predicted facial image data, taking into account factors such as aging and cosmetic surgery. The generated predicted facial image data has multiple variations, each reflecting a different possibility.

[0748] Step 5:

[0749] The server stores the generated predicted facial image data in a database, which makes it available for subsequent facial image matching processes.

[0750] Step 6:

[0751] The terminal (surveillance camera system) acquires video data in real time. These surveillance cameras are installed in many public institutions and facilities and continuously transmit the video data to a server.

[0752] Step 7:

[0753] The server receives real-time video data transmitted from the surveillance cameras, which is then analyzed using a facial recognition algorithm to extract facial images.

[0754] Step 8:

[0755] The server compares the facial images extracted from the real-time video data with the stored predicted facial image data. A matching algorithm calculates the match rate and evaluates the likelihood of identifying the person.

[0756] Step 9:

[0757] The server generates an alert when the match rate exceeds a set threshold, and the alert includes the matching face image, the surveillance camera's location information, and time information.

[0758] Step 10:

[0759] The server sends the generated alert to the police terminal, which can then respond immediately to the incident.

[0760] Step 11:

[0761] After receiving the alert, police will head to the scene, identify the person, and take them into custody. After that, the police will confirm that the person is a missing person and contact their family.

[0762] Step 12:

[0763] Police will arrange for the missing person to be reunited with their family and return them safely home, a process that will ensure the early detection and reunion of missing people.

[0764] Example 1

[0765] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0766] Locating missing people often becomes more difficult as time passes. This becomes even more difficult when changes in appearance or age occur due to factors such as dementia. Technology is also needed to efficiently analyze video data acquired in real time from multiple surveillance devices. The present invention aims to solve these problems and improve the rate at which missing people are found.

[0767] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0768] In this invention, the server includes a means for preprocessing image data of the missing person and converting it into an appropriate format, a means for operating a generative model for generating predicted facial image data from the preprocessed image data, and a means for saving the generated predicted facial image data. This enables the generation of predicted facial image data that takes into account changes over time, such as changes in the patient's appearance or surgical procedures. The server also includes a means for acquiring video data from multiple monitoring devices in real time, extracting facial images from the video data, comparing them with the predicted facial image data, and calculating the match rate. This allows for the prompt generation of an alert and notification when the match rate exceeds a certain threshold.

[0769] "Missing person" refers to a person whose whereabouts are unknown.

[0770] "Image data" refers to data that digitally represents visual information, including the face of a missing person.

[0771] "Preprocessing" refers to processes such as resizing, noise removal, and color adjustment that are performed to convert image data into a format suitable for the generative model.

[0772] A "generative model" refers to artificial intelligence that generates predicted facial image data based on input past image data, taking into account changes over time, such as changes in appearance or surgical procedures.

[0773] "Predicted facial image data" refers to image data that predicts the current appearance of a missing person, generated by a generative model.

[0774] "Monitoring equipment" refers to devices installed at stations and public places that capture video data in real time.

[0775] "Video data" refers to real-time image and video data captured by a monitoring device.

[0776] "Facial image" refers to image data that includes the facial portion of a person extracted from video data.

[0777] "Matching" refers to the process of comparing the extracted facial image with predicted facial image data and calculating the match rate.

[0778] The "match rate" refers to the ratio indicating the degree to which the matched face image and predicted face image data match.

[0779] An "alert" refers to warning information that is generated when the match rate exceeds a certain threshold.

[0780] "Notification" refers to the process of communicating generated alerts to police and other relevant parties.

[0781] The present invention is a system for improving the rate at which missing persons are found. Specifically, the system generates predicted facial image data using a generative model based on facial image data of the missing person, and then compares this with video data from surveillance equipment installed in public institutions, thereby efficiently finding the missing person. The overall flow of the system will be explained in detail below, with the roles of the user, server, and terminal.

[0782] First, the user uploads facial image data of the missing person to the system. This facial image data is a past photograph of the missing person, and usually multiple photographs are provided. The user accesses a dedicated page of the system using a web browser, uploads any image of the missing person they have, and enters basic information such as the missing person's name, age, and gender.

[0783] Next, the server stores the facial image data received from the user in a database and performs preprocessing, including resizing the image, removing noise, and adjusting color tone, before converting it into a format suitable for input to the generative model. The hardware used is a server with high-performance computing power and large storage capacity (e.g., the Dell PowerEdge series), and the software used is a facial recognition library such as OpenCV or Dlib.

[0784] The facial image data after preprocessing is input into a generative model on the server. This generative model generates multiple predicted facial image data to estimate the missing person's current appearance, taking into account changes over time, such as changes due to surgery, etc. A generative AI model such as StyleGAN is used as this generative model. The generated predicted facial image data is stored on the server.

[0785] The server then connects to multiple monitoring devices. These devices continuously capture video data in real time at stations and public places and send it to the server. The terminals (monitoring devices) use a surveillance system that includes high-resolution cameras (e.g., cameras manufactured by Axis Communications).

[0786] The server applies a facial recognition algorithm to extract facial images from the received video data. The extracted facial images are compared with predicted facial image data stored in a database to calculate the match rate. Facial recognition algorithms used include OpenCV and Dlib. If the match rate exceeds a certain threshold, the server generates an alert. The alert includes the matched facial image, the location of the surveillance device, and the time the match was detected.

[0787] The alert is sent to the police terminal, which is a sophisticated computer system that receives the alert in real time. Once the police receive the alert, they will dispatch a response team to the scene to identify and protect the individual. After identification, the police will contact the family and arrange for the missing person to be reunited with their family.

[0788] Specific examples

[0789] For example, consider a case where a person with dementia goes missing. Family members upload past photos of the person to the system and enter basic information such as age and gender. The server preprocesses these images and generates multiple facial images that are input into a generative model to predict their current appearance. Then, a surveillance device extracts facial images from real-time footage taken at station ticket gates and compares them with the stored predicted facial images. If there is a high match, the server sends an alert to the police, who respond to the scene and protect the person. The family is then immediately contacted to ensure the reunion.

[0790] Example prompt sentence:

[0791] 1. "Generate facial image data to predict current appearance based on a facial photo taken 40 years ago."

[0792] 2. "Generate a current facial image of a missing person based on past photographs, taking into account age-related changes."

[0793] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0794] Step 1:

[0795] The user uploads facial image data of the missing person.

[0796] Input: Past facial images of the missing person, basic information such as name, age, and gender

[0797] Output: Sending face image data and basic information to the server

[0798] Specific operations: The user opens a web browser and accesses the system's dedicated page. They click the "Upload Images" button, select and upload multiple past photos of the missing person they have, and then enter basic information about the missing person into the form.

[0799] Step 2:

[0800] The server stores the facial image data received from the user in a database and performs preprocessing.

[0801] Input: Facial image data and basic information sent to the server

[0802] Output: Preprocessed image data stored in a database

[0803] Specific operation: The server records the received facial images in a database. Then, it performs preprocessing such as resizing, noise removal, and color adjustment to convert the images into a format suitable for input to the generative model.

[0804] Step 3:

[0805] The server inputs the preprocessed facial image data into a generative model to generate predicted facial image data.

[0806] Input: Preprocessed face image data

[0807] Output: Generated predicted face image data

[0808] Specific operation: The server inputs the preprocessed facial image into the generative model. The generative model generates multiple predicted facial image data, taking into account changes over time, such as aging and surgical procedures. The prompts used are, "Based on a facial photo from 40 years ago, please generate facial image data to predict current appearance," and "Based on past photographs of a missing person, please generate a current facial image that takes into account changes due to age."

[0809] Step 4:

[0810] The server stores the generated predicted face image data in a database.

[0811] Input: Generated predicted face image data

[0812] Output: Predicted face image data stored in a database

[0813] Specific operation: The server stores the generated predicted face image data in a database.

[0814] Step 5:

[0815] The terminal (monitoring device) transmits video data to the server in real time.

[0816] Input: Real-time video data captured by a surveillance camera

[0817] Output: Real-time video data sent to the server

[0818] Specific operation: The terminal (monitoring device) streams video footage taken at station ticket gates and public places to a server in real time.

[0819] Step 6:

[0820] The server extracts a facial image from the video data and compares it with predicted facial image data.

[0821] Input: Real-time video data

[0822] Output: Matching result (match rate)

[0823] Specific operation: The server applies a facial recognition algorithm to the received video data to extract facial images. The extracted facial images are compared with the stored predicted facial image data and the match rate is calculated. OpenCV and Dlib are used here.

[0824] Step 7:

[0825] The server generates an alert and notifies you if the match rate exceeds a certain threshold.

[0826] Input: Matching result (match rate)

[0827] Output: Alert information

[0828] Specific operation: If the calculated match rate exceeds a threshold, the server generates an alert. The alert includes the matched face image, the location information of the monitoring device, and the time when the match was detected. The generated alert is sent to the police terminal.

[0829] Step 8:

[0830] Police receive an alert, provide on-site information to response units, and initiate action.

[0831] Input: Alert information sent from the server

[0832] Output: On-site confirmation and protective action

[0833] Specific operations: The police terminal receives the alert and provides the relevant scene information to the response team. The police then go to the scene to identify and protect the person in question, and after confirmation, contact the family. Finally, they arrange for the missing person to be reunited with their family.

[0834] This concludes the detailed explanation of each processing step in the system program, which will enable efficient detection of missing persons and prompt response.

[0835] (Application example 1)

[0836] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0837] There is a need for an efficient and rapid method to improve the rate at which missing persons are found. However, current systems only use public surveillance cameras, and efficient searches using mobile information devices owned by individuals or at home are not performed. Furthermore, there is a lack of reliable predictive methods for responding to changes in facial images. As a result, searches for missing persons are delayed and the rate at which they are found decreases. Therefore, there is a need for a rapid and widespread method for finding missing persons.

[0838] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0839] In this invention, the server includes means for acquiring image data of a missing person, means for operating a generative model for generating predicted facial image data from the image data, means for saving the generated predicted facial image data, means for acquiring video data in real time from a surveillance camera or a camera of a mobile information terminal, means for extracting a facial image from the video data, means for comparing the extracted facial image with the predicted facial image data and calculating a match rate, means for generating and notifying an alert when the match rate exceeds a certain threshold, and means for sharing location information based on the alert, thereby enabling a wide-area search for missing people in real time.

[0840] A "missing person" is someone whose current whereabouts are unknown and who requires a search.

[0841] "Image data" refers to digital data that includes past photographs of faces and other visual information about missing persons.

[0842] A "generative model" is an artificial intelligence model that predicts and generates current facial image data from input past facial image data.

[0843] A "personal digital assistant" is a portable device such as a smartphone or tablet.

[0844] A "surveillance camera" is a camera device installed in a public place that continuously records and monitors footage.

[0845] "Acquiring video data in real time" means that the video obtained from the camera is immediately sent to the system for processing.

[0846] "Extracting a face image" means cutting out and acquiring only the face portion of a person from video data.

[0847] "Matching" means comparing and verifying a match against different data sets.

[0848] The "match rate" is a ratio indicating the degree to which the extracted face image and the predicted face image data are identical.

[0849] An "alert" is a warning or notification that is generated when the match rate exceeds a certain threshold.

[0850] "Sharing location information" means sending and receiving geographical data of the location where a missing person was found between multiple devices.

[0851] The present invention is a system for efficiently locating missing persons. This system consists of three main components: a server, a terminal, and a user.

[0852] First, the user uploads past facial image data of the missing person to the system. This can be done using a device such as a smartphone or PC. The image data provided by the user is sent to the server. The server receives this data and performs preprocessing. This preprocessing involves resizing the image, removing noise, and adjusting the color tone. This process converts the data into a format suitable for the generative model.

[0853] Next, a generative model generates predicted facial image data from the preprocessed facial image data. This generative model outputs multiple predicted facial images, taking into account facial changes such as aging and cosmetic surgery. This predicted facial image data is stored in a database on the server.

[0854] The server receives video data in real time from surveillance cameras and mobile devices. For example, when a smartphone camera captures video in a public place, the video data is sent to the server. The server then applies a facial recognition algorithm to extract facial images from the video data. This algorithm can use models such as InceptionResnetV1 or MTCNN.

[0855] The extracted facial image is compared with predicted facial image data stored in the server's database. At this time, the match rate is calculated. If the match rate exceeds a certain threshold, the server generates an alert. This alert includes the matched facial image, detected location information, and time information. The alert is sent to the devices of the police and relevant parties.

[0856] Additionally, the alert notification includes a location sharing function, enabling a rapid response if a missing person is found. For example, the smartphone app can send geographical data of the detected location to a server, allowing police or family members to receive the data and take action.

[0857] Examples:

[0858] Below are some example prompts for the generative AI model:

[0859] "Generate a current image of a 50-year-old woman's face from an image taken 20 years ago."

[0860] "Please output an image of a 20-year-old man's face converted to look like a 30-year-old man."

[0861] This enables wide-area and real-time searches for missing people. The server calculates the facial image match rate and can quickly generate an alert and notify relevant parties if a match exceeds a certain threshold. This enables the early detection and protection of missing people and promotes reunification with their families.

[0862] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0863] Step 1:

[0864] The user uploads the missing person's past facial image data to the system. The user uses a device such as a smartphone or PC to send the image data to the server. The input at this time is the missing person's past facial image data, and the output is the data transferred to the server.

[0865] Step 2:

[0866] The server preprocesses the facial image data received from the user. It resizes the image, removes noise, adjusts color tone, and converts the data format into a format that can be input to the generative model. The input of this process is the uploaded facial image data, and the output is the preprocessed facial image data.

[0867] Step 3:

[0868] The server inputs the preprocessed facial image data into a generative model, which generates multiple predicted facial image data by taking into account factors such as aging and cosmetic surgery. The input is the preprocessed facial image data, and the output is predicted facial image data.

[0869] Step 4:

[0870] The generated predicted facial image data is stored in a database on the server. This database is used in subsequent facial recognition processes. The input is the predicted facial image data, and the output is the data stored in the database.

[0871] Step 5:

[0872] A camera on a device (e.g., a smartphone) or a public surveillance camera captures video data in real time. The captured video data is sent to a server. The input is the real-time video data, and the output is the data sent to the server.

[0873] Step 6:

[0874] The server applies a facial recognition algorithm to extract facial images from the transmitted video data, such as InceptionResnetV1 or MTCNN. The input of this process is real-time video data, and the output is extracted facial image data.

[0875] Step 7:

[0876] The extracted facial image data is compared with predicted facial image data stored in the server's database, and a match rate is calculated. This calculation confirms whether the image matches the missing person. The input is the extracted facial image data and predicted facial image data, and the output is the calculated match rate.

[0877] Step 8:

[0878] If the match rate exceeds a certain threshold, the server generates an alert, which includes the matched face image, location information, and time information. The input is the situation where the match rate exceeds a certain threshold, and the output is the generated alert.

[0879] Step 9:

[0880] The generated alert is sent to the devices of the police and other relevant parties, and a response is initiated at the scene. The input is the generated alert, and the output is the notified information. The police and other relevant parties then proceed to the scene based on the location information to identify and protect the missing person.

[0881] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0882] The present invention is a system for improving the rate at which missing persons are found, and aims to efficiently find missing persons by generating predicted facial image data using a generative model based on facial image data of the missing person and comparing this with video data from surveillance cameras installed in public institutions.Furthermore, the system is characterized by improving the responsiveness of the system by combining it with an emotion engine that recognizes the user's emotions.

[0883] First, family members and other people involved (hereafter referred to as "users") upload facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. The emotion engine then analyzes the user's emotions in real time and optimizes the operation of the entire system based on that information.

[0884] The server stores the facial image data received from the user in a database and then preprocesses it. This involves resizing the image, removing noise, adjusting color, and converting it into a format suitable for input to the generative model. The emotion engine also obtains the user's emotional data during this process, which can be used as additional information to help search for missing people.

[0885] The facial image data after preprocessing is input into a generative model on the server. This generative model generates multiple predicted facial image data, taking into account changes such as aging and cosmetic surgery. The generated predicted facial image data is important for estimating what the missing person currently looks like.

[0886] Next, the generated predicted facial image data is stored on a server and linked to a surveillance camera system. Multiple surveillance cameras continuously capture video data in real time at stations and public places. The surveillance camera terminals then transmit this video data to the server in real time.

[0887] The server applies a facial recognition algorithm to extract facial images from the received video data. An emotion engine then analyzes the user's emotional data and sets optimal parameters for searching for the missing person. The extracted facial images are compared with predicted facial image data stored in a database to calculate the match rate. If the match rate exceeds a certain threshold, the server generates an alert.

[0888] The alert includes the matched facial image, the surveillance camera's location, and the time the match was detected. The alert is sent to the police device. The emotion engine takes the user's emotional state into account to customize the alert content and notification method. The police device receives the alert and provides on-site information to response teams. The police then arrive at the scene to identify and protect the person. After identification, the police contact the family and arrange for the missing person to be reunited with their family.

[0889] Specific examples

[0890] For example, consider a case where a person with dementia has gone missing. Family members upload past photos of the person to the system, along with basic information such as age and gender. The server preprocesses these images and inputs them into a generative model to generate multiple facial images for predicting their current appearance. The emotion engine also analyzes the family's emotional state of concern and adjusts the system's overall responsiveness and alertness. Next, facial images are extracted from real-time footage captured by surveillance cameras at station ticket gates and compared with the stored predicted facial images. If the match rate is high, the server sends an alert to the police, who then dispatch to the scene to protect the person. The family is then immediately contacted, enabling the reunion. The emotion engine's analysis of the user's emotions also allows for efficient care and support after the reunion.

[0891] The above is an embodiment of the present invention, and by combining this process with an emotion engine, it is expected that the early discovery and reunion of missing persons will proceed even more smoothly.

[0892] The processing flow will be explained below.

[0893] Step 1:

[0894] The user uploads facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. At the same time, the user inputs emotional data (such as worry, anxiety, and impatience). The emotional data can be in text format or voice input.

[0895] Step 2:

[0896] The server stores the facial image data and emotion data received from the user in a database, with the facial image data stored in high quality for subsequent processing.

[0897] Step 3:

[0898] The server preprocesses the facial image data, specifically resizing the image, removing noise, adjusting color, etc. The preprocessed facial image data is then converted into a suitable format for input to the generative model.

[0899] Step 4:

[0900] The server inputs the preprocessed facial image data into a generative model, which then generates multiple predicted facial image data, taking into account factors such as aging and cosmetic surgery.

[0901] Step 5:

[0902] The server stores the generated predicted facial image data in a database. At this stage, the system has multiple facial image variations that can be used to more easily identify missing people.

[0903] Step 6:

[0904] The server uses an emotion engine to analyze the user's emotion data. Based on the analysis results, the system adjusts its responsiveness and alertness. For example, if the user shows a high level of impatience, the system will use a high-precision model to generate predicted facial images more quickly.

[0905] Step 7:

[0906] The terminals (surveillance camera systems) acquire video data in real time. These surveillance cameras are installed in stations and public places, and continuously transmit the video data to a server.

[0907] Step 8:

[0908] The server receives real-time video data transmitted from the surveillance cameras, which is analyzed using a facial recognition algorithm to extract facial images.

[0909] Step 9:

[0910] The server compares the extracted facial image with predicted facial image data stored in a database, calculates the match rate, and generates an alert if the match rate exceeds a certain threshold.

[0911] Step 10:

[0912] The server then sends the generated alert to the police terminal, which includes the matching face image, the location information of the surveillance camera, and the time when the match was detected.

[0913] Step 11:

[0914] Police devices receive the alert and provide on-site information to responding teams, who then arrive on-site to identify and protect the individual.

[0915] Step 12:

[0916] Once the police confirm that the person is missing, they will contact the family. Based on the user's emotional state, the emotion engine will customize the notification content and contact the user in the most appropriate way to reassure them.

[0917] Step 13:

[0918] Police will arrange for missing people to be reunited with their families and return home safely, and the emotion engine will analyze the user's emotions and coordinate support and care after the reunion.

[0919] These steps will ensure that missing persons are quickly identified and reunited with their families.

[0920] Example 2

[0921] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0922] The present invention aims to improve the rate at which missing people are found, and to solve the problems of cases where finding a missing person is difficult using conventional search methods, or where searches take a long time. Furthermore, since a flexible response that takes into account the user's emotions is required, a system that can optimize search activities and provide care after a person is found is also needed.

[0923] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring image data of a missing person, means for operating a generative model for generating predicted face image data from the image data, means for saving the generated predicted face image data, means for acquiring video data from a surveillance camera in real time, means for extracting a face image from the video data, means for matching the extracted face image with the predicted face image data and calculating a match rate, means for generating and notifying an alert when the match rate exceeds a certain threshold, means for analyzing emotional data acquired from a user, and means for optimizing the operation of the entire system based on the emotional data. This enables the missing person to be found quickly and efficiently, and also allows for flexible responses that take user emotions into consideration.

[0924] "Image data of a missing person" is digital data that shows the facial image and physical characteristics of a missing person.

[0925] A "generative model" refers to an algorithm or program that uses artificial intelligence to generate predicted facial image data from input image data.

[0926] "Predicted facial image data" is image data that predicts the future appearance of a missing person, taking into account changes such as aging and cosmetic surgery.

[0927] A "surveillance camera" is an electronic device installed in a public place to capture and record video in real time.

[0928] "Means of acquiring video data in real time" refers to the technology and equipment used to collect ongoing video footage from surveillance cameras and send it to a server.

[0929] "Means for extracting facial images" refers to an algorithm or technology for identifying a person's face from video data and extracting it as a facial image.

[0930] The "means for calculating the matching rate" refers to an algorithm or program that compares the extracted facial image with predicted facial image data and calculates the degree of matching as a numerical value.

[0931] "Means for generating and notifying alerts" refers to a system for creating a warning message and notifying relevant organizations and users when the match rate exceeds a certain threshold.

[0932] "Emotion data" is data that expresses the user's emotional state numerically or by category.

[0933] "Means for analyzing emotional data" refers to algorithms or technologies that analyze the emotional state of a user from input, facial expressions, and voice and obtain the data.

[0934] "Means for optimizing the operation of the entire system" refers to techniques and methods for adjusting the operation and response of each part of the system based on the acquired emotional data, allowing it to operate in the most effective manner.

[0935] The present invention is a system for improving the rate at which missing persons are found, and aims to efficiently find missing persons by generating predicted facial image data based on facial image data of the missing person and comparing this with video data from surveillance cameras installed in public institutions.Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system's responsiveness is improved.

[0936] First, family members or people involved (hereinafter referred to as "users") upload facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. The uploaded data is then sent to the server, which stores it in a database and begins pre-processing the image data. This pre-processing includes processes such as image resizing, noise removal, and color adjustment.

[0937] The server inputs the preprocessed facial image data into a generative AI model (e.g., StyleGAN) to generate multiple predicted facial image data that take into account changes over time, such as cosmetic surgery, etc. The generated predicted facial image data is then stored in a database by the server.

[0938] Meanwhile, surveillance cameras are installed in stations and public places and capture video data in real time. The surveillance camera terminals send the captured video data to a server. The server uses facial recognition algorithms such as OpenCV and Dlib to extract facial images from the video data and compare them with predicted facial image data stored in a database. The server calculates the match rate and generates an alert if this match rate exceeds a certain threshold.

[0939] The generated alert includes the matched facial image, the location of the surveillance camera, and the time the match was detected. The alert is sent to the device of the responding agency, such as the police, who will then go to the site to identify and protect the person and arrange for them to be reunited with their family.

[0940] The emotion engine also analyzes users' emotional data and optimizes the operation of the entire system based on that data. Specifically, when a user uploads data, the emotion engine analyzes the emotions in real time from the user's face and sentences. The server obtains the analyzed emotional data and adjusts the search priority and alert content.

[0941] Specific examples

[0942] For example, consider a case where a person with dementia has gone missing. Family members upload past photos of the person to the system, along with basic information such as age and gender. The server preprocesses these images and inputs them into a generative model to generate multiple facial images for predicting their current appearance. The emotion engine also analyzes the family's emotional state of concern and adjusts the system's overall responsiveness and alertness. Next, facial images are extracted from real-time footage captured by surveillance cameras at station ticket gates and compared with the stored predicted facial images. If the match rate is high, the server sends an alert to the police, who then dispatch to the scene to protect the person. The family is then immediately contacted, enabling the reunion. The emotion engine's analysis of the user's emotions also allows for efficient care and support after the reunion.

[0943] Prompt Sentence Examples

[0944] "Predict the current appearance of a person who has gone missing due to dementia based on past facial photographs. Age: 70, gender: male, last seen in: XX City, XX Prefecture."

[0945] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0946] Step 1:

[0947] The user uploads facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. This data is then sent to the server. At this time, the facial image data and basic information are prepared as input, and the raw facial image data and basic information stored on the server are obtained as output. Specifically, the user selects an image file from a dedicated web form, enters basic information in the text box, and presses the send button.

[0948] Step 2:

[0949] The server stores the received facial image data in a database and begins preprocessing. Preprocessing includes image resizing, noise removal, and color adjustment. The input here is the raw facial image data stored on the server, and the output is preprocessed facial image data converted into a format suitable for the generative AI model. Specifically, the server uses an image processing library (e.g., OpenCV) to resize the facial image (e.g., to 256x256 pixels), apply a noise removal filter, and equalize the color tone.

[0950] Step 3:

[0951] The server inputs the preprocessed facial image data into a generative AI model (e.g., StyleGAN) to generate multiple predicted facial image data that take into account changes such as aging and cosmetic surgery. The input here is preprocessed facial image data, and the output is predicted facial image data. Specifically, the server sends an API request to the generative model and stores the generated results in a database.

[0952] Step 4:

[0953] The server stores the generated predicted face image data in a database. The input of this step is the predicted face image data output from the generative AI model, and the output is the predicted face image data stored in the database. Specifically, the server uses a database management system (e.g., MySQL) to store the predicted face image data in an appropriate format.

[0954] Step 5:

[0955] Surveillance camera terminals continuously capture video data in real time at stations and public places. The captured video data is sent to a server. The input of this step is the real-time video data, and the output is the video data sent to the server. Specifically, the surveillance camera terminals have the function of sending video streams to the server over the network (e.g., using the RTSP protocol).

[0956] Step 6:

[0957] The server applies a facial recognition algorithm to extract facial images from the received video data. The input to this step is the video data sent to the server, and the output is the facial images extracted from the video data. Specifically, the server uses libraries such as OpenCV and Dlib to detect facial areas from the video frames and extract facial images.

[0958] Step 7:

[0959] The server matches the extracted facial image with predicted facial image data stored in a database. The inputs to this step are the extracted facial image and predicted facial image data, and the output is a match rate. Specifically, the server uses a facial recognition algorithm to compare the features of each facial image and calculate the match rate (e.g., cosine similarity).

[0960] Step 8:

[0961] The server generates an alert and notifies if the match rate exceeds a certain threshold. The input of this step is the calculated match rate, and the output is the generated alert. Specifically, the server generates an alert message when the match rate exceeds a set threshold (e.g., 90%) and sends an alert to the police or related organizations through the notification system.

[0962] Step 9:

[0963] The emotion engine analyzes the user's emotional data and optimizes the overall system behavior based on that data. The input to this step is the user's emotional data, and the output is optimized system behavior. Specifically, the emotion engine analyzes the facial expressions and sentences used when the user uploads data to the system, and sends the estimated emotional state as numerical data to the server. The server then adjusts responsiveness and alert content based on this data.

[0964] (Application example 2)

[0965] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0966] Conventional missing person search systems have had problems with low accuracy in handling facial image data and matching it with actual facial images, resulting in low rates of finding missing persons. In particular, as time passes, the appearance of a missing person changes, further reducing the accuracy of matching. Furthermore, because searches are performed without taking the user's emotional state into consideration, there are also problems with low responsiveness and efficiency of the entire system. The purpose of the present invention is to solve these problems and improve the rate of finding missing persons and the responsiveness of the system.

[0967] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0968] In this invention, the server includes means for acquiring image data of a missing person, means for operating a generative model for generating predicted facial image data from the image data, means for saving the generated predicted facial image data, means for acquiring video data from a surveillance camera in real time, means for extracting a facial image from the video data, means for comparing the extracted facial image with the predicted facial image data and calculating a match rate, means for generating and notifying an alert when the match rate exceeds a certain threshold, means for analyzing user emotional data and optimizing system operation, and means for inputting and managing information about the missing person using an application installed on a smart device. This enables the generation of predicted facial images that take into account changes in the missing person's appearance and optimal system operation based on the user's emotional state, improving the rate at which missing people are found and the responsiveness of the entire system.

[0969] "Image data of a missing person" is data that includes a facial image of a missing person and related personal information.

[0970] A "generative model" is a machine learning algorithm for generating predicted facial image data from input facial image data of a missing person.

[0971] "Predicted facial image data" is facial image data obtained by a generative model that predicts the current appearance of a missing person.

[0972] A "surveillance camera" is a device that is installed in public facilities or public places and captures video data in real time.

[0973] "Video data" refers to real-time video information acquired from a surveillance camera.

[0974] "Means for extracting facial images" refers to techniques or algorithms for extracting only the facial portion from video data.

[0975] The "means for matching and calculating the degree of match" is a technique for comparing the extracted face image with predicted face image data and quantifying the degree of similarity between the two.

[0976] "Means for generating and notifying alerts" refers to technology that generates an alert when the matching rate exceeds a certain threshold and notifies relevant institutions and parties of that information.

[0977] "Means for analyzing user emotional data and optimizing system operation" refers to technology that analyzes the user's emotional state and adjusts the system's responsiveness and alertness.

[0978] An "application installed on a smart device" is application software installed on a mobile device such as a smartphone or smart glasses for inputting and managing information about missing persons.

[0979] "Preprocessing means" refers to a technique for performing processes such as noise removal and resizing before inputting facial image data into a generative model.

[0980] This invention is a system for improving the rate at which missing people are found. The system is mainly composed of a server, a surveillance camera, an application, and an emotion analysis engine.

[0981] 1. System Configuration

[0982] server:

[0983] The server is the central data processing unit and performs the following main processes:

[0984] Data Acquisition: Acquires image data and basic information of the missing person provided by the user.

[0985] Generative model: Operate a generative AI model to generate predicted facial image data from the provided image data. Here, TensorFlow or PyTorch are used as the generative AI model.

[0986] Data storage: The generated predicted face image data is stored in a database.

[0987] Video data acquisition: Acquire video data from surveillance cameras in real time.

[0988] Facial image extraction: Facial images are extracted from video data using OpenCV.

[0989] Matching and match rate calculation: Using a face recognition library such as Dlib, the extracted face image is matched with the predicted face image data and the match rate is calculated.

[0990] Alert generation: Provides a function to generate an alert and notify when the match rate exceeds a certain threshold. Firebase Cloud Messaging is used for alert notification.

[0991] Emotion analysis: Analyzes user emotional data using emotion analysis engines such as Affectiva. Optimize system operation based on the results of this analysis.

[0992] Surveillance cameras:

[0993] It is installed in public places and is responsible for transmitting video data to a server in real time. Video data is managed via a network video recorder (NVR).

[0994] Smart device applications:

[0995] An application that allows users to input and manage information about missing persons through smart devices such as smartphones or smart glasses, and provides specific prompts for:

[0996] "Please upload a photo of the subject's face. Enter basic information (name, age, gender, last seen location)."

[0997] "We will optimize search parameters based on the user's emotional state. We are analyzing emotional data in real time."

[0998] 2. Working Example

[0999] Examples:

[1000] For example, if someone with dementia goes missing, a family member can use a smartphone application to upload a past photo of the person to the system, along with basic information such as name, age, gender, and last known location.

[1001] Preprocessing of facial image data:

[1002] The server performs preprocessing such as resizing the images, removing noise, and adjusting color tones, and converts them into a format that can be input to the generative model. This processing is done using OpenCV.

[1003] Generate predicted face images:

[1004] The preprocessed facial image data is input into the server's generative model (TensorFlow or PyTorch), and multiple predicted facial image data are generated, taking into account changes such as aging and cosmetic surgery.

[1005] Real-time video data collection and analysis:

[1006] Surveillance cameras capture video data in real time at stations and public places and send it to a server, which then extracts facial images from the video data and compares them with predicted facial image data stored in a database.

[1007] Alert Notifications:

[1008] If the match rate of the matched facial images exceeds a certain threshold, the server generates an alert and notifies police and other relevant parties in real time using Firebase Cloud Messaging, allowing relevant authorities to respond to the scene and protect the individual.

[1009] The above process enables the early detection and protection of missing persons, and also makes it possible to optimize the system's responsiveness by reflecting the results of user emotion analysis.

[1010] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1011] Step 1:

[1012] User Input

[1013] Using a smart device application, users input a facial image and basic information (name, age, gender, last seen location) of the missing person, which is then sent to a server via the application.

[1014] Input: Facial image data and basic information of the missing person

[1015] Output: Facial image data and basic information sent to the server

[1016] Step 2:

[1017] Image data preprocessing

[1018] The server processes the received facial image data. This preprocessing involves resizing the image, removing noise, adjusting color tone, etc., and converting it into a format that can be input to the generative model. OpenCV is used here.

[1019] Input: Acquired facial image data

[1020] Output: Preprocessed face image data

[1021] Specific operations: resize image (e.g., change to 128x128 pixels), remove noise (apply Gaussian filter), adjust color tone (convert to grayscale)

[1022] Step 3:

[1023] Generating predicted face images

[1024] The preprocessed facial image data is input into a generative model on the server to generate a predicted facial image that reflects the missing person's current appearance. This generation is done using TensorFlow and PyTorch. The generated predicted facial image data is stored in a database.

[1025] Input: Preprocessed face image data

[1026] Output: Predicted face image data

[1027] Specific operation: An image is input into the generative model, and multiple predicted facial images are generated, taking into account changes such as aging and cosmetic surgery.

[1028] Step 4:

[1029] Collecting video data from surveillance cameras

[1030] The server receives video data in real time from surveillance cameras installed in stations and public places, and manages the video data via a network video recorder (NVR).

[1031] Input: Surveillance camera video data

[1032] Output: Real-time video data sent to the server

[1033] Specific operation: Video data from a surveillance camera is sent to a server via a network.

[1034] Step 5:

[1035] Facial image extraction from video data

[1036] The server uses OpenCV to extract facial images from the video data received in real time, resulting in image data with only the facial area extracted.

[1037] Input: Real-time video data

[1038] Output: Extracted face image

[1039] Specific operation: Detects faces from video frames and extracts only the facial parts

[1040] Step 6:

[1041] Facial image matching and matching rate calculation

[1042] The server uses a facial recognition library such as Dlib to match the extracted facial image with stored predicted facial image data and calculate the match rate.

[1043] Input: Extracted face image, predicted face image data

[1044] Output: Match rate

[1045] Specific operation: Calculate the similarity between the two using a face recognition algorithm

[1046] Step 7:

[1047] Alerting and Notifications

[1048] If the match rate exceeds a certain threshold, the server generates an alert using Firebase Cloud Messaging to notify relevant authorities and relatives, including information such as the matched face image, the detection location, and the detection time.

[1049] Input: Match rate, face image, location information, time information

[1050] Output: Alert notification

[1051] What it does: Sends alerts to mobile devices and computers using Firebase Cloud Messaging

[1052] Step 8:

[1053] Emotional data analysis and system optimization

[1054] The server uses an emotion analysis engine such as Affectiva to analyze user emotion data and optimize the responsiveness and alertness of the entire system.

[1055] Input: User emotion data

[1056] Output: Optimized system response

[1057] Specific operation: Drives the emotion analysis engine and adjusts the system's operating parameters based on the analysis results.

[1058] In this way, the entire system goes through a series of processing steps to help find missing people early and realize optimal search activities that take into account the user's emotional state.

[1059] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1060] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1061] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1062] [Fourth embodiment]

[1063] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1064] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1065] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1066] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1067] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1068] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1069] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1070] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1071] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1072] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1073] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1074] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1075] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1076] The present invention is a system for improving the rate at which missing persons are found. Its purpose is to efficiently find missing persons by generating predicted facial image data using a generative model based on facial image data of the missing person and then comparing this with video data from surveillance cameras installed in public institutions.

[1077] First, family members or other people involved (hereinafter referred to as "users") upload facial image data of the missing person to the system. This facial image data is a past photo of the person who went missing due to dementia or other factors, and in most cases, multiple photos are provided.

[1078] The server stores the facial image data received from the user in a database and then performs preprocessing, such as resizing the image, removing noise, and adjusting color tones, to convert it into a format suitable for input to the generative model.

[1079] The preprocessed facial image data is input into a generative model on the server. This generative model generates multiple predicted facial image data, taking into account changes such as aging and cosmetic surgery. These predicted facial image data are important for estimating the current appearance of the missing person.

[1080] Next, the generated predicted facial image data is stored on a server and linked to a surveillance camera system. Multiple surveillance cameras continuously capture video data in real time at stations and public places. The surveillance camera terminals then transmit this video data to the server in real time.

[1081] The server applies a facial recognition algorithm to extract facial images from the received video data. The extracted facial images are compared with predicted facial image data stored in a database to calculate a match rate. If the match rate exceeds a certain threshold, the server generates an alert.

[1082] The alert will include the matched facial image, the CCTV camera location, and the time the match was detected. The alert will be sent to a police device. The police device will receive the alert and provide on-site information to response teams. Police will then arrive at the scene to identify and protect the individual. After identification, police will contact the family and arrange for the missing person to be reunited with their family.

[1083] Specific examples

[1084] For example, consider a case where a person with dementia has gone missing. Family members upload past photos of the person to the system, along with basic information such as age and gender. The server preprocesses these images and generates multiple facial images that are input into a generative model to predict their current appearance. Then, facial images are extracted from real-time footage captured by surveillance cameras at station ticket gates and compared with the stored predicted facial images. If there is a high match, the server sends an alert to the police, who respond to the scene and protect the person. The family is then immediately contacted to ensure the reunion.

[1085] The above is an embodiment of the present invention, and it is expected that this process will significantly improve the early detection of missing persons and reunification with their families.

[1086] The processing flow will be explained below.

[1087] Step 1:

[1088] Users upload facial image data and basic information (such as name, age, gender, and last seen location) of missing people to the system, which is used to accurately understand the latest status of missing people.

[1089] Step 2:

[1090] The server stores the facial image data received from the user in a database in an appropriate format to maintain the quality of the image data.

[1091] Step 3:

[1092] The server preprocesses the stored facial image data, specifically by resizing the image, removing noise, adjusting color, etc., and converting it into a format suitable for the generative model.

[1093] Step 4:

[1094] The server inputs the preprocessed facial image data into a generative model. This generative model generates predicted facial image data, taking into account factors such as aging and cosmetic surgery. The generated predicted facial image data has multiple variations, each reflecting a different possibility.

[1095] Step 5:

[1096] The server stores the generated predicted facial image data in a database, which makes it available for subsequent facial image matching processes.

[1097] Step 6:

[1098] The terminal (surveillance camera system) acquires video data in real time. These surveillance cameras are installed in many public institutions and facilities and continuously transmit the video data to a server.

[1099] Step 7:

[1100] The server receives real-time video data transmitted from the surveillance cameras, which is then analyzed using a facial recognition algorithm to extract facial images.

[1101] Step 8:

[1102] The server compares the facial images extracted from the real-time video data with the stored predicted facial image data. A matching algorithm calculates the match rate and evaluates the likelihood of identifying the person.

[1103] Step 9:

[1104] The server generates an alert when the match rate exceeds a set threshold, and the alert includes the matching face image, the surveillance camera's location information, and time information.

[1105] Step 10:

[1106] The server sends the generated alert to the police terminal, which can then respond immediately to the incident.

[1107] Step 11:

[1108] After receiving the alert, police will head to the scene, identify the person, and take them into custody. After that, the police will confirm that the person is a missing person and contact their family.

[1109] Step 12:

[1110] Police will arrange for the missing person to be reunited with their family and return them safely home, a process that will ensure the early detection and reunion of missing people.

[1111] Example 1

[1112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1113] Locating missing people often becomes more difficult as time passes. This becomes even more difficult when changes in appearance or age occur due to factors such as dementia. Technology is also needed to efficiently analyze video data acquired in real time from multiple surveillance devices. The present invention aims to solve these problems and improve the rate at which missing people are found.

[1114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1115] In this invention, the server includes a means for preprocessing image data of the missing person and converting it into an appropriate format, a means for operating a generative model for generating predicted facial image data from the preprocessed image data, and a means for saving the generated predicted facial image data. This enables the generation of predicted facial image data that takes into account changes over time, such as changes in the patient's appearance or surgical procedures. The server also includes a means for acquiring video data from multiple monitoring devices in real time, extracting facial images from the video data, comparing them with the predicted facial image data, and calculating the match rate. This allows for the prompt generation of an alert and notification when the match rate exceeds a certain threshold.

[1116] "Missing person" refers to a person whose whereabouts are unknown.

[1117] "Image data" refers to data that digitally represents visual information, including the face of a missing person.

[1118] "Preprocessing" refers to processes such as resizing, noise removal, and color adjustment that are performed to convert image data into a format suitable for the generative model.

[1119] A "generative model" refers to artificial intelligence that generates predicted facial image data based on input past image data, taking into account changes over time, such as changes in appearance or surgical procedures.

[1120] "Predicted facial image data" refers to image data that predicts the current appearance of a missing person, generated by a generative model.

[1121] "Monitoring equipment" refers to devices installed at stations and public places that capture video data in real time.

[1122] "Video data" refers to real-time image and video data captured by a monitoring device.

[1123] "Facial image" refers to image data that includes the facial portion of a person extracted from video data.

[1124] "Matching" refers to the process of comparing the extracted facial image with predicted facial image data and calculating the match rate.

[1125] The "match rate" refers to the ratio indicating the degree to which the matched face image and predicted face image data match.

[1126] An "alert" refers to warning information that is generated when the match rate exceeds a certain threshold.

[1127] "Notification" refers to the process of communicating generated alerts to police and other relevant parties.

[1128] The present invention is a system for improving the rate at which missing persons are found. Specifically, the system generates predicted facial image data using a generative model based on facial image data of the missing person, and then compares this with video data from surveillance equipment installed in public institutions, thereby efficiently finding the missing person. The overall flow of the system will be explained in detail below, with the roles of the user, server, and terminal.

[1129] First, the user uploads facial image data of the missing person to the system. This facial image data is a past photograph of the missing person, and usually multiple photographs are provided. The user accesses a dedicated page of the system using a web browser, uploads any image of the missing person they have, and enters basic information such as the missing person's name, age, and gender.

[1130] Next, the server stores the facial image data received from the user in a database and performs preprocessing, including resizing the image, removing noise, and adjusting color tone, before converting it into a format suitable for input to the generative model. The hardware used is a server with high-performance computing power and large storage capacity (e.g., the Dell PowerEdge series), and the software used is a facial recognition library such as OpenCV or Dlib.

[1131] The facial image data after preprocessing is input into a generative model on the server. This generative model generates multiple predicted facial image data to estimate the missing person's current appearance, taking into account changes over time, such as changes due to surgery, etc. A generative AI model such as StyleGAN is used as this generative model. The generated predicted facial image data is stored on the server.

[1132] The server then connects to multiple monitoring devices. These devices continuously capture video data in real time at stations and public places and send it to the server. The terminals (monitoring devices) use a surveillance system that includes high-resolution cameras (e.g., cameras manufactured by Axis Communications).

[1133] The server applies a facial recognition algorithm to extract facial images from the received video data. The extracted facial images are compared with predicted facial image data stored in a database to calculate the match rate. Facial recognition algorithms used include OpenCV and Dlib. If the match rate exceeds a certain threshold, the server generates an alert. The alert includes the matched facial image, the location of the surveillance device, and the time the match was detected.

[1134] The alert is sent to the police terminal, which is a sophisticated computer system that receives the alert in real time. Once the police receive the alert, they will dispatch a response team to the scene to identify and protect the individual. After identification, the police will contact the family and arrange for the missing person to be reunited with their family.

[1135] Specific examples

[1136] For example, consider a case where a person with dementia goes missing. Family members upload past photos of the person to the system and enter basic information such as age and gender. The server preprocesses these images and generates multiple facial images that are input into a generative model to predict their current appearance. Then, a surveillance device extracts facial images from real-time footage taken at station ticket gates and compares them with the stored predicted facial images. If there is a high match, the server sends an alert to the police, who respond to the scene and protect the person. The family is then immediately contacted to ensure the reunion.

[1137] Example prompt sentence:

[1138] 1. "Generate facial image data to predict current appearance based on a facial photo taken 40 years ago."

[1139] 2. "Generate a current facial image of a missing person based on past photographs, taking into account age-related changes."

[1140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1141] Step 1:

[1142] The user uploads facial image data of the missing person.

[1143] Input: Past facial images of the missing person, basic information such as name, age, and gender

[1144] Output: Sending face image data and basic information to the server

[1145] Specific operations: The user opens a web browser and accesses the system's dedicated page. They click the "Upload Images" button, select and upload multiple past photos of the missing person they have, and then enter basic information about the missing person into the form.

[1146] Step 2:

[1147] The server stores the facial image data received from the user in a database and performs preprocessing.

[1148] Input: Facial image data and basic information sent to the server

[1149] Output: Preprocessed image data stored in a database

[1150] Specific operation: The server records the received facial images in a database. Then, it performs preprocessing such as resizing, noise removal, and color adjustment to convert the images into a format suitable for input to the generative model.

[1151] Step 3:

[1152] The server inputs the preprocessed facial image data into a generative model to generate predicted facial image data.

[1153] Input: Preprocessed face image data

[1154] Output: Generated predicted face image data

[1155] Specific operation: The server inputs the preprocessed facial image into the generative model. The generative model generates multiple predicted facial image data, taking into account changes over time, such as aging and surgical procedures. The prompts used are, "Based on a facial photo from 40 years ago, please generate facial image data to predict current appearance," and "Based on past photographs of a missing person, please generate a current facial image that takes into account changes due to age."

[1156] Step 4:

[1157] The server stores the generated predicted face image data in a database.

[1158] Input: Generated predicted face image data

[1159] Output: Predicted face image data stored in a database

[1160] Specific operation: The server stores the generated predicted face image data in a database.

[1161] Step 5:

[1162] The terminal (monitoring device) transmits video data to the server in real time.

[1163] Input: Real-time video data captured by a surveillance camera

[1164] Output: Real-time video data sent to the server

[1165] Specific operation: The terminal (monitoring device) streams video footage taken at station ticket gates and public places to a server in real time.

[1166] Step 6:

[1167] The server extracts a facial image from the video data and compares it with predicted facial image data.

[1168] Input: Real-time video data

[1169] Output: Matching result (match rate)

[1170] Specific operation: The server applies a facial recognition algorithm to the received video data to extract facial images. The extracted facial images are compared with the stored predicted facial image data and the match rate is calculated. OpenCV and Dlib are used here.

[1171] Step 7:

[1172] The server generates an alert and notifies you if the match rate exceeds a certain threshold.

[1173] Input: Matching result (match rate)

[1174] Output: Alert information

[1175] Specific operation: If the calculated match rate exceeds a threshold, the server generates an alert. The alert includes the matched face image, the location information of the monitoring device, and the time when the match was detected. The generated alert is sent to the police terminal.

[1176] Step 8:

[1177] Police receive an alert, provide on-site information to response units, and initiate action.

[1178] Input: Alert information sent from the server

[1179] Output: On-site confirmation and protective action

[1180] Specific operations: The police terminal receives the alert and provides the relevant scene information to the response team. The police then go to the scene to identify and protect the person in question, and after confirmation, contact the family. Finally, they arrange for the missing person to be reunited with their family.

[1181] This concludes the detailed explanation of each processing step in the system program, which will enable efficient detection of missing persons and prompt response.

[1182] (Application example 1)

[1183] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1184] There is a need for an efficient and rapid method to improve the rate at which missing persons are found. However, current systems only use public surveillance cameras, and efficient searches using mobile information devices owned by individuals or at home are not performed. Furthermore, there is a lack of reliable predictive methods for responding to changes in facial images. As a result, searches for missing persons are delayed and the rate at which they are found decreases. Therefore, there is a need for a rapid and widespread method for finding missing persons.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1186] In this invention, the server includes means for acquiring image data of a missing person, means for operating a generative model for generating predicted facial image data from the image data, means for saving the generated predicted facial image data, means for acquiring video data in real time from a surveillance camera or a camera of a mobile information terminal, means for extracting a facial image from the video data, means for comparing the extracted facial image with the predicted facial image data and calculating a match rate, means for generating and notifying an alert when the match rate exceeds a certain threshold, and means for sharing location information based on the alert, thereby enabling a wide-area search for missing people in real time.

[1187] A "missing person" is someone whose current whereabouts are unknown and who requires a search.

[1188] "Image data" refers to digital data that includes past photographs of faces and other visual information about missing persons.

[1189] A "generative model" is an artificial intelligence model that predicts and generates current facial image data from input past facial image data.

[1190] A "personal digital assistant" is a portable device such as a smartphone or tablet.

[1191] A "surveillance camera" is a camera device installed in a public place that continuously records and monitors footage.

[1192] "Acquiring video data in real time" means that the video obtained from the camera is immediately sent to the system for processing.

[1193] "Extracting a face image" means cutting out and acquiring only the face portion of a person from video data.

[1194] "Matching" means comparing and verifying a match against different data sets.

[1195] The "match rate" is a ratio indicating the degree to which the extracted face image and the predicted face image data are identical.

[1196] An "alert" is a warning or notification that is generated when the match rate exceeds a certain threshold.

[1197] "Sharing location information" means sending and receiving geographical data of the location where a missing person was found between multiple devices.

[1198] The present invention is a system for efficiently locating missing persons. This system consists of three main components: a server, a terminal, and a user.

[1199] First, the user uploads past facial image data of the missing person to the system. This can be done using a device such as a smartphone or PC. The image data provided by the user is sent to the server. The server receives this data and performs preprocessing. This preprocessing involves resizing the image, removing noise, and adjusting the color tone. This process converts the data into a format suitable for the generative model.

[1200] Next, a generative model generates predicted facial image data from the preprocessed facial image data. This generative model outputs multiple predicted facial images, taking into account facial changes such as aging and cosmetic surgery. This predicted facial image data is stored in a database on the server.

[1201] The server receives video data in real time from surveillance cameras and mobile devices. For example, when a smartphone camera captures video in a public place, the video data is sent to the server. The server then applies a facial recognition algorithm to extract facial images from the video data. This algorithm can use models such as InceptionResnetV1 or MTCNN.

[1202] The extracted facial image is compared with predicted facial image data stored in the server's database. At this time, the match rate is calculated. If the match rate exceeds a certain threshold, the server generates an alert. This alert includes the matched facial image, detected location information, and time information. The alert is sent to the devices of the police and relevant parties.

[1203] Additionally, the alert notification includes a location sharing function, enabling a rapid response if a missing person is found. For example, the smartphone app can send geographical data of the detected location to a server, allowing police or family members to receive the data and take action.

[1204] Examples:

[1205] Below are some example prompts for the generative AI model:

[1206] "Generate a current image of a 50-year-old woman's face from an image taken 20 years ago."

[1207] "Please output an image of a 20-year-old man's face converted to look like a 30-year-old man."

[1208] This enables wide-area and real-time searches for missing people. The server calculates the facial image match rate and can quickly generate an alert and notify relevant parties if a match exceeds a certain threshold. This enables the early detection and protection of missing people and promotes reunification with their families.

[1209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1210] Step 1:

[1211] The user uploads the missing person's past facial image data to the system. The user uses a device such as a smartphone or PC to send the image data to the server. The input at this time is the missing person's past facial image data, and the output is the data transferred to the server.

[1212] Step 2:

[1213] The server preprocesses the facial image data received from the user. It resizes the image, removes noise, adjusts color tone, and converts the data format into a format that can be input to the generative model. The input of this process is the uploaded facial image data, and the output is the preprocessed facial image data.

[1214] Step 3:

[1215] The server inputs the preprocessed facial image data into a generative model, which generates multiple predicted facial image data by taking into account factors such as aging and cosmetic surgery. The input is the preprocessed facial image data, and the output is predicted facial image data.

[1216] Step 4:

[1217] The generated predicted facial image data is stored in a database on the server. This database is used in subsequent facial recognition processes. The input is the predicted facial image data, and the output is the data stored in the database.

[1218] Step 5:

[1219] A camera on a device (e.g., a smartphone) or a public surveillance camera captures video data in real time. The captured video data is sent to a server. The input is the real-time video data, and the output is the data sent to the server.

[1220] Step 6:

[1221] The server applies a facial recognition algorithm to extract facial images from the transmitted video data, such as InceptionResnetV1 or MTCNN. The input of this process is real-time video data, and the output is extracted facial image data.

[1222] Step 7:

[1223] The extracted facial image data is compared with predicted facial image data stored in the server's database, and a match rate is calculated. This calculation confirms whether the image matches the missing person. The input is the extracted facial image data and predicted facial image data, and the output is the calculated match rate.

[1224] Step 8:

[1225] If the match rate exceeds a certain threshold, the server generates an alert, which includes the matched face image, location information, and time information. The input is the situation where the match rate exceeds a certain threshold, and the output is the generated alert.

[1226] Step 9:

[1227] The generated alert is sent to the devices of the police and other relevant parties, and a response is initiated at the scene. The input is the generated alert, and the output is the notified information. The police and other relevant parties then proceed to the scene based on the location information to identify and protect the missing person.

[1228] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1229] The present invention is a system for improving the rate at which missing persons are found, and aims to efficiently find missing persons by generating predicted facial image data using a generative model based on facial image data of the missing person and comparing this with video data from surveillance cameras installed in public institutions.Furthermore, the system is characterized by improving the responsiveness of the system by combining it with an emotion engine that recognizes the user's emotions.

[1230] First, family members and other people involved (hereafter referred to as "users") upload facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. The emotion engine then analyzes the user's emotions in real time and optimizes the operation of the entire system based on that information.

[1231] The server stores the facial image data received from the user in a database and then preprocesses it. This involves resizing the image, removing noise, adjusting color, and converting it into a format suitable for input to the generative model. The emotion engine also obtains the user's emotional data during this process, which can be used as additional information to help search for missing people.

[1232] The facial image data after preprocessing is input into a generative model on the server. This generative model generates multiple predicted facial image data, taking into account changes such as aging and cosmetic surgery. The generated predicted facial image data is important for estimating what the missing person currently looks like.

[1233] Next, the generated predicted facial image data is stored on a server and linked to a surveillance camera system. Multiple surveillance cameras continuously capture video data in real time at stations and public places. The surveillance camera terminals then transmit this video data to the server in real time.

[1234] The server applies a facial recognition algorithm to extract facial images from the received video data. An emotion engine then analyzes the user's emotional data and sets optimal parameters for searching for the missing person. The extracted facial images are compared with predicted facial image data stored in a database to calculate the match rate. If the match rate exceeds a certain threshold, the server generates an alert.

[1235] The alert includes the matched facial image, the surveillance camera's location, and the time the match was detected. The alert is sent to the police device. The emotion engine takes the user's emotional state into account to customize the alert content and notification method. The police device receives the alert and provides on-site information to response teams. The police then arrive at the scene to identify and protect the person. After identification, the police contact the family and arrange for the missing person to be reunited with their family.

[1236] Specific examples

[1237] For example, consider a case where a person with dementia has gone missing. Family members upload past photos of the person to the system, along with basic information such as age and gender. The server preprocesses these images and inputs them into a generative model to generate multiple facial images for predicting their current appearance. The emotion engine also analyzes the family's emotional state of concern and adjusts the system's overall responsiveness and alertness. Next, facial images are extracted from real-time footage captured by surveillance cameras at station ticket gates and compared with the stored predicted facial images. If the match rate is high, the server sends an alert to the police, who then dispatch to the scene to protect the person. The family is then immediately contacted, enabling the reunion. The emotion engine's analysis of the user's emotions also allows for efficient care and support after the reunion.

[1238] The above is an embodiment of the present invention, and by combining this process with an emotion engine, it is expected that the early discovery and reunion of missing persons will proceed even more smoothly.

[1239] The processing flow will be explained below.

[1240] Step 1:

[1241] The user uploads facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. At the same time, the user inputs emotional data (such as worry, anxiety, and impatience). The emotional data can be in text format or voice input.

[1242] Step 2:

[1243] The server stores the facial image data and emotion data received from the user in a database, with the facial image data stored in high quality for subsequent processing.

[1244] Step 3:

[1245] The server preprocesses the facial image data, specifically resizing the image, removing noise, adjusting color, etc. The preprocessed facial image data is then converted into a suitable format for input to the generative model.

[1246] Step 4:

[1247] The server inputs the preprocessed facial image data into a generative model, which then generates multiple predicted facial image data, taking into account factors such as aging and cosmetic surgery.

[1248] Step 5:

[1249] The server stores the generated predicted facial image data in a database. At this stage, the system has multiple facial image variations that can be used to more easily identify missing people.

[1250] Step 6:

[1251] The server uses an emotion engine to analyze the user's emotion data. Based on the analysis results, the system adjusts its responsiveness and alertness. For example, if the user shows a high level of impatience, the system will use a high-precision model to generate predicted facial images more quickly.

[1252] Step 7:

[1253] The terminals (surveillance camera systems) acquire video data in real time. These surveillance cameras are installed in stations and public places, and continuously transmit the video data to a server.

[1254] Step 8:

[1255] The server receives real-time video data transmitted from the surveillance cameras, which is analyzed using a facial recognition algorithm to extract facial images.

[1256] Step 9:

[1257] The server compares the extracted facial image with predicted facial image data stored in a database, calculates the match rate, and generates an alert if the match rate exceeds a certain threshold.

[1258] Step 10:

[1259] The server then sends the generated alert to the police terminal, which includes the matching face image, the location information of the surveillance camera, and the time when the match was detected.

[1260] Step 11:

[1261] Police devices receive the alert and provide on-site information to responding teams, who then arrive on-site to identify and protect the individual.

[1262] Step 12:

[1263] Once the police confirm that the person is missing, they will contact the family. Based on the user's emotional state, the emotion engine will customize the notification content and contact the user in the most appropriate way to reassure them.

[1264] Step 13:

[1265] Police will arrange for missing people to be reunited with their families and return home safely, and the emotion engine will analyze the user's emotions and coordinate support and care after the reunion.

[1266] These steps will ensure that missing persons are quickly identified and reunited with their families.

[1267] Example 2

[1268] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1269] The present invention aims to improve the rate at which missing people are found, and to solve the problems of cases where finding a missing person is difficult using conventional search methods, or where searches take a long time. Furthermore, since a flexible response that takes into account the user's emotions is required, a system that can optimize search activities and provide care after a person is found is also needed.

[1270] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring image data of a missing person, means for operating a generative model for generating predicted face image data from the image data, means for saving the generated predicted face image data, means for acquiring video data from a surveillance camera in real time, means for extracting a face image from the video data, means for matching the extracted face image with the predicted face image data and calculating a match rate, means for generating and notifying an alert when the match rate exceeds a certain threshold, means for analyzing emotional data acquired from a user, and means for optimizing the operation of the entire system based on the emotional data. This enables the missing person to be found quickly and efficiently, and also allows for flexible responses that take user emotions into consideration.

[1271] "Image data of a missing person" is digital data that shows the facial image and physical characteristics of a missing person.

[1272] A "generative model" refers to an algorithm or program that uses artificial intelligence to generate predicted facial image data from input image data.

[1273] "Predicted facial image data" is image data that predicts the future appearance of a missing person, taking into account changes such as aging and cosmetic surgery.

[1274] A "surveillance camera" is an electronic device installed in a public place to capture and record video in real time.

[1275] "Means of acquiring video data in real time" refers to the technology and equipment used to collect ongoing video footage from surveillance cameras and send it to a server.

[1276] "Means for extracting facial images" refers to an algorithm or technology for identifying a person's face from video data and extracting it as a facial image.

[1277] The "means for calculating the matching rate" refers to an algorithm or program that compares the extracted facial image with predicted facial image data and calculates the degree of matching as a numerical value.

[1278] "Means for generating and notifying alerts" refers to a system for creating a warning message and notifying relevant organizations and users when the match rate exceeds a certain threshold.

[1279] "Emotion data" is data that expresses the user's emotional state numerically or by category.

[1280] "Means for analyzing emotional data" refers to algorithms or technologies that analyze the emotional state of a user from input, facial expressions, and voice and obtain the data.

[1281] "Means for optimizing the operation of the entire system" refers to techniques and methods for adjusting the operation and response of each part of the system based on the acquired emotional data, allowing it to operate in the most effective manner.

[1282] The present invention is a system for improving the rate at which missing persons are found, and aims to efficiently find missing persons by generating predicted facial image data based on facial image data of the missing person and comparing this with video data from surveillance cameras installed in public institutions.Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system's responsiveness is improved.

[1283] First, family members or people involved (hereinafter referred to as "users") upload facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. The uploaded data is then sent to the server, which stores it in a database and begins pre-processing the image data. This pre-processing includes processes such as image resizing, noise removal, and color adjustment.

[1284] The server inputs the preprocessed facial image data into a generative AI model (e.g., StyleGAN) to generate multiple predicted facial image data that take into account changes over time, such as cosmetic surgery, etc. The generated predicted facial image data is then stored in a database by the server.

[1285] Meanwhile, surveillance cameras are installed in stations and public places and capture video data in real time. The surveillance camera terminals send the captured video data to a server. The server uses facial recognition algorithms such as OpenCV and Dlib to extract facial images from the video data and compare them with predicted facial image data stored in a database. The server calculates the match rate and generates an alert if this match rate exceeds a certain threshold.

[1286] The generated alert includes the matched facial image, the location of the surveillance camera, and the time the match was detected. The alert is sent to the device of the responding agency, such as the police, who will then go to the site to identify and protect the person and arrange for them to be reunited with their family.

[1287] The emotion engine also analyzes users' emotional data and optimizes the operation of the entire system based on that data. Specifically, when a user uploads data, the emotion engine analyzes the emotions in real time from the user's face and sentences. The server obtains the analyzed emotional data and adjusts the search priority and alert content.

[1288] Specific examples

[1289] For example, consider a case where a person with dementia has gone missing. Family members upload past photos of the person to the system, along with basic information such as age and gender. The server preprocesses these images and inputs them into a generative model to generate multiple facial images for predicting their current appearance. The emotion engine also analyzes the family's emotional state of concern and adjusts the system's overall responsiveness and alertness. Next, facial images are extracted from real-time footage captured by surveillance cameras at station ticket gates and compared with the stored predicted facial images. If the match rate is high, the server sends an alert to the police, who then dispatch to the scene to protect the person. The family is then immediately contacted, enabling the reunion. The emotion engine's analysis of the user's emotions also allows for efficient care and support after the reunion.

[1290] Prompt Sentence Examples

[1291] "Predict the current appearance of a person who has gone missing due to dementia based on past facial photographs. Age: 70, gender: male, last seen in: XX City, XX Prefecture."

[1292] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1293] Step 1:

[1294] The user uploads facial image data and basic information (such as name, age, gender, and last seen location) of the missing person to the system. This data is then sent to the server. At this time, the facial image data and basic information are prepared as input, and the raw facial image data and basic information stored on the server are obtained as output. Specifically, the user selects an image file from a dedicated web form, enters basic information in the text box, and presses the send button.

[1295] Step 2:

[1296] The server stores the received facial image data in a database and begins preprocessing. Preprocessing includes image resizing, noise removal, and color adjustment. The input here is the raw facial image data stored on the server, and the output is preprocessed facial image data converted into a format suitable for the generative AI model. Specifically, the server uses an image processing library (e.g., OpenCV) to resize the facial image (e.g., to 256x256 pixels), apply a noise removal filter, and equalize the color tone.

[1297] Step 3:

[1298] The server inputs the preprocessed facial image data into a generative AI model (e.g., StyleGAN) to generate multiple predicted facial image data that take into account changes such as aging and cosmetic surgery. The input here is preprocessed facial image data, and the output is predicted facial image data. Specifically, the server sends an API request to the generative model and stores the generated results in a database.

[1299] Step 4:

[1300] The server stores the generated predicted face image data in a database. The input of this step is the predicted face image data output from the generative AI model, and the output is the predicted face image data stored in the database. Specifically, the server uses a database management system (e.g., MySQL) to store the predicted face image data in an appropriate format.

[1301] Step 5:

[1302] Surveillance camera terminals continuously capture video data in real time at stations and public places. The captured video data is sent to a server. The input of this step is the real-time video data, and the output is the video data sent to the server. Specifically, the surveillance camera terminals have the function of sending video streams to the server over the network (e.g., using the RTSP protocol).

[1303] Step 6:

[1304] The server applies a facial recognition algorithm to extract facial images from the received video data. The input to this step is the video data sent to the server, and the output is the facial images extracted from the video data. Specifically, the server uses libraries such as OpenCV and Dlib to detect facial areas from the video frames and extract facial images.

[1305] Step 7:

[1306] The server matches the extracted facial image with predicted facial image data stored in a database. The inputs to this step are the extracted facial image and predicted facial image data, and the output is a match rate. Specifically, the server uses a facial recognition algorithm to compare the features of each facial image and calculate the match rate (e.g., cosine similarity).

[1307] Step 8:

[1308] The server generates an alert and notifies if the match rate exceeds a certain threshold. The input of this step is the calculated match rate, and the output is the generated alert. Specifically, the server generates an alert message when the match rate exceeds a set threshold (e.g., 90%) and sends an alert to the police or related organizations through the notification system.

[1309] Step 9:

[1310] The emotion engine analyzes the user's emotional data and optimizes the overall system behavior based on that data. The input to this step is the user's emotional data, and the output is optimized system behavior. Specifically, the emotion engine analyzes the facial expressions and sentences used when the user uploads data to the system, and sends the estimated emotional state as numerical data to the server. The server then adjusts responsiveness and alert content based on this data.

[1311] (Application example 2)

[1312] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1313] Conventional missing person search systems have had problems with low accuracy in handling facial image data and matching it with actual facial images, resulting in low rates of finding missing persons. In particular, as time passes, the appearance of a missing person changes, further reducing the accuracy of matching. Furthermore, because searches are performed without taking the user's emotional state into consideration, there are also problems with low responsiveness and efficiency of the entire system. The purpose of the present invention is to solve these problems and improve the rate of finding missing persons and the responsiveness of the system.

[1314] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1315] In this invention, the server includes means for acquiring image data of a missing person, means for operating a generative model for generating predicted facial image data from the image data, means for saving the generated predicted facial image data, means for acquiring video data from a surveillance camera in real time, means for extracting a facial image from the video data, means for comparing the extracted facial image with the predicted facial image data and calculating a match rate, means for generating and notifying an alert when the match rate exceeds a certain threshold, means for analyzing user emotional data and optimizing system operation, and means for inputting and managing information about the missing person using an application installed on a smart device. This enables the generation of predicted facial images that take into account changes in the missing person's appearance and optimal system operation based on the user's emotional state, improving the rate at which missing people are found and the responsiveness of the entire system.

[1316] "Image data of a missing person" is data that includes a facial image of a missing person and related personal information.

[1317] A "generative model" is a machine learning algorithm for generating predicted facial image data from input facial image data of a missing person.

[1318] "Predicted facial image data" is facial image data obtained by a generative model that predicts the current appearance of a missing person.

[1319] A "surveillance camera" is a device that is installed in public facilities or public places and captures video data in real time.

[1320] "Video data" refers to real-time video information acquired from a surveillance camera.

[1321] "Means for extracting facial images" refers to techniques or algorithms for extracting only the facial portion from video data.

[1322] The "means for matching and calculating the degree of match" is a technique for comparing the extracted face image with predicted face image data and quantifying the degree of similarity between the two.

[1323] "Means for generating and notifying alerts" refers to technology that generates an alert when the matching rate exceeds a certain threshold and notifies relevant institutions and parties of that information.

[1324] "Means for analyzing user emotional data and optimizing system operation" refers to technology that analyzes the user's emotional state and adjusts the system's responsiveness and alertness.

[1325] An "application installed on a smart device" is application software installed on a mobile device such as a smartphone or smart glasses for inputting and managing information about missing persons.

[1326] "Preprocessing means" refers to a technique for performing processes such as noise removal and resizing before inputting facial image data into a generative model.

[1327] This invention is a system for improving the rate at which missing people are found. The system is mainly composed of a server, a surveillance camera, an application, and an emotion analysis engine.

[1328] 1. System Configuration

[1329] server:

[1330] The server is the central data processing unit and performs the following main processes:

[1331] Data Acquisition: Acquires image data and basic information of the missing person provided by the user.

[1332] Generative model: Operate a generative AI model to generate predicted facial image data from the provided image data. Here, TensorFlow or PyTorch are used as the generative AI model.

[1333] Data storage: The generated predicted face image data is stored in a database.

[1334] Video data acquisition: Acquire video data from surveillance cameras in real time.

[1335] Facial image extraction: Facial images are extracted from video data using OpenCV.

[1336] Matching and match rate calculation: Using a face recognition library such as Dlib, the extracted face image is matched with the predicted face image data and the match rate is calculated.

[1337] Alert generation: Provides a function to generate an alert and notify when the match rate exceeds a certain threshold. Firebase Cloud Messaging is used for alert notification.

[1338] Emotion analysis: Analyzes user emotional data using emotion analysis engines such as Affectiva. Optimize system operation based on the results of this analysis.

[1339] Surveillance cameras:

[1340] It is installed in public places and is responsible for transmitting video data to a server in real time. Video data is managed via a network video recorder (NVR).

[1341] Smart device applications:

[1342] An application that allows users to input and manage information about missing persons through smart devices such as smartphones or smart glasses, and provides specific prompts for:

[1343] "Please upload a photo of the subject's face. Enter basic information (name, age, gender, last seen location)."

[1344] "We will optimize search parameters based on the user's emotional state. We are analyzing emotional data in real time."

[1345] 2. Working Example

[1346] Examples:

[1347] For example, if someone with dementia goes missing, a family member can use a smartphone application to upload a past photo of the person to the system, along with basic information such as name, age, gender, and last known location.

[1348] Preprocessing of facial image data:

[1349] The server performs preprocessing such as resizing the images, removing noise, and adjusting color tones, and converts them into a format that can be input to the generative model. This processing is done using OpenCV.

[1350] Generate predicted face images:

[1351] The preprocessed facial image data is input into the server's generative model (TensorFlow or PyTorch), and multiple predicted facial image data are generated, taking into account changes such as aging and cosmetic surgery.

[1352] Real-time video data collection and analysis:

[1353] Surveillance cameras capture video data in real time at stations and public places and send it to a server, which then extracts facial images from the video data and compares them with predicted facial image data stored in a database.

[1354] Alert Notifications:

[1355] If the match rate of the matched facial images exceeds a certain threshold, the server generates an alert and notifies police and other relevant parties in real time using Firebase Cloud Messaging, allowing relevant authorities to respond to the scene and protect the individual.

[1356] The above process enables the early detection and protection of missing persons, and also makes it possible to optimize the system's responsiveness by reflecting the results of user emotion analysis.

[1357] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1358] Step 1:

[1359] User Input

[1360] Using a smart device application, users input a facial image and basic information (name, age, gender, last seen location) of the missing person, which is then sent to a server via the application.

[1361] Input: Facial image data and basic information of the missing person

[1362] Output: Facial image data and basic information sent to the server

[1363] Step 2:

[1364] Image data preprocessing

[1365] The server processes the received facial image data. This preprocessing involves resizing the image, removing noise, adjusting color tone, etc., and converting it into a format that can be input to the generative model. OpenCV is used here.

[1366] Input: Acquired facial image data

[1367] Output: Preprocessed face image data

[1368] Specific operations: resize image (e.g., change to 128x128 pixels), remove noise (apply Gaussian filter), adjust color tone (convert to grayscale)

[1369] Step 3:

[1370] Generating predicted face images

[1371] The preprocessed facial image data is input into a generative model on the server to generate a predicted facial image that reflects the missing person's current appearance. This generation is done using TensorFlow and PyTorch. The generated predicted facial image data is stored in a database.

[1372] Input: Preprocessed face image data

[1373] Output: Predicted face image data

[1374] Specific operation: An image is input into the generative model, and multiple predicted facial images are generated, taking into account changes such as aging and cosmetic surgery.

[1375] Step 4:

[1376] Collecting video data from surveillance cameras

[1377] The server receives video data in real time from surveillance cameras installed in stations and public places, and manages the video data via a network video recorder (NVR).

[1378] Input: Surveillance camera video data

[1379] Output: Real-time video data sent to the server

[1380] Specific operation: Video data from a surveillance camera is sent to a server via a network.

[1381] Step 5:

[1382] Facial image extraction from video data

[1383] The server uses OpenCV to extract facial images from the video data received in real time, resulting in image data with only the facial area extracted.

[1384] Input: Real-time video data

[1385] Output: Extracted face image

[1386] Specific operation: Detects faces from video frames and extracts only the facial parts

[1387] Step 6:

[1388] Facial image matching and matching rate calculation

[1389] The server uses a facial recognition library such as Dlib to match the extracted facial image with stored predicted facial image data and calculate the match rate.

[1390] Input: Extracted face image, predicted face image data

[1391] Output: Match rate

[1392] Specific operation: Calculate the similarity between the two using a face recognition algorithm

[1393] Step 7:

[1394] Alerting and Notifications

[1395] If the match rate exceeds a certain threshold, the server generates an alert using Firebase Cloud Messaging to notify relevant authorities and relatives, including information such as the matched face image, the detection location, and the detection time.

[1396] Input: Match rate, face image, location information, time information

[1397] Output: Alert notification

[1398] What it does: Sends alerts to mobile devices and computers using Firebase Cloud Messaging

[1399] Step 8:

[1400] Emotional data analysis and system optimization

[1401] The server uses an emotion analysis engine such as Affectiva to analyze user emotion data and optimize the responsiveness and alertness of the entire system.

[1402] Input: User emotion data

[1403] Output: Optimized system response

[1404] Specific operation: Drives the emotion analysis engine and adjusts the system's operating parameters based on the analysis results.

[1405] In this way, the entire system goes through a series of processing steps to help find missing people early and realize optimal search activities that take into account the user's emotional state.

[1406] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1407] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1408] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1409] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1410] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1411] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1412] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1413] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1414] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1415] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1416] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1417] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1418] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1419] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1420] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1421] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1422] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1423] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1424] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1425] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1426] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1427] The following is further disclosed regarding the above embodiment.

[1428] (Claim 1)

[1429] a means for obtaining image data of the missing person;

[1430] means for operating a generative model for generating predicted facial image data from the image data;

[1431] A means for storing the generated predicted face image data;

[1432] A means for acquiring video data from a surveillance camera in real time;

[1433] means for extracting a facial image from the video data;

[1434] a means for comparing the extracted facial image with the predicted facial image data and calculating a match rate;

[1435] A means for generating an alert and notifying when the match rate exceeds a certain threshold;

[1436] A system including:

[1437] (Claim 2)

[1438] 10. The system of claim 1, further comprising means for pre-processing image data of the missing person provided by a family member.

[1439] (Claim 3)

[1440] The system of claim 1, wherein the generative model comprises means for generating predicted facial image data that takes into account changes such as aging and cosmetic surgery.

[1441] (Claim 4)

[1442] 10. The system of claim 1, wherein the alert is configured to be transmitted to a police terminal.

[1443] (Claim 5)

[1444] 10. The system of claim 1, wherein the alert is configured to include a matched face image, location information of the surveillance camera, and time information.

[1445] "Example 1"

[1446] (Claim 1)

[1447] a means for obtaining image data of the missing person;

[1448] means for pre-processing and converting said image data into a suitable format;

[1449] means for operating a generative model to generate predicted facial image data from the preprocessed image data;

[1450] A means for storing the generated predicted face image data;

[1451] means for acquiring video data in real time from a plurality of monitoring devices;

[1452] means for extracting a facial image from the video data;

[1453] a means for comparing the extracted facial image with the predicted facial image data and calculating a match rate;

[1454] A means for generating an alert and notifying when the match rate exceeds a certain threshold;

[1455] A system including:

[1456] (Claim 2)

[1457] 10. The system of claim 1, further comprising means for pre-processing image data of the missing person provided by family members or interested parties.

[1458] (Claim 3)

[1459] 2. The system according to claim 1, further comprising means for generating predicted facial image data in which the generative model takes into account changes due to aging, surgery, etc.

[1460] "Application Example 1"

[1461] (Claim 1)

[1462] a means for obtaining image data of the missing person;

[1463] means for operating a generative model for generating predicted facial image data from the image data;

[1464] A means for storing the generated predicted face image data;

[1465] A means for acquiring video data in real time from a surveillance camera or a camera of a mobile information terminal;

[1466] means for extracting a facial image from the video data;

[1467] a means for comparing the extracted facial image with the predicted facial image data and calculating a match rate;

[1468] A means for generating an alert and notifying when the match rate exceeds a certain threshold;

[1469] means for sharing location information based on the alert;

[1470] A system including:

[1471] (Claim 2)

[1472] 10. The system of claim 1, further comprising means for pre-processing image data of the missing person provided by family or related parties.

[1473] (Claim 3)

[1474] The system of claim 1, wherein the generative model comprises means for generating predicted facial image data that takes into account changes such as aging and cosmetic surgery.

[1475] "Example 2: Combining Emotion Engines"

[1476] (Claim 1)

[1477] a means for obtaining image data of the missing person;

[1478] means for operating a generative model for generating predicted facial image data from the image data;

[1479] A means for storing the generated predicted face image data;

[1480] A means for acquiring video data from a surveillance camera in real time;

[1481] means for extracting a facial image from the video data;

[1482] a means for comparing the extracted facial image with the predicted facial image data and calculating a match rate;

[1483] A means for generating an alert and notifying when the match rate exceeds a certain threshold;

[1484] means for analyzing emotion data acquired from a user;

[1485] A means for optimizing the overall system operation based on the emotion data;

[1486] A system including:

[1487] (Claim 2)

[1488] 10. The system of claim 1, further comprising means for pre-processing image data of the missing person provided by a family member.

[1489] (Claim 3)

[1490] The system of claim 1, wherein the generative model comprises means for generating predicted facial image data that takes into account changes such as aging and cosmetic surgery.

[1491] (Claim 4)

[1492] 10. The system of claim 1, further comprising means for acquiring video data in real time using a plurality of monitoring devices.

[1493] (Claim 5)

[1494] 10. The system of claim 1, further comprising means for a response agency to arrive at the scene after sending an alert to the user and arrange for the missing person to be rescued and reunited with family.

[1495] "Application example 2 when combining emotion engines"

[1496] (Claim 1)

[1497] a means for obtaining image data of the missing person;

[1498] means for operating a generative model for generating predicted facial image data from the image data;

[1499] A means for storing the generated predicted face image data;

[1500] A means for acquiring video data from a surveillance camera in real time;

[1501] means for extracting a facial image from the video data;

[1502] a means for comparing the extracted facial image with the predicted facial image data and calculating a match rate;

[1503] A means for generating an alert and notifying when the match rate exceeds a certain threshold;

[1504] A means for analyzing user emotion data and optimizing system operation;

[1505] a means for inputting and managing information about missing persons in an application installed on the smart device;

[1506] A system including:

[1507] (Claim 2)

[1508] 10. The system of claim 1, further comprising means for pre-processing image data of the missing person provided by a family member.

[1509] (Claim 3)

[1510] The system of claim 1, wherein the generative model comprises means for generating predicted facial image data that takes into account changes such as aging and cosmetic surgery. [Explanation of symbols]

[1511] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for obtaining image data of the missing person; means for operating a generative model for generating predicted facial image data from the image data; A means for storing the generated predicted face image data; A means for acquiring video data from a surveillance camera in real time; means for extracting a facial image from the video data; a means for comparing the extracted facial image with the predicted facial image data and calculating a match rate; A means for generating an alert and notifying when the match rate exceeds a certain threshold; A system including:

2. The system of claim 1 , further comprising means for pre-processing image data of the missing person provided by a family member.

3. The system of claim 1 , further comprising means for generating predicted facial image data in which the generative model takes into account changes such as aging and cosmetic surgery.

4. The system of claim 1 , wherein the alert is configured to be transmitted to a police terminal.

5. The system of claim 1 , wherein the alert is configured to include a matched facial image, location information of the surveillance camera, and time information.

Images