System

Abstract

A system is provided.SOLUTION: A system, comprising: a terminal means for uploading a picture related to a missing pet; a server means for receiving the uploaded picture and extracting features; and a server means for matching the features with a shelter database.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, pets are increasingly being treated as family members, but many pets go missing, placing a great mental burden on owners. Current pet search methods make it difficult to quickly locate pets, and there is often a time lag. In particular, matching with information from shelters and online sites is often done manually, making efficient search difficult. Furthermore, real-time tracking using GPS and behavior prediction technologies are not being fully utilized. There is a need for a system that can solve these issues and enable the rapid discovery and protection of pets. [Means for solving the problem]

[0005] The present invention provides a system for quickly locating and protecting missing pets. Specifically, the system includes a terminal for uploading photos of the missing pet, a server for receiving the uploaded photos and extracting their characteristics, and a server for matching the photos with a pet shelter database based on the characteristics. The system also includes a terminal for uploading the sounds of the missing pet, a server for receiving the uploaded sounds and performing audio analysis, and a server for matching the results of the audio analysis with online information. The system also includes a GPS device for tracking the location of the missing pet, a server for receiving location information and determining the location in real time, and a server for predicting behavior based on the location information. This allows pets to be found and protected quickly and efficiently, reducing the anxiety of their owners.

[0006] "Terminal means" refers to a device that allows owners to upload information about their missing pets (photos, sounds, GPS data, etc.).

[0007] The "server means" is a computer system that receives, stores, analyzes, and processes information sent from the terminal means.

[0008] "Extracting features" means extracting the information (shape, color, sound waveform, etc.) necessary to identify a pet from photographs and audio data.

[0009] The "shelter database" is a database that collects information on facilities where missing pets are being cared for.

[0010] "Matching" refers to matching the same or similar pets with the shelter's database or online information based on the extracted features.

[0011] "Audio analysis" involves analyzing the uploaded pet sounds and extracting their waveform and frequency characteristics.

[0012] "Online information" refers to information about missing pets that is shared via the Internet on bulletin boards, social media, etc.

[0013] The "GPS device means" is a device for acquiring the location information of the pet and transmitting it to a terminal or a server.

[0014] "Knowing your location in real time" means receiving location information sent from a GPS device and instantly checking your current location.

[0015] "Predicting behavior based on location information" means analyzing the received location information and estimating the pet's movement patterns and where it may go next. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] Overall system description

[0038] This invention is an AI system that supports the rapid discovery and protection of missing pets. This system uses a server to receive and analyze photos, sounds, and GPS data of pets uploaded by owners using a terminal, and then identifies the pet's location and predicts its behavior by performing analysis and matching.

[0039] Data collection and storage

[0040] Users use a dedicated application to upload photos, sounds, and GPS data of their pets, which are then sent from the device to a server, which then stores the data in an internal database.

[0041] Data analysis and feature extraction

[0042] The server inputs the saved pet photos into the AI ​​model to extract specific features (shape, color, etc.). It also performs audio analysis on the uploaded pet's cries to extract their waveform and frequency characteristics. GPS data is analyzed in real time to capture location information and generate a map.

[0043] Data Matching

[0044] The server then matches the extracted features and the results of the voice analysis with information from shelter databases and online sources. The shelter database contains information registered when a missing pet is rescued. This allows the server to quickly notify owners if the same or similar pet is found.

[0045] Behavioral prediction

[0046] Furthermore, the server predicts pet behavior based on GPS data, taking into account the pet's breed, age, personality, surrounding terrain, etc., to estimate where the pet may head next. This behavior prediction helps to efficiently narrow the search area.

[0047] Notification and Discovery Assistance

[0048] Matching results and behavior prediction results are sent to the device in real time. Users can receive this notification and begin searching for their pet based on the provided information. If necessary, the system can also connect with a specialized pet rescue team to provide optimal support.

[0049] Specific examples

[0050] For example, if a user were to search for a missing dog, the process would be as follows: First, the user would use a dedicated application to upload the dog's photo, bark, and GPS data. The server receives this data, extracts the dog's characteristics from the photo, performs audio analysis of the bark, and analyzes the GPS data in real time. The server then matches the data with information from shelters and online to see if a similar dog is being cared for. At the same time, it predicts the dog's behavior based on the GPS data and determines the expected search area. The matching and behavior prediction results are notified to the device in real time, allowing the user to check them and search for the dog efficiently.

[0051] In this way, the present invention is a system that combines AI technology and data matching technology to enable the quick and effective discovery and protection of lost pets.

[0052] The processing flow will be explained below.

[0053] Step 1:

[0054] Users launch a dedicated application, capture photos of their missing pet, record its cries, and collect GPS data, which are then uploaded to a server via the application.

[0055] Step 2:

[0056] The server receives the uploaded photos, bird calls, and GPS data and stores them in a designated database, where they are prepared for analysis.

[0057] Step 3:

[0058] The server inputs the stored pet photos into an AI analysis module and uses image recognition technology to extract the pet's characteristics, such as fur color, body shape, and pattern.

[0059] Step 4:

[0060] The server inputs the saved bark data into a voice analysis module and extracts the characteristics of the pet's voice by analyzing the frequency characteristics and waveform patterns.

[0061] Step 5:

[0062] The server analyzes the GPS data in real time and plots the pet's current location on a map. The location information is used by the behavior prediction module.

[0063] Step 6:

[0064] The server compares the extracted photo characteristics and audio analysis results with information from shelter databases and online sources, and retrieves information about pets with similar characteristics.

[0065] Step 7:

[0066] The server predicts the pet's next move based on GPS data and its behavioral patterns, taking into account the pet's type, age, personality, and surrounding terrain information, for example.

[0067] Step 8:

[0068] The server generates a map of the search area based on the matching results and behavior prediction results and notifies the terminal in real time.

[0069] Step 9:

[0070] The device displays the notified information to the user, allowing the user to check the provided search area and pet characteristics. The user can then quickly start searching based on this information.

[0071] Step 10:

[0072] Users follow the instructions in the app to search for their pet within the provided search area, and if necessary, the app will connect them with a specialized pet rescue team to provide optimal support to help them find their pet.

[0073] Example 1

[0074] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0075] To quickly find and protect lost pets, it is necessary to accurately analyze various data such as photos, sounds, and GPS data, and efficiently search for matching information. It is also necessary to provide clues for the search by predicting the pet's current location and behavior.

[0076] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0077] In this invention, the server includes an information processing terminal means for uploading data on missing pets, an information processing device means for receiving the uploaded data and extracting features, an information processing device means for comparing the features and analysis results with records of the shelter, an information processing device means for predicting behavior based on the data, and an information processing terminal means for notifying the analysis results and prediction results, thereby enabling the rapid identification and behavior prediction of missing pets and further real-time notification.

[0078] The "information processing terminal means" is a device for collecting data about missing pets and transmitting it to an information processing device.

[0079] The "information processing device means" is a device for analyzing received data, extracting and collating necessary information, and predicting behavior.

[0080] The "location information acquisition device means" refers to a device for acquiring coordinate data of the pet's current location, such as a GPS device.

[0081] "Feature extraction" is the process of extracting specific attributes, such as shape or waveform, from uploaded photos or audio data.

[0082] "Audio analysis" is the process of analyzing uploaded bird call data and extracting its waveform, frequency characteristics, etc.

[0083] "Data matching" is the process of checking whether the extracted characteristics match the shelter's records or online information.

[0084] "Behavior prediction" is the process of estimating where your pet will go next, taking into account the pet's breed, age, personality, terrain, GPS data, and more.

[0085] "Real-time notification" is a notification function that instantly notifies users of analysis results and behavior prediction results.

[0086] System Overview

[0087] The present invention is an AI system designed to support the rapid discovery and protection of missing pets. This system is composed of an information processing terminal means, an information processing device means, and a location information acquisition device means. The information processing terminal means is a device that collects and transmits photos, audio, GPS data, etc. of pets, and the information processing device means analyzes, collates, and predicts the received data and transmits the results. The location information acquisition device means is a device that acquires pet location information in real time.

[0088] Hardware and software used

[0089] The server uses a computer with high-performance data processing capabilities. Deep learning frameworks such as TensorFlow and PyTorch are used to analyze the photo data. Audio analysis tools such as Librosa and WaveSurfer are used to analyze the audio data. Geographic information systems such as PostGIS are used to process GPS data. An SQL-based database system is used for database management.

[0090] Data collection and transmission

[0091] Users use a dedicated application to collect and upload data about their missing pets. This application runs on mobile platforms such as iOS and Android. For example, by entering data such as a photo of the missing dog, the dog's barking sound, and the current GPS location, users can easily send the data to the server by pressing the send button.

[0092] Data analysis and feature extraction

[0093] The server receives the uploaded data and performs the following analysis on each data. Photo data is input into a TensorFlow model to extract features such as the pet's shape and color. Audio data is analyzed using Librosa to extract waveforms and frequency characteristics. GPS data is processed using PostGIS and plotted on a map.

[0094] Data collation and behavior prediction

[0095] The analyzed feature data is compared with the shelter's database and data on the internet, for example by running a query such as "SELECT FROM shelter_pets WHERE shape = 'dog' AND color = 'brown'." The server then uses machine learning models such as random forests and neural networks to predict the pet's behavior and estimate its next likely destination.

[0096] Notification and Discovery Assistance

[0097] The server notifies the device of the matching results and behavior prediction results in real time. This notification is provided to the user in the form of a push notification or an in-app notification. For example, specific information such as "A similar pet has been found at shelter A" or "The next likely destination is park B" is provided.

[0098] Specific examples

[0099] For example, if a user is searching for a missing dog, they use a dedicated application to upload the following data: a photo of the dog (e.g., Fido_missing.jpg), the dog's barks (e.g., Fido_bark.wav), and current GPS data (e.g., GPS_location.csv), which are then sent directly to the server. The server receives this data and analyzes and extracts features using TensorFlow, Librosa, PostGIS, and other software. The results are compared with the shelter's database and information on the Internet to predict behavior. Finally, the results of the comparison and predicted behavior are sent to the user's device in real time, allowing for efficient searching for the pet.

[0100] The system combines AI and data analysis technologies to enable the fast and effective discovery and protection of lost pets.

[0101] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0102] Step 1: Upload your data

[0103] A user uses a dedicated application to collect photos, sounds, and GPS data related to a missing pet, and then sends this data to a server from an information processing device. For example, a user presses the "Add a new pet" button in the application, takes a photo of the pet, and uploads it. They also record the sound of the pet's sounds and upload that file. Next, the device's GPS function is used to obtain the pet's current location, which is also sent. The input data consists of a photo file (e.g., Fido_missing.jpg), an audio file (e.g., Fido_sound.wav), and GPS text data (e.g., GPS_location.csv). This data is sent to the server.

[0104] Step 2: Receiving and storing data

[0105] The server receives the data sent from the device and checks the format and consistency of each piece of data. After checking, it stores the received photo data in the "photos" table, the bird sound data in the "audio" table, and the GPS data in the "gps_logs" table. Specifically, the server stores each piece of data in temporary storage, verifies whether it is in the correct data format, and then inserts it into the corresponding database table. At this time, metadata (e.g., data timestamp, user ID) is also saved. If the data is successfully saved in the database, the server generates a success message and proceeds to the next step.

[0106] Step 3: Data analysis and feature extraction

[0107] The server analyzes the received and stored data and extracts its features. The photo data is input into a TensorFlow model to extract features such as the pet's shape and color. The input data is the stored photo file, and the output data is the extracted feature vector. Similarly, Librosa is used to analyze the sound waveform and frequency characteristics of the pet's bark data and extract a feature vector. The GPS data is plotted on a map based on the coordinate information, and the location information is confirmed. Specifically, the server feeds the photo data into a deep learning model and extracts features using a convolutional neural network (CNN). The analysis results are saved as a feature vector. Similarly, audio features such as power spectrograms and Mel-Frequency Cepstrum (MFCC) are extracted from the audio data and saved as a feature vector. PostGIS is used to convert the GPS data into geographic coordinates and plot them on a map.

[0108] Step 4: Collating the data

[0109] The server uses the extracted feature vectors to match the search results with the shelter's database and information on the internet. This process involves generating an SQL-based query to perform database matching, for example, "SELECT FROM shelter_pets WHERE shape = 'dog' AND color = 'brown'". The input data is the extracted feature vectors, and the output data is a list of matching database records. After the matching process, the server passes the details of any matching records to the next step.

[0110] Step 5: Predict behavior

[0111] The server comprehensively evaluates information such as the pet's species, age, personality, and GPS data to predict its behavior. It then uses machine learning algorithms such as random forests and neural networks to estimate the pet's next likely destination. The input data is the pet's characteristics and current GPS data, and the output data is the predicted range of movement and the next destination. Specifically, the server inputs known behavioral pattern data and current location information into a machine learning model to obtain a prediction result. For example, it generates a prediction result in the form of "The most likely next destination is a nearby park."

[0112] Step 6: Notification

[0113] The server notifies the user of the matching results and behavior prediction results in real time via push notifications or in-app notifications. The input data are the matching results and behavior prediction results obtained in the previous step, and the output data is the notification message. For example, messages such as "A similar pet is at shelter A" or "The next likely destination is park B" are sent to the user's device.

[0114] Step 7: Search support

[0115] The user receives the notification and quickly begins searching for the pet based on the provided information. For example, by heading to Park B based on the notification, the search can be carried out efficiently. If necessary, the system can also cooperate with a pet rescue team to carry out more effective search activities. When searching, the device's map function and GPS can be used to confirm the specific search location. This makes it possible to quickly find and protect the missing pet.

[0116] (Application example 1)

[0117] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0118] There is a lack of methods to quickly and efficiently find and protect missing animals. There is also a need for a system that can predict behavior in real time based on information provided by owners, thereby increasing the probability of finding animals.

[0119] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0120] In this invention, the server includes a terminal means for uploading photos of missing animals, an information processing means for receiving the uploaded photos and extracting features, an information processing means for performing information matching with an information storage means of a shelter based on the features, an information processing means for predicting behavior based on the results of the information matching, and a terminal means for notifying the results of the behavior prediction in real time, thereby enabling the efficient discovery and protection of missing animals.

[0121] A "missing animal" is a domestic animal that has become separated from its owner and whose whereabouts cannot be confirmed.

[0122] "Terminal means" refers to devices or equipment that can be operated by users, and specifically includes smartphones, tablets, and personal computers.

[0123] "Information processing means" refers to a device or system that analyzes and processes received data, and specifically includes a server or computer system.

[0124] "Photo feature extraction" is the process of using image processing technology to analyze features such as shape and color from uploaded photos and extract them as data.

[0125] A "shelter" is a facility where lost animals are rescued and temporarily housed, including animal shelters and rescue centers.

[0126] "Information storage means" refers to a device or system that stores and saves information in the form of a database or the like.

[0127] "Information matching" is the process of comparing and matching extracted data with other data sets.

[0128] "Behavioral prediction" is the process of predicting the future behavior and destinations of animals based on past data and current conditions.

[0129] "Notification means" refers to the means of communication used to inform users of analysis and prediction results, and specifically includes push notifications, email, SMS, etc.

[0130] The present invention is a comprehensive AI system for quickly locating and protecting missing animals. This system begins when a user uploads data about the missing animal (photos, sounds, GPS information) using a terminal. Specific embodiments of this system are described below.

[0131] Overall system description

[0132] The system mainly consists of the following components:

[0133] Terminal means

[0134] Server Means

[0135] Information Processing Means

[0136] Information storage means

[0137] Information matching method

[0138] Behavioral prediction methods

[0139] Notification means

[0140] Data collection

[0141] Users use a dedicated smartphone application to upload photos, sounds, and GPS data of missing animals from the terminal means, and all of this data is sent to the server means.

[0142] Data analysis and feature extraction

[0143] The server analyzes the received photo data using image analysis software called OpenCV to extract specific features (such as the animal's shape and body color). Similarly, audio analysis using Librosa analyzes the call data for characteristics such as frequency characteristics and sound spectrum. Additionally, geopy is used to analyze GPS data to determine the animal's current location and movement patterns.

[0144] Data matching and behavior prediction

[0145] The server compares the analyzed features with the shelter's database and information on the Internet. If the same or similar animal is found, the server extracts its information. Furthermore, the server predicts the animal's behavior based on the analyzed GPS data. This predicts the animal's next likely destination.

[0146] Notification and Discovery Assistance

[0147] The matching results and behavior prediction results are sent to the user's terminal in real time. Users can use this information to efficiently search for animals. If necessary, they can also work with a specialized rescue team to carry out search activities.

[0148] Specific examples

[0149] For example, if a user were to search for a missing dog, the process would go something like this: First, the user would use a dedicated application to upload a photo of the dog, its barks, and GPS data showing its recent movement route. The server receives the data, extracts the dog's characteristics from the photo, performs audio analysis of the barks, and analyzes the GPS data. The server then matches the data with information from shelters and online sources to see if a similar dog is being cared for. At the same time, it predicts the dog's behavior based on the GPS data and determines the expected search area. These results are then sent to the user's device in real time, allowing the user to efficiently search for the dog.

[0150] Prompt Sentence Examples

[0151] I have lost my dog ​​and would like some help finding him. I am providing the following information:

[0152] Photo: <Attached image file>

[0153] Sounds: <audio file>

[0154] GPS information: Latitude: 35.6895, Longitude: 139.6917

[0155] Please provide a location prediction, including recent movement patterns from the starting point.

[0156] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0157] Step 1: Upload your data

[0158] Description: A user uses a smartphone application to upload photos, sounds, and GPS data about missing animals.

[0159] Input: Photo files, audio files, GPS data (latitude and longitude)

[0160] Output: Photos, audio data, and GPS data uploaded to the server

[0161] Specific operation: The user accesses the input form within the app, selects and attaches each data file, and then presses the send button. This action sends the data from the device to the server.

[0162] Step 2: Save your data

[0163] Description: The server stores the received data in a database.

[0164] Input: Uploaded photos, audio data, GPS data

[0165] Output: Photos, audio data, and GPS data stored in the database

[0166] What happens: The server stores the received data in temporary storage and creates the appropriate database entries to store each file.

[0167] Step 3: Extracting image features

[0168] Description: The server extracts specific features from the stored photo data using OpenCV.

[0169] Input: Saved photo data

[0170] Output: Image feature data (specific patterns such as shape, color, etc.)

[0171] Specific operation: The server converts the image to grayscale using the OpenCV library, and detects key points and extracts feature vectors using the SIFT (Scale-Invariant Feature Transform) algorithm.

[0172] Step 4: Extract audio features

[0173] Description: The server analyzes the stored audio data using Librosa to extract certain audio features.

[0174] Input: Stored audio data

[0175] Output: Audio feature data (frequency characteristics, spectrum, etc.)

[0176] Specific operation: The server loads the audio data using the Librosa library and calculates MFCCs (Mel-Frequency Cepstrum Coefficients) to extract audio features.

[0177] Step 5: Analyzing the GPS data

[0178] Description: The server analyzes stored GPS data to determine the current location and movement patterns of animals.

[0179] Input: Stored GPS data

[0180] Output: Analyzed location information and movement patterns

[0181] What it does: The server uses the geopy library to analyze GPS data, identify movement patterns from past history, and predict possible next destinations.

[0182] Step 6: Data collation

[0183] Description: The server compares the extracted feature data with the shelter's database and online information.

[0184] Input: Image feature data, audio feature data, GPS analysis data

[0185] Output: Matched animal information

[0186] How it works: The server accesses shelter and online databases, applies a comparison and matching algorithm to the extracted feature data, calculates a similarity score, and identifies matching animals.

[0187] Step 7: Predict behavior

[0188] Description: The server makes animal behavior predictions based on the analyzed GPS data.

[0189] Input: Analyzed location and movement patterns

[0190] Output: Predictions of where the animal might go next

[0191] How it works: The server uses past movement data and current location information to apply an algorithm to predict where the animal will go next, along with other data such as the animal's species, age, and surrounding environment.

[0192] Step 8: Notification

[0193] Description: The server notifies the user of the matching results and behavior prediction results to the user's terminal means.

[0194] Input: Matched animal information, behavior prediction information

[0195] Output: A notification message that is displayed on the user's terminal.

[0196] Specific operation: The server compiles the matching results and behavior prediction information and sends them to the user's device via push notification, email, SMS, etc. This allows the user to know where the animal was found and potential locations to find it next.

[0197] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0198] Overall system description

[0199] This invention is an AI system that supports the rapid discovery and protection of lost pets, and improves the user experience by combining it with an emotion engine that recognizes the user's emotions. In this system, a server receives photos, sounds, GPS data, etc. of pets uploaded by owners using a terminal, analyzes and matches them, and adjusts search support based on the user's emotions recognized by the emotion engine.

[0200] Data collection and storage

[0201] Users use a dedicated application to upload photos, sounds, and GPS data of their pets. This data is sent from the device to a server, which then stores the received data in an internal database. In addition, the user's emotional data is also collected through an emotion engine.

[0202] Data analysis and feature extraction

[0203] The server inputs the saved pet photos into an AI analysis module to extract specific features (shape, color, etc.). The audio analysis module also analyzes the uploaded pet's cries for waveform and frequency characteristics. GPS data is analyzed in real time and displayed as a map. An emotion engine analyzes the user's emotional state and combines that data with other data.

[0204] Data Matching

[0205] The server then matches the extracted features and the results of the voice analysis with information from shelter databases and online sources. The shelter database contains information registered when a missing pet is rescued. If an identical or similar pet is found, the server can quickly notify the owner.

[0206] Behavioral prediction

[0207] The server predicts the pet's next likely location based on GPS data and its movement patterns. For example, it takes into account the pet's type, age, personality, and surrounding terrain information. This behavior prediction effectively narrows the search area.

[0208] Emotional Engine Adjustment

[0209] The emotion engine analyzes the user's emotional state and adjusts its search assistance based on its output. For example, if the user is extremely anxious, it will change the priority of notifications and provide urgent information immediately. It also provides customized messages and advice based on the user's emotions, reducing anxiety and stress.

[0210] Notification and Discovery Assistance

[0211] Matching results and behavior prediction results are sent to the device in real time. The user receives this notification and begins searching for a pet based on the provided information. If necessary, the system can also work with a specialized pet rescue team to provide optimal support. The analysis results of the emotion engine are also reflected in the notification, and support is provided according to the user's feelings.

[0212] Specific examples

[0213] For example, if a user were searching for a missing cat, the process would be as follows: First, the user would use a dedicated application to upload the cat's photo, meows, and GPS data. The server receives this data, extracts the cat's characteristics from the photo, performs audio analysis of the meows, and analyzes the GPS data in real time. The server then matches the data with information from shelters and online sources to see if a similar cat is being cared for. At the same time, it predicts the cat's behavior based on the GPS data and determines the expected search area. The matching and predicted behavior results are notified to the device in real time, allowing the user to check them and search for the cat efficiently. Furthermore, the emotion engine analyzes the user's emotions and provides messages and notifications according to the level of urgency, thereby reducing the user's anxiety and supporting the search process.

[0214] In this way, the present invention is a system that combines AI technology, data matching technology, and an emotion engine to quickly and effectively find and protect lost pets, while also providing support that takes into consideration the emotions of pet owners.

[0215] The processing flow will be explained below.

[0216] Step 1:

[0217] Users launch a dedicated application, capture photos of their missing pet, record its cries, and collect GPS data, which are then uploaded to a server via the application.

[0218] Step 2:

[0219] To collect the user's emotional state, the device uses an emotion engine to analyze the user's facial expressions, tone of voice, input text, etc. The analyzed emotional data is also sent to the server.

[0220] Step 3:

[0221] The server receives the uploaded pet photos, sounds, and GPS data and stores them in a designated database, as well as the received emotion data.

[0222] Step 4:

[0223] The server inputs the stored pet photos into an AI analysis module and uses image recognition technology to extract the pet's characteristics, such as fur color, body shape, and pattern.

[0224] Step 5:

[0225] The server inputs the saved bark data into a voice analysis module and extracts the characteristics of the pet's voice by analyzing the frequency characteristics and waveform patterns.

[0226] Step 6:

[0227] The server analyzes the GPS data in real time and plots the pet's current location on a map. The location information is used by the behavior prediction module.

[0228] Step 7:

[0229] The server compares the extracted photo characteristics and audio analysis results with information from shelter databases and online sources, and retrieves information about pets with similar characteristics.

[0230] Step 8:

[0231] The server predicts the pet's next likely location based on GPS data and its movement patterns, taking into account factors such as the pet's type, age, personality, and surrounding terrain.

[0232] Step 9:

[0233] The server generates a map of the search area based on the matching results and behavior prediction results, and notifies the device in real time. The content and priority of notifications are adjusted based on the output of the emotion engine.

[0234] Step 10:

[0235] The device displays the notified information to the user, allowing the user to check the provided search area and pet characteristics. The user can then quickly start searching based on this information.

[0236] Step 11:

[0237] Users follow the instructions in the app to search for their pet within the provided search area, and if necessary, the app will connect them with a specialized pet rescue team to provide optimal support to help them find their pet.

[0238] Step 12:

[0239] The server collects emotional data in real time as the user explores, analyzes the user's stress level and anxiety using an emotion engine, and provides customized messages and advice to the user via their device based on the analysis results.

[0240] Step 13:

[0241] The device displays customized messages and advice provided by the emotion engine to the user to support their exploration activities. If necessary, it re-analyzes the user's emotional state and makes further adjustments.

[0242] This allows for quick and effective detection and protection of pets, reducing the mental burden on owners.

[0243] Example 2

[0244] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0245] Conventional technologies have not been sufficient to quickly and effectively locate and protect lost pets. Furthermore, there is a lack of support that takes into account the owner's emotions. Therefore, there is a need for methods to improve the efficiency of searches and the user experience. In particular, there has been a challenge in developing a method to effectively locate pets using their appearance, sounds, and location information.

[0246] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0247] In this invention, the server includes a terminal means for uploading photos, sounds, and GPS data related to the missing pet, a server means for receiving the uploaded photos, sounds, and GPS data and extracting features including data derived from emotions, a server means for matching the features with data on shelters and information on the Internet, and an emotion engine means for analyzing the user's emotion data and adjusting the priority of notifications based on the output. This makes it possible to quickly and effectively find and protect missing pets and provide support that takes the user's emotions into consideration.

[0248] "Terminal means" refers to a device or application that allows a user to upload information about a missing pet.

[0249] "Server means" means a computer system for processing, storing and analyzing data received from the terminal.

[0250] The "emotion engine means" is a software module that analyzes the user's emotional state and adjusts the system's operation based on the results.

[0251] "Features" are identifying information such as shape, color, and sound characteristics extracted from photos and sounds of pets.

[0252] "Matching" is the process of comparing the extracted features with data from shelters and online sources.

[0253] "Shelter" means a placement facility where lost pets may be taken.

[0254] "GPS data" is geographic information that indicates your pet's current location and last known location.

[0255] "Audio analysis means" is a technology for analyzing uploaded bird call data and extracting its characteristics.

[0256] "Data analysis means" refers to technology for predicting pet behavior based on GPS data and past behavioral patterns.

[0257] "Notification means" is a system function for notifying users of matching results and behavior prediction results.

[0258] MODE FOR CARRYING OUT THE INVENTION

[0259] This invention is an AI system that supports the rapid discovery and protection of lost pets, and improves the user experience by combining it with an emotion engine that recognizes the user's emotions. In this system, the server receives, analyzes, and matches photos, sounds, and GPS data of pets uploaded by owners using their devices, and adjusts search support based on the user's emotions recognized by the emotion engine.

[0260] Terminal means:

[0261] Users use a dedicated application to upload photos of their pets, their pet's cries, and GPS data. This data is sent from the device to the server. Photos taken by the user are sent in JPEG format, recorded pet cries are sent in WAV format, and GPS data is sent in JSON format. In addition, the device captures the user's facial expressions with its camera and sends the analysis results to the server via an emotion engine.

[0262] Server means:

[0263] The server stores the received data in an internal database. The stored pet photos are input into an AI image analysis module (e.g., YOLO or ResNet) to extract specific features (shape, color, pattern, etc.). An audio analysis module (e.g., MFCC or WaveNet) is used to analyze the waveform and frequency characteristics of the pet's bark data. The server also analyzes GPS data in conjunction with a map API (e.g., Google Maps API) to determine location information in real time. The emotion engine analyzes the user's emotional state and stores the data on the server.

[0264] Feature data matching:

[0265] The server then matches the extracted features and the results of the voice analysis with information from shelter databases and on the Internet. The shelter database contains information about lost pets that are registered when they are rescued. The server compares this data and promptly notifies the user if the same or similar pet is found.

[0266] Behavioral prediction and emotional data adjustment:

[0267] The server predicts the pet's next likely location based on GPS data and its behavioral patterns. This prediction is performed using an AI behavioral prediction model (e.g., LSTM or RNN model). It also adjusts the priority of notification messages based on the analysis results of the emotion engine. It generates customized messages according to specific emotional states and provides them to the user.

[0268] Notifications and Support:

[0269] Matching results and behavior prediction results are sent to the device in real time. Users can receive this notification and begin searching for their pet based on the provided information. Furthermore, if necessary, they can work with a specialized pet rescue team to provide optimal support.

[0270] Specific examples

[0271] For example, if a user were searching for a missing cat, the process would be as follows: First, the user would use a dedicated application to upload the cat's photo, meows, and GPS data. The server receives this data, extracts the cat's characteristics from the photo, performs audio analysis of the meows, and analyzes the GPS data in real time. The server then matches the data with information from shelters and online sources to see if a similar cat is being cared for. At the same time, it predicts the cat's behavior based on the GPS data and determines the expected search area. These matching and behavior prediction results are notified to the device in real time, allowing the user to search for the cat efficiently. Furthermore, the emotion engine analyzes the user's emotions and provides messages and notifications according to the level of urgency, thereby reducing the user's anxiety and supporting the search process.

[0272] Prompt Sentence Examples

[0273] Please explain the specific process of how the server uses the cat photos, meows, and GPS data uploaded from the dedicated app.

[0274] In this way, the present invention combines AI technology, data matching technology, and an emotion engine to quickly and effectively locate and protect lost pets, and provide emotional support to pet owners.

[0275] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0276] Step 1: Data collection

[0277] Users use a dedicated application to upload photos, sounds, and GPS data of their pets.

[0278] Input: Pet photo (JPEG format), pet sound (WAV format), GPS data (JSON format)

[0279] Output: Collected data is temporarily stored on the device and ready to be sent.

[0280] Specific operation: The user takes a photo of the cat using a smartphone and saves it in the application. The user records the cat's meows and uses the smartphone's GPS function to collect location information.

[0281] Step 2: Send data

[0282] The terminal transmits the uploaded data from the user to a server via the Internet.

[0283] Input: Collected photos, calls, and GPS data

[0284] Output: Data is sent to the server.

[0285] How it works: The device uses Wi-Fi or mobile data to upload photos and bird call data to the server, and GPS data is also sent to the server at the same time.

[0286] Step 3: Save Data

[0287] The server stores the received data in an internal database.

[0288] Input: Photos, sounds, and GPS data sent to the server

[0289] Output: Photos, calls, and GPS data stored in a database

[0290] Specific operation: The server stores JPEG images in the "Pet Image" table, WAV audio data in the "Pet Audio" table, and JSON-formatted GPS data in the "Pet Location Information" table.

[0291] Step 4: Sentiment Data Analysis

[0292] The device analyzes the user's emotional state using a dedicated emotion engine and sends the data to a server.

[0293] Input: User's facial expression data

[0294] Output: Parsed emotional state data

[0295] Specific operation: The device uses an emotion engine to analyze the user's facial expressions and recognize emotional states such as "anxiety" or "impatience." The analysis results are sent to the server.

[0296] Step 5: Image analysis

[0297] The server uses an image analysis module to extract specific features from stored photos of pets.

[0298] Input: Saved pet photo

[0299] Output: Extracted feature data (shape, color, pattern, etc.)

[0300] How it works: The server uses an AI image analysis model (such as YOLO or ResNet) to scan the pet's image and extract features such as shape, color, and pattern. These feature data are stored in a "feature data" table.

[0301] Step 6: Audio analysis

[0302] The server uses an audio analysis module to analyze the stored bird call data.

[0303] Input: Stored call data

[0304] Output: Analyzed audio feature data (waveform, frequency characteristics, etc.)

[0305] Specific operation: The server uses an audio analysis module (such as MFCC or WaveNet) to analyze the waveform and frequency characteristics of the bird call data and generate audio feature data. This data is stored in the "audio feature data" table.

[0306] Step 7: GPS data analysis

[0307] The server analyzes the GPS data in real time to determine the location.

[0308] Input: Stored GPS data

[0309] Output: Parsed map information

[0310] What it does: The server connects the GPS data to the Google Maps API and visualizes the location information in real time. This information is stored in the "Map Data" table.

[0311] Step 8: Data Matching

[0312] The server matches the extracted features and the results of voice analysis with information from the shelter's database and online sources.

[0313] Input: Pet feature data, audio feature data

[0314] Output: Matching result data

[0315] Specific operation: The server accesses the shelter's database to search for pets with matching characteristics, and then compares them with online pet information to obtain matching results, which are then stored in a "matching results" table.

[0316] Step 9: Behavioral Prediction

[0317] The server uses GPS data and the pet's behavioral patterns to predict where the pet may move next.

[0318] Input: Stored GPS data, behavioral pattern data

[0319] Output: Predicted action location data

[0320] Specific operation: The server uses an AI behavior prediction model (LSTM or RNN model) based on past behavior data to predict the pet's movement pattern. The predicted location data is saved in the "Behavior prediction data" table.

[0321] Step 10: Emotional Data Adjustment

[0322] The server analyzes the user's emotional state and adjusts the search assistance based on its output.

[0323] Input: Parsed emotion data

[0324] Output: Reconciled notification data

[0325] Specific operation: The server adjusts the notification priority based on the analysis results of the emotion engine and generates messages that quickly provide the most important information to the user.

[0326] Step 11: Notification and Support

[0327] The server notifies the terminal of the matching results and behavior prediction results in real time and notifies the user.

[0328] Input: Matching result data, behavior prediction data

[0329] Output: Notification data to the device

[0330] Specific operation: The server compiles the matching results and behavior prediction results and sends them to the device as a push notification. The user can use this notification to continue searching for their pet. In addition, in the event of an emergency, the system can cooperate with a specialized pet rescue team to provide support.

[0331] (Application example 2)

[0332] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0333] Conventional food delivery systems have difficulty improving customer satisfaction and providing efficient services because they do not adequately consider users' emotional state and real-time feedback, which leads to problems such as user frustration and insufficient order optimization.

[0334] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means for uploading photos of missing pets, server means for receiving the uploaded photos and extracting features, server means for matching the photos with a database of shelters based on the features, server means including an emotion engine for analyzing the user's emotion data, and server means for adjusting notifications based on the emotion data. This makes it possible to grasp the user's emotional state in real time and provide optimal menus and notifications based on that.

[0335] "Terminal" refers to a device that allows a user to input information or send data to a server.

[0336] "Server" refers to a computer system that receives and stores uploaded data and performs various analyses.

[0337] "Features" refer to identifiable information or patterns extracted from image or audio data.

[0338] "Server means" refers to a combination of hardware and software that implements a particular function within a server.

[0339] An "emotion engine" refers to an artificial intelligence mechanism that analyzes the user's emotional state and outputs analytical results based on that data.

[0340] "Notification" refers to information or messages sent by the server to the user.

[0341] "Matching" refers to the process of searching for and matching relevant data and information based on characteristics and analysis results.

[0342] A "database" refers to a collection of information that stores specific information in an organized manner and is configured to facilitate search and retrieval.

[0343] In this invention, a system is constructed for analyzing the emotional state of a user and providing optimal menus and notifications in real time in the operation of a food delivery system. Specific embodiments of this system are described below.

[0344] Overall system description

[0345] A food delivery service application consists of the following main components:

[0346] Terminal means: A device such as a smartphone through which a user enters information and transmits the data to a server.

[0347] Server means: A computer system that receives, stores, and performs various analyses of data.

[0348] Emotion Engine: An AI module that analyzes the user's emotional state.

[0349] Notification vehicle: A mechanism for delivering information from a server to a user.

[0350] Data collection and storage

[0351] Users use a smartphone app to upload food photos, ratings, GPS data, and comments. This data is sent from the smartphone to a server, which stores the received data in an internal database. User emotion data is also collected through an emotion engine.

[0352] Data analysis and feature extraction

[0353] The server inputs the saved food photos into an AI analysis module to extract specific features (shape, color, presentation, etc.). It also performs sentiment analysis on uploaded comments using natural language processing technology. GPS data is analyzed in real time and converted into map information. The emotion engine analyzes the user's emotional state and combines that data with other data.

[0354] Data Matching

[0355] The server then matches the extracted features and sentiment analysis results with food delivery menus and store information, enabling users to quickly obtain menus and store information that meet their desired criteria.

[0356] Behavioral prediction

[0357] The server predicts users' behavior based on GPS data and their past order history. For example, it takes into account their preferences and rating history to predict what menu items they are likely to order next. This behavior prediction allows delivery companies and restaurants to efficiently manage and prepare inventory.

[0358] Emotional Engine Adjustment

[0359] The emotion engine analyzes the user's emotional state and tailors notifications based on its output. For example, if the user is feeling stressed, it may send relaxing menu recommendations or inspiring messages to reduce anxiety and stress.

[0360] Notifications and Support

[0361] Matching results and behavioral prediction results are sent to the user's smartphone in real time. Users can receive these notifications and place orders immediately. In addition, the analysis results of the emotion engine are reflected in the notifications, providing support that takes the user's emotions into consideration.

[0362] Specific examples

[0363] For example, if a user wants to eat delicious pasta, the process goes like this: First, the user uses a smartphone app to upload a photo of themselves along with a comment saying, "I want to eat delicious pasta." The server receives this data, analyzes the user's emotions from the photo, and analyzes the comment using natural language processing. Then, based on the extracted features, it matches relevant store and menu information and notifies the user's smartphone. The emotion engine analyzes the user's emotions and sends notifications recommending relaxing menu items, thereby reducing the user's anxiety and providing the optimal food delivery service.

[0364] Prompt Sentence Examples

[0365] "Implement an application that identifies emotions from a user's facial image, sends it to a server along with GPS data and comments, and then notifies a food delivery service of menu recommendations in real time based on that data."

[0366] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0367] Step 1:

[0368] Users use a smartphone app to upload photos of food, GPS data, and comments, along with a photo of their face, which the system receives as input. The uploaded data is then sent to a server via an internet connection.

[0369] Step 2:

[0370] The server stores the received data. The stored data includes the user's photos, GPS data, and comments. The server first sends the user's photos to the emotion engine and identifies emotions from the facial images. This emotion data is analyzed by the emotion engine's AI model.

[0371] Step 3:

[0372] The server receives the emotion data output by the emotion engine and sends it along with the comment data to a natural language processing (NLP) module. The NLP module analyzes the emotion data and comments to identify the user's current emotional state and the specific content of the comment. This data analysis clarifies the user's desires and emotional state.

[0373] Step 4:

[0374] Based on the analysis results, the server sends the stored food photo data to a feature extraction module, which extracts features such as shape, color, and presentation from the image data. This feature data is used to match menus and restaurant information.

[0375] Step 5:

[0376] The server searches internal databases and external related information based on the extracted feature data and emotion data to match the optimal menu and restaurant information. The matching results are selected based on the user's current emotional state and the content of their comments.

[0377] Step 6:

[0378] The server organizes the matching results and sends real-time notifications to the user's smartphone, including recommended menu items, restaurant information, and a customized message based on the results of the emotion engine's analysis.

[0379] Step 7:

[0380] The user receives a notification, reviews the information displayed in the application, and places an order if necessary, ensuring that the user selects the menu that best suits their emotional state and needs.

[0381] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0382] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0383] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0384] [Second embodiment]

[0385] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0386] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0387] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0388] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0389] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0390] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0391] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0392] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0393] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0394] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0395] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0396] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0397] Overall system description

[0398] This invention is an AI system that supports the rapid discovery and protection of missing pets. This system uses a server to receive and analyze photos, sounds, and GPS data of pets uploaded by owners using a terminal, and then identifies the pet's location and predicts its behavior by performing analysis and matching.

[0399] Data collection and storage

[0400] Users use a dedicated application to upload photos, sounds, and GPS data of their pets, which are then sent from the device to a server, which then stores the data in an internal database.

[0401] Data analysis and feature extraction

[0402] The server inputs the saved pet photos into the AI ​​model to extract specific features (shape, color, etc.). It also performs audio analysis on the uploaded pet's cries to extract their waveform and frequency characteristics. GPS data is analyzed in real time to capture location information and generate a map.

[0403] Data Matching

[0404] The server then matches the extracted features and the results of the voice analysis with information from shelter databases and online sources. The shelter database contains information registered when a missing pet is rescued. This allows the server to quickly notify owners if the same or similar pet is found.

[0405] Behavioral prediction

[0406] Furthermore, the server predicts pet behavior based on GPS data, taking into account the pet's breed, age, personality, surrounding terrain, etc., to estimate where the pet may head next. This behavior prediction helps to efficiently narrow the search area.

[0407] Notification and Discovery Assistance

[0408] Matching results and behavior prediction results are sent to the device in real time. Users can receive this notification and begin searching for their pet based on the provided information. If necessary, the system can also connect with a specialized pet rescue team to provide optimal support.

[0409] Specific examples

[0410] For example, if a user were to search for a missing dog, the process would be as follows: First, the user would use a dedicated application to upload the dog's photo, bark, and GPS data. The server receives this data, extracts the dog's characteristics from the photo, performs audio analysis of the bark, and analyzes the GPS data in real time. The server then matches the data with information from shelters and online to see if a similar dog is being cared for. At the same time, it predicts the dog's behavior based on the GPS data and determines the expected search area. The matching and behavior prediction results are notified to the device in real time, allowing the user to check them and search for the dog efficiently.

[0411] In this way, the present invention is a system that combines AI technology and data matching technology to enable the quick and effective discovery and protection of lost pets.

[0412] The processing flow will be explained below.

[0413] Step 1:

[0414] Users launch a dedicated application, capture photos of their missing pet, record its cries, and collect GPS data, which are then uploaded to a server via the application.

[0415] Step 2:

[0416] The server receives the uploaded photos, bird calls, and GPS data and stores them in a designated database, where they are prepared for analysis.

[0417] Step 3:

[0418] The server inputs the stored pet photos into an AI analysis module and uses image recognition technology to extract the pet's characteristics, such as fur color, body shape, and pattern.

[0419] Step 4:

[0420] The server inputs the saved bark data into a voice analysis module and extracts the characteristics of the pet's voice by analyzing the frequency characteristics and waveform patterns.

[0421] Step 5:

[0422] The server analyzes the GPS data in real time and plots the pet's current location on a map. The location information is used by the behavior prediction module.

[0423] Step 6:

[0424] The server compares the extracted photo characteristics and audio analysis results with information from shelter databases and online sources, and retrieves information about pets with similar characteristics.

[0425] Step 7:

[0426] The server predicts the pet's next move based on GPS data and its behavioral patterns, taking into account the pet's type, age, personality, and surrounding terrain information, for example.

[0427] Step 8:

[0428] The server generates a map of the search area based on the matching results and behavior prediction results and notifies the terminal in real time.

[0429] Step 9:

[0430] The device displays the notified information to the user, allowing the user to check the provided search area and pet characteristics. The user can then quickly start searching based on this information.

[0431] Step 10:

[0432] Users follow the instructions in the app to search for their pet within the provided search area, and if necessary, the app will connect them with a specialized pet rescue team to provide optimal support to help them find their pet.

[0433] Example 1

[0434] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0435] To quickly find and protect lost pets, it is necessary to accurately analyze various data such as photos, sounds, and GPS data, and efficiently search for matching information. It is also necessary to provide clues for the search by predicting the pet's current location and behavior.

[0436] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0437] In this invention, the server includes an information processing terminal means for uploading data on missing pets, an information processing device means for receiving the uploaded data and extracting features, an information processing device means for comparing the features and analysis results with records of the shelter, an information processing device means for predicting behavior based on the data, and an information processing terminal means for notifying the analysis results and prediction results, thereby enabling the rapid identification and behavior prediction of missing pets and further real-time notification.

[0438] The "information processing terminal means" is a device for collecting data about missing pets and transmitting it to an information processing device.

[0439] The "information processing device means" is a device for analyzing received data, extracting and collating necessary information, and predicting behavior.

[0440] The "location information acquisition device means" refers to a device for acquiring coordinate data of the pet's current location, such as a GPS device.

[0441] "Feature extraction" is the process of extracting specific attributes, such as shape or waveform, from uploaded photos or audio data.

[0442] "Audio analysis" is the process of analyzing uploaded bird call data and extracting its waveform, frequency characteristics, etc.

[0443] "Data matching" is the process of checking whether the extracted characteristics match the shelter's records or online information.

[0444] "Behavior prediction" is the process of estimating where your pet will go next, taking into account the pet's breed, age, personality, terrain, GPS data, and more.

[0445] "Real-time notification" is a notification function that instantly notifies users of analysis results and behavior prediction results.

[0446] System Overview

[0447] The present invention is an AI system designed to support the rapid discovery and protection of missing pets. This system is composed of an information processing terminal means, an information processing device means, and a location information acquisition device means. The information processing terminal means is a device that collects and transmits photos, audio, GPS data, etc. of pets, and the information processing device means analyzes, collates, and predicts the received data and transmits the results. The location information acquisition device means is a device that acquires pet location information in real time.

[0448] Hardware and software used

[0449] The server uses a computer with high-performance data processing capabilities. Deep learning frameworks such as TensorFlow and PyTorch are used to analyze the photo data. Audio analysis tools such as Librosa and WaveSurfer are used to analyze the audio data. Geographic information systems such as PostGIS are used to process GPS data. An SQL-based database system is used for database management.

[0450] Data collection and transmission

[0451] Users use a dedicated application to collect and upload data about their missing pets. This application runs on mobile platforms such as iOS and Android. For example, by entering data such as a photo of the missing dog, the dog's barking sound, and the current GPS location, users can easily send the data to the server by pressing the send button.

[0452] Data analysis and feature extraction

[0453] The server receives the uploaded data and performs the following analysis on each data. Photo data is input into a TensorFlow model to extract features such as the pet's shape and color. Audio data is analyzed using Librosa to extract waveforms and frequency characteristics. GPS data is processed using PostGIS and plotted on a map.

[0454] Data collation and behavior prediction

[0455] The analyzed feature data is compared with the shelter's database and data on the internet, for example by running a query such as "SELECT FROM shelter_pets WHERE shape = 'dog' AND color = 'brown'." The server then uses machine learning models such as random forests and neural networks to predict the pet's behavior and estimate its next likely destination.

[0456] Notification and Discovery Assistance

[0457] The server notifies the device of the matching results and behavior prediction results in real time. This notification is provided to the user in the form of a push notification or an in-app notification. For example, specific information such as "A similar pet has been found at shelter A" or "The next likely destination is park B" is provided.

[0458] Specific examples

[0459] For example, if a user is searching for a missing dog, they use a dedicated application to upload the following data: a photo of the dog (e.g., Fido_missing.jpg), the dog's barks (e.g., Fido_bark.wav), and current GPS data (e.g., GPS_location.csv), which are then sent directly to the server. The server receives this data and analyzes and extracts features using TensorFlow, Librosa, PostGIS, and other software. The results are compared with the shelter's database and information on the Internet to predict behavior. Finally, the results of the comparison and predicted behavior are sent to the user's device in real time, allowing for efficient searching for the pet.

[0460] The system combines AI and data analysis technologies to enable the fast and effective discovery and protection of lost pets.

[0461] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0462] Step 1: Upload your data

[0463] A user uses a dedicated application to collect photos, sounds, and GPS data related to a missing pet, and then sends this data to a server from an information processing device. For example, a user presses the "Add a new pet" button in the application, takes a photo of the pet, and uploads it. They also record the sound of the pet's sounds and upload that file. Next, the device's GPS function is used to obtain the pet's current location, which is also sent. The input data consists of a photo file (e.g., Fido_missing.jpg), an audio file (e.g., Fido_sound.wav), and GPS text data (e.g., GPS_location.csv). This data is sent to the server.

[0464] Step 2: Receiving and storing data

[0465] The server receives the data sent from the device and checks the format and consistency of each piece of data. After checking, it stores the received photo data in the "photos" table, the bird sound data in the "audio" table, and the GPS data in the "gps_logs" table. Specifically, the server stores each piece of data in temporary storage, verifies whether it is in the correct data format, and then inserts it into the corresponding database table. At this time, metadata (e.g., data timestamp, user ID) is also saved. If the data is successfully saved in the database, the server generates a success message and proceeds to the next step.

[0466] Step 3: Data analysis and feature extraction

[0467] The server analyzes the received and stored data and extracts its features. The photo data is input into a TensorFlow model to extract features such as the pet's shape and color. The input data is the stored photo file, and the output data is the extracted feature vector. Similarly, Librosa is used to analyze the sound waveform and frequency characteristics of the pet's bark data and extract a feature vector. The GPS data is plotted on a map based on the coordinate information, and the location information is confirmed. Specifically, the server feeds the photo data into a deep learning model and extracts features using a convolutional neural network (CNN). The analysis results are saved as a feature vector. Similarly, audio features such as power spectrograms and Mel-Frequency Cepstrum (MFCC) are extracted from the audio data and saved as a feature vector. PostGIS is used to convert the GPS data into geographic coordinates and plot them on a map.

[0468] Step 4: Collating the data

[0469] The server uses the extracted feature vectors to match the search results with the shelter's database and information on the internet. This process involves generating an SQL-based query to perform database matching, for example, "SELECT FROM shelter_pets WHERE shape = 'dog' AND color = 'brown'". The input data is the extracted feature vectors, and the output data is a list of matching database records. After the matching process, the server passes the details of any matching records to the next step.

[0470] Step 5: Predict behavior

[0471] The server comprehensively evaluates information such as the pet's species, age, personality, and GPS data to predict its behavior. It then uses machine learning algorithms such as random forests and neural networks to estimate the pet's next likely destination. The input data is the pet's characteristics and current GPS data, and the output data is the predicted range of movement and the next destination. Specifically, the server inputs known behavioral pattern data and current location information into a machine learning model to obtain a prediction result. For example, it generates a prediction result in the form of "The most likely next destination is a nearby park."

[0472] Step 6: Notification

[0473] The server notifies the user of the matching results and behavior prediction results in real time via push notifications or in-app notifications. The input data are the matching results and behavior prediction results obtained in the previous step, and the output data is the notification message. For example, messages such as "A similar pet is at shelter A" or "The next likely destination is park B" are sent to the user's device.

[0474] Step 7: Search support

[0475] The user receives the notification and quickly begins searching for the pet based on the provided information. For example, by heading to Park B based on the notification, the search can be carried out efficiently. If necessary, the system can also cooperate with a pet rescue team to carry out more effective search activities. When searching, the device's map function and GPS can be used to confirm the specific search location. This makes it possible to quickly find and protect the missing pet.

[0476] (Application example 1)

[0477] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0478] There is a lack of methods to quickly and efficiently find and protect missing animals. There is also a need for a system that can predict behavior in real time based on information provided by owners, thereby increasing the probability of finding animals.

[0479] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0480] In this invention, the server includes a terminal means for uploading photos of missing animals, an information processing means for receiving the uploaded photos and extracting features, an information processing means for performing information matching with an information storage means of a shelter based on the features, an information processing means for predicting behavior based on the results of the information matching, and a terminal means for notifying the results of the behavior prediction in real time, thereby enabling the efficient discovery and protection of missing animals.

[0481] A "missing animal" is a domestic animal that has become separated from its owner and whose whereabouts cannot be confirmed.

[0482] "Terminal means" refers to devices or equipment that can be operated by users, and specifically includes smartphones, tablets, and personal computers.

[0483] "Information processing means" refers to a device or system that analyzes and processes received data, and specifically includes a server or computer system.

[0484] "Photo feature extraction" is the process of using image processing technology to analyze features such as shape and color from uploaded photos and extract them as data.

[0485] A "shelter" is a facility where lost animals are rescued and temporarily housed, including animal shelters and rescue centers.

[0486] "Information storage means" refers to a device or system that stores and saves information in the form of a database or the like.

[0487] "Information matching" is the process of comparing and matching extracted data with other data sets.

[0488] "Behavioral prediction" is the process of predicting the future behavior and destinations of animals based on past data and current conditions.

[0489] "Notification means" refers to the means of communication used to inform users of analysis and prediction results, and specifically includes push notifications, email, SMS, etc.

[0490] The present invention is a comprehensive AI system for quickly locating and protecting missing animals. This system begins when a user uploads data about the missing animal (photos, sounds, GPS information) using a terminal. Specific embodiments of this system are described below.

[0491] Overall system description

[0492] The system mainly consists of the following components:

[0493] Terminal means

[0494] Server Means

[0495] Information Processing Means

[0496] Information storage means

[0497] Information matching method

[0498] Behavioral prediction methods

[0499] Notification means

[0500] Data collection

[0501] Users use a dedicated smartphone application to upload photos, sounds, and GPS data of missing animals from the terminal means, and all of this data is sent to the server means.

[0502] Data analysis and feature extraction

[0503] The server analyzes the received photo data using image analysis software called OpenCV to extract specific features (such as the animal's shape and body color). Similarly, audio analysis using Librosa analyzes the call data for characteristics such as frequency characteristics and sound spectrum. Additionally, geopy is used to analyze GPS data to determine the animal's current location and movement patterns.

[0504] Data matching and behavior prediction

[0505] The server compares the analyzed features with the shelter's database and information on the Internet. If the same or similar animal is found, the server extracts its information. Furthermore, the server predicts the animal's behavior based on the analyzed GPS data. This predicts the animal's next likely destination.

[0506] Notification and Discovery Assistance

[0507] The matching results and behavior prediction results are sent to the user's terminal in real time. Users can use this information to efficiently search for animals. If necessary, they can also work with a specialized rescue team to carry out search activities.

[0508] Specific examples

[0509] For example, if a user were to search for a missing dog, the process would go something like this: First, the user would use a dedicated application to upload a photo of the dog, its barks, and GPS data showing its recent movement route. The server receives the data, extracts the dog's characteristics from the photo, performs audio analysis of the barks, and analyzes the GPS data. The server then matches the data with information from shelters and online sources to see if a similar dog is being cared for. At the same time, it predicts the dog's behavior based on the GPS data and determines the expected search area. These results are then sent to the user's device in real time, allowing the user to efficiently search for the dog.

[0510] Prompt Sentence Examples

[0511] I have lost my dog ​​and would like some help finding him. I am providing the following information:

[0512] Photo: <Attached image file>

[0513] Sounds: <audio file>

[0514] GPS information: Latitude: 35.6895, Longitude: 139.6917

[0515] Please provide a location prediction, including recent movement patterns from the starting point.

[0516] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0517] Step 1: Upload your data

[0518] Description: A user uses a smartphone application to upload photos, sounds, and GPS data about missing animals.

[0519] Input: Photo files, audio files, GPS data (latitude and longitude)

[0520] Output: Photos, audio data, and GPS data uploaded to the server

[0521] Specific operation: The user accesses the input form within the app, selects and attaches each data file, and then presses the send button. This action sends the data from the device to the server.

[0522] Step 2: Save your data

[0523] Description: The server stores the received data in a database.

[0524] Input: Uploaded photos, audio data, GPS data

[0525] Output: Photos, audio data, and GPS data stored in the database

[0526] What happens: The server stores the received data in temporary storage and creates the appropriate database entries to store each file.

[0527] Step 3: Extracting image features

[0528] Description: The server extracts specific features from the stored photo data using OpenCV.

[0529] Input: Saved photo data

[0530] Output: Image feature data (specific patterns such as shape, color, etc.)

[0531] Specific operation: The server converts the image to grayscale using the OpenCV library, and detects key points and extracts feature vectors using the SIFT (Scale-Invariant Feature Transform) algorithm.

[0532] Step 4: Extract audio features

[0533] Description: The server analyzes the stored audio data using Librosa to extract certain audio features.

[0534] Input: Stored audio data

[0535] Output: Audio feature data (frequency characteristics, spectrum, etc.)

[0536] Specific operation: The server loads the audio data using the Librosa library and calculates MFCCs (Mel-Frequency Cepstrum Coefficients) to extract audio features.

[0537] Step 5: Analyzing the GPS data

[0538] Description: The server analyzes stored GPS data to determine the current location and movement patterns of animals.

[0539] Input: Stored GPS data

[0540] Output: Analyzed location information and movement patterns

[0541] What it does: The server uses the geopy library to analyze GPS data, identify movement patterns from past history, and predict possible next destinations.

[0542] Step 6: Data collation

[0543] Description: The server compares the extracted feature data with the shelter's database and online information.

[0544] Input: Image feature data, audio feature data, GPS analysis data

[0545] Output: Matched animal information

[0546] How it works: The server accesses shelter and online databases, applies a comparison and matching algorithm to the extracted feature data, calculates a similarity score, and identifies matching animals.

[0547] Step 7: Predict behavior

[0548] Description: The server makes animal behavior predictions based on the analyzed GPS data.

[0549] Input: Analyzed location and movement patterns

[0550] Output: Predictions of where the animal might go next

[0551] How it works: The server uses past movement data and current location information to apply an algorithm to predict where the animal will go next, along with other data such as the animal's species, age, and surrounding environment.

[0552] Step 8: Notification

[0553] Description: The server notifies the user of the matching results and behavior prediction results to the user's terminal means.

[0554] Input: Matched animal information, behavior prediction information

[0555] Output: A notification message that is displayed on the user's terminal.

[0556] Specific operation: The server compiles the matching results and behavior prediction information and sends them to the user's device via push notification, email, SMS, etc. This allows the user to know where the animal was found and potential locations to find it next.

[0557] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0558] Overall system description

[0559] This invention is an AI system that supports the rapid discovery and protection of lost pets, and improves the user experience by combining it with an emotion engine that recognizes the user's emotions. In this system, a server receives photos, sounds, GPS data, etc. of pets uploaded by owners using a terminal, analyzes and matches them, and adjusts search support based on the user's emotions recognized by the emotion engine.

[0560] Data collection and storage

[0561] Users use a dedicated application to upload photos, sounds, and GPS data of their pets. This data is sent from the device to a server, which then stores the received data in an internal database. In addition, the user's emotional data is also collected through an emotion engine.

[0562] Data analysis and feature extraction

[0563] The server inputs the saved pet photos into an AI analysis module to extract specific features (shape, color, etc.). The audio analysis module also analyzes the uploaded pet's cries for waveform and frequency characteristics. GPS data is analyzed in real time and displayed as a map. An emotion engine analyzes the user's emotional state and combines that data with other data.

[0564] Data Matching

[0565] The server then matches the extracted features and the results of the voice analysis with information from shelter databases and online sources. The shelter database contains information registered when a missing pet is rescued. If an identical or similar pet is found, the server can quickly notify the owner.

[0566] Behavioral prediction

[0567] The server predicts the pet's next likely location based on GPS data and its movement patterns. For example, it takes into account the pet's type, age, personality, and surrounding terrain information. This behavior prediction effectively narrows the search area.

[0568] Emotional Engine Adjustment

[0569] The emotion engine analyzes the user's emotional state and adjusts its search assistance based on its output. For example, if the user is extremely anxious, it will change the priority of notifications and provide urgent information immediately. It also provides customized messages and advice based on the user's emotions, reducing anxiety and stress.

[0570] Notification and Discovery Assistance

[0571] Matching results and behavior prediction results are sent to the device in real time. The user receives this notification and begins searching for a pet based on the provided information. If necessary, the system can also work with a specialized pet rescue team to provide optimal support. The analysis results of the emotion engine are also reflected in the notification, and support is provided according to the user's feelings.

[0572] Specific examples

[0573] For example, if a user were searching for a missing cat, the process would be as follows: First, the user would use a dedicated application to upload the cat's photo, meows, and GPS data. The server receives this data, extracts the cat's characteristics from the photo, performs audio analysis of the meows, and analyzes the GPS data in real time. The server then matches the data with information from shelters and online sources to see if a similar cat is being cared for. At the same time, it predicts the cat's behavior based on the GPS data and determines the expected search area. The matching and predicted behavior results are notified to the device in real time, allowing the user to check them and search for the cat efficiently. Furthermore, the emotion engine analyzes the user's emotions and provides messages and notifications according to the level of urgency, thereby reducing the user's anxiety and supporting the search process.

[0574] In this way, the present invention is a system that combines AI technology, data matching technology, and an emotion engine to quickly and effectively find and protect lost pets, while also providing support that takes into consideration the emotions of pet owners.

[0575] The processing flow will be explained below.

[0576] Step 1:

[0577] Users launch a dedicated application, capture photos of their missing pet, record its cries, and collect GPS data, which are then uploaded to a server via the application.

[0578] Step 2:

[0579] To collect the user's emotional state, the device uses an emotion engine to analyze the user's facial expressions, tone of voice, input text, etc. The analyzed emotional data is also sent to the server.

[0580] Step 3:

[0581] The server receives the uploaded pet photos, sounds, and GPS data and stores them in a designated database, as well as the received emotion data.

[0582] Step 4:

[0583] The server inputs the stored pet photos into an AI analysis module and uses image recognition technology to extract the pet's characteristics, such as fur color, body shape, and pattern.

[0584] Step 5:

[0585] The server inputs the saved bark data into a voice analysis module and extracts the characteristics of the pet's voice by analyzing the frequency characteristics and waveform patterns.

[0586] Step 6:

[0587] The server analyzes the GPS data in real time and plots the pet's current location on a map. The location information is used by the behavior prediction module.

[0588] Step 7:

[0589] The server compares the extracted photo characteristics and audio analysis results with information from shelter databases and online sources, and retrieves information about pets with similar characteristics.

[0590] Step 8:

[0591] The server predicts the pet's next likely location based on GPS data and its movement patterns, taking into account factors such as the pet's type, age, personality, and surrounding terrain.

[0592] Step 9:

[0593] The server generates a map of the search area based on the matching results and behavior prediction results, and notifies the device in real time. The content and priority of notifications are adjusted based on the output of the emotion engine.

[0594] Step 10:

[0595] The device displays the notified information to the user, allowing the user to check the provided search area and pet characteristics. The user can then quickly start searching based on this information.

[0596] Step 11:

[0597] Users follow the instructions in the app to search for their pet within the provided search area, and if necessary, the app will connect them with a specialized pet rescue team to provide optimal support to help them find their pet.

[0598] Step 12:

[0599] The server collects emotional data in real time as the user explores, analyzes the user's stress level and anxiety using an emotion engine, and provides customized messages and advice to the user via their device based on the analysis results.

[0600] Step 13:

[0601] The device displays customized messages and advice provided by the emotion engine to the user to support their exploration activities. If necessary, it re-analyzes the user's emotional state and makes further adjustments.

[0602] This allows for quick and effective detection and protection of pets, reducing the mental burden on owners.

[0603] Example 2

[0604] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0605] Conventional technologies have not been sufficient to quickly and effectively locate and protect lost pets. Furthermore, there is a lack of support that takes into account the owner's emotions. Therefore, there is a need for methods to improve the efficiency of searches and the user experience. In particular, there has been a challenge in developing a method to effectively locate pets using their appearance, sounds, and location information.

[0606] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0607] In this invention, the server includes a terminal means for uploading photos, sounds, and GPS data related to the missing pet, a server means for receiving the uploaded photos, sounds, and GPS data and extracting features including data derived from emotions, a server means for matching the features with data on shelters and information on the Internet, and an emotion engine means for analyzing the user's emotion data and adjusting the priority of notifications based on the output. This makes it possible to quickly and effectively find and protect missing pets and provide support that takes the user's emotions into consideration.

[0608] "Terminal means" refers to a device or application that allows a user to upload information about a missing pet.

[0609] "Server means" means a computer system for processing, storing and analyzing data received from the terminal.

[0610] The "emotion engine means" is a software module that analyzes the user's emotional state and adjusts the system's operation based on the results.

[0611] "Features" are identifying information such as shape, color, and sound characteristics extracted from photos and sounds of pets.

[0612] "Matching" is the process of comparing the extracted features with data from shelters and online sources.

[0613] "Shelter" means a placement facility where lost pets may be taken.

[0614] "GPS data" is geographic information that indicates your pet's current location and last known location.

[0615] "Audio analysis means" is a technology for analyzing uploaded bird call data and extracting its characteristics.

[0616] "Data analysis means" refers to technology for predicting pet behavior based on GPS data and past behavioral patterns.

[0617] "Notification means" is a system function for notifying users of matching results and behavior prediction results.

[0618] MODE FOR CARRYING OUT THE INVENTION

[0619] This invention is an AI system that supports the rapid discovery and protection of lost pets, and improves the user experience by combining it with an emotion engine that recognizes the user's emotions. In this system, the server receives, analyzes, and matches photos, sounds, and GPS data of pets uploaded by owners using their devices, and adjusts search support based on the user's emotions recognized by the emotion engine.

[0620] Terminal means:

[0621] Users use a dedicated application to upload photos of their pets, their pet's cries, and GPS data. This data is sent from the device to the server. Photos taken by the user are sent in JPEG format, recorded pet cries are sent in WAV format, and GPS data is sent in JSON format. In addition, the device captures the user's facial expressions with its camera and sends the analysis results to the server via an emotion engine.

[0622] Server means:

[0623] The server stores the received data in an internal database. The stored pet photos are input into an AI image analysis module (e.g., YOLO or ResNet) to extract specific features (shape, color, pattern, etc.). An audio analysis module (e.g., MFCC or WaveNet) is used to analyze the waveform and frequency characteristics of the pet's bark data. The server also analyzes GPS data in conjunction with a map API (e.g., Google Maps API) to determine location information in real time. The emotion engine analyzes the user's emotional state and stores the data on the server.

[0624] Feature data matching:

[0625] The server then matches the extracted features and the results of the voice analysis with information from shelter databases and on the Internet. The shelter database contains information about lost pets that are registered when they are rescued. The server compares this data and promptly notifies the user if the same or similar pet is found.

[0626] Behavioral prediction and emotional data adjustment:

[0627] The server predicts the pet's next likely location based on GPS data and its behavioral patterns. This prediction is performed using an AI behavioral prediction model (e.g., LSTM or RNN model). It also adjusts the priority of notification messages based on the analysis results of the emotion engine. It generates customized messages according to specific emotional states and provides them to the user.

[0628] Notifications and Support:

[0629] Matching results and behavior prediction results are sent to the device in real time. Users can receive this notification and begin searching for their pet based on the provided information. Furthermore, if necessary, they can work with a specialized pet rescue team to provide optimal support.

[0630] Specific examples

[0631] For example, if a user were searching for a missing cat, the process would be as follows: First, the user would use a dedicated application to upload the cat's photo, meows, and GPS data. The server receives this data, extracts the cat's characteristics from the photo, performs audio analysis of the meows, and analyzes the GPS data in real time. The server then matches the data with information from shelters and online sources to see if a similar cat is being cared for. At the same time, it predicts the cat's behavior based on the GPS data and determines the expected search area. These matching and behavior prediction results are notified to the device in real time, allowing the user to search for the cat efficiently. Furthermore, the emotion engine analyzes the user's emotions and provides messages and notifications according to the level of urgency, thereby reducing the user's anxiety and supporting the search process.

[0632] Prompt Sentence Examples

[0633] Please explain the specific process of how the server uses the cat photos, meows, and GPS data uploaded from the dedicated app.

[0634] In this way, the present invention combines AI technology, data matching technology, and an emotion engine to quickly and effectively locate and protect lost pets, and provide emotional support to pet owners.

[0635] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0636] Step 1: Data collection

[0637] Users use a dedicated application to upload photos, sounds, and GPS data of their pets.

[0638] Input: Pet photo (JPEG format), pet sound (WAV format), GPS data (JSON format)

[0639] Output: Collected data is temporarily stored on the device and ready to be sent.

[0640] Specific operation: The user takes a photo of the cat using a smartphone and saves it in the application. The user records the cat's meows and uses the smartphone's GPS function to collect location information.

[0641] Step 2: Send data

[0642] The terminal transmits the uploaded data from the user to a server via the Internet.

[0643] Input: Collected photos, calls, and GPS data

[0644] Output: Data is sent to the server.

[0645] How it works: The device uses Wi-Fi or mobile data to upload photos and bird call data to the server, and GPS data is also sent to the server at the same time.

[0646] Step 3: Save Data

[0647] The server stores the received data in an internal database.

[0648] Input: Photos, sounds, and GPS data sent to the server

[0649] Output: Photos, calls, and GPS data stored in a database

[0650] Specific operation: The server stores JPEG images in the "Pet Image" table, WAV audio data in the "Pet Audio" table, and JSON-formatted GPS data in the "Pet Location Information" table.

[0651] Step 4: Sentiment Data Analysis

[0652] The device analyzes the user's emotional state using a dedicated emotion engine and sends the data to a server.

[0653] Input: User's facial expression data

[0654] Output: Parsed emotional state data

[0655] Specific operation: The device uses an emotion engine to analyze the user's facial expressions and recognize emotional states such as "anxiety" or "impatience." The analysis results are sent to the server.

[0656] Step 5: Image analysis

[0657] The server uses an image analysis module to extract specific features from stored photos of pets.

[0658] Input: Saved pet photo

[0659] Output: Extracted feature data (shape, color, pattern, etc.)

[0660] How it works: The server uses an AI image analysis model (such as YOLO or ResNet) to scan the pet's image and extract features such as shape, color, and pattern. These feature data are stored in a "feature data" table.

[0661] Step 6: Audio analysis

[0662] The server uses an audio analysis module to analyze the stored bird call data.

[0663] Input: Stored call data

[0664] Output: Analyzed audio feature data (waveform, frequency characteristics, etc.)

[0665] Specific operation: The server uses an audio analysis module (such as MFCC or WaveNet) to analyze the waveform and frequency characteristics of the bird call data and generate audio feature data. This data is stored in the "audio feature data" table.

[0666] Step 7: GPS data analysis

[0667] The server analyzes the GPS data in real time to determine the location.

[0668] Input: Stored GPS data

[0669] Output: Parsed map information

[0670] What it does: The server connects the GPS data to the Google Maps API and visualizes the location information in real time. This information is stored in the "Map Data" table.

[0671] Step 8: Data Matching

[0672] The server matches the extracted features and the results of voice analysis with information from the shelter's database and online sources.

[0673] Input: Pet feature data, audio feature data

[0674] Output: Matching result data

[0675] Specific operation: The server accesses the shelter's database to search for pets with matching characteristics, and then compares them with online pet information to obtain matching results, which are then stored in a "matching results" table.

[0676] Step 9: Behavioral Prediction

[0677] The server uses GPS data and the pet's behavioral patterns to predict where the pet may move next.

[0678] Input: Stored GPS data, behavioral pattern data

[0679] Output: Predicted action location data

[0680] Specific operation: The server uses an AI behavior prediction model (LSTM or RNN model) based on past behavior data to predict the pet's movement pattern. The predicted location data is saved in the "Behavior prediction data" table.

[0681] Step 10: Emotional Data Adjustment

[0682] The server analyzes the user's emotional state and adjusts the search assistance based on its output.

[0683] Input: Parsed emotion data

[0684] Output: Reconciled notification data

[0685] Specific operation: The server adjusts the notification priority based on the analysis results of the emotion engine and generates messages that quickly provide the most important information to the user.

[0686] Step 11: Notification and Support

[0687] The server notifies the terminal of the matching results and behavior prediction results in real time and notifies the user.

[0688] Input: Matching result data, behavior prediction data

[0689] Output: Notification data to the device

[0690] Specific operation: The server compiles the matching results and behavior prediction results and sends them to the device as a push notification. The user can use this notification to continue searching for their pet. In addition, in the event of an emergency, the system can cooperate with a specialized pet rescue team to provide support.

[0691] (Application example 2)

[0692] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0693] Conventional food delivery systems have difficulty improving customer satisfaction and providing efficient services because they do not adequately consider users' emotional state and real-time feedback, which leads to problems such as user frustration and insufficient order optimization.

[0694] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means for uploading photos of missing pets, server means for receiving the uploaded photos and extracting features, server means for matching the photos with a database of shelters based on the features, server means including an emotion engine for analyzing the user's emotion data, and server means for adjusting notifications based on the emotion data. This makes it possible to grasp the user's emotional state in real time and provide optimal menus and notifications based on that.

[0695] "Terminal" refers to a device that allows a user to input information or send data to a server.

[0696] "Server" refers to a computer system that receives and stores uploaded data and performs various analyses.

[0697] "Features" refer to identifiable information or patterns extracted from image or audio data.

[0698] "Server means" refers to a combination of hardware and software that implements a particular function within a server.

[0699] An "emotion engine" refers to an artificial intelligence mechanism that analyzes the user's emotional state and outputs analytical results based on that data.

[0700] "Notification" refers to information or messages sent by the server to the user.

[0701] "Matching" refers to the process of searching for and matching relevant data and information based on characteristics and analysis results.

[0702] A "database" refers to a collection of information that stores specific information in an organized manner and is configured to facilitate search and retrieval.

[0703] In this invention, a system is constructed for analyzing the emotional state of a user and providing optimal menus and notifications in real time in the operation of a food delivery system. Specific embodiments of this system are described below.

[0704] Overall system description

[0705] A food delivery service application consists of the following main components:

[0706] Terminal means: A device such as a smartphone through which a user enters information and transmits the data to a server.

[0707] Server means: A computer system that receives, stores, and performs various analyses of data.

[0708] Emotion Engine: An AI module that analyzes the user's emotional state.

[0709] Notification vehicle: A mechanism for delivering information from a server to a user.

[0710] Data collection and storage

[0711] Users use a smartphone app to upload food photos, ratings, GPS data, and comments. This data is sent from the smartphone to a server, which stores the received data in an internal database. User emotion data is also collected through an emotion engine.

[0712] Data analysis and feature extraction

[0713] The server inputs the saved food photos into an AI analysis module to extract specific features (shape, color, presentation, etc.). It also performs sentiment analysis on uploaded comments using natural language processing technology. GPS data is analyzed in real time and converted into map information. The emotion engine analyzes the user's emotional state and combines that data with other data.

[0714] Data Matching

[0715] The server then matches the extracted features and sentiment analysis results with food delivery menus and store information, enabling users to quickly obtain menus and store information that meet their desired criteria.

[0716] Behavioral prediction

[0717] The server predicts users' behavior based on GPS data and their past order history. For example, it takes into account their preferences and rating history to predict what menu items they are likely to order next. This behavior prediction allows delivery companies and restaurants to efficiently manage and prepare inventory.

[0718] Emotional Engine Adjustment

[0719] The emotion engine analyzes the user's emotional state and tailors notifications based on its output. For example, if the user is feeling stressed, it may send relaxing menu recommendations or inspiring messages to reduce anxiety and stress.

[0720] Notifications and Support

[0721] Matching results and behavioral prediction results are sent to the user's smartphone in real time. Users can receive these notifications and place orders immediately. In addition, the analysis results of the emotion engine are reflected in the notifications, providing support that takes the user's emotions into consideration.

[0722] Specific examples

[0723] For example, if a user wants to eat delicious pasta, the process goes like this: First, the user uses a smartphone app to upload a photo of themselves along with a comment saying, "I want to eat delicious pasta." The server receives this data, analyzes the user's emotions from the photo, and analyzes the comment using natural language processing. Then, based on the extracted features, it matches relevant store and menu information and notifies the user's smartphone. The emotion engine analyzes the user's emotions and sends notifications recommending relaxing menu items, thereby reducing the user's anxiety and providing the optimal food delivery service.

[0724] Prompt Sentence Examples

[0725] "Implement an application that identifies emotions from a user's facial image, sends it to a server along with GPS data and comments, and then notifies a food delivery service of menu recommendations in real time based on that data."

[0726] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0727] Step 1:

[0728] Users use a smartphone app to upload photos of food, GPS data, and comments, along with a photo of their face, which the system receives as input. The uploaded data is then sent to a server via an internet connection.

[0729] Step 2:

[0730] The server stores the received data. The stored data includes the user's photos, GPS data, and comments. The server first sends the user's photos to the emotion engine and identifies emotions from the facial images. This emotion data is analyzed by the emotion engine's AI model.

[0731] Step 3:

[0732] The server receives the emotion data output by the emotion engine and sends it along with the comment data to a natural language processing (NLP) module. The NLP module analyzes the emotion data and comments to identify the user's current emotional state and the specific content of the comment. This data analysis clarifies the user's desires and emotional state.

[0733] Step 4:

[0734] Based on the analysis results, the server sends the stored food photo data to a feature extraction module, which extracts features such as shape, color, and presentation from the image data. This feature data is used to match menus and restaurant information.

[0735] Step 5:

[0736] The server searches internal databases and external related information based on the extracted feature data and emotion data to match the optimal menu and restaurant information. The matching results are selected based on the user's current emotional state and the content of their comments.

[0737] Step 6:

[0738] The server organizes the matching results and sends real-time notifications to the user's smartphone, including recommended menu items, restaurant information, and a customized message based on the results of the emotion engine's analysis.

[0739] Step 7:

[0740] The user receives a notification, reviews the information displayed in the application, and places an order if necessary, ensuring that the user selects the menu that best suits their emotional state and needs.

[0741] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0742] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0743] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0744] [Third embodiment]

[0745] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0746] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0747] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0748] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0749] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0750] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0751] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0752] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0753] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0754] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0755] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0756] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0757] Overall system description

[0758] This invention is an AI system that supports the rapid discovery and protection of missing pets. This system uses a server to receive and analyze photos, sounds, and GPS data of pets uploaded by owners using a terminal, and then identifies the pet's location and predicts its behavior by performing analysis and matching.

[0759] Data collection and storage

[0760] Users use a dedicated application to upload photos, sounds, and GPS data of their pets, which are then sent from the device to a server, which then stores the data in an internal database.

[0761] Data analysis and feature extraction

[0762] The server inputs the saved pet photos into the AI ​​model to extract specific features (shape, color, etc.). It also performs audio analysis on the uploaded pet's cries to extract their waveform and frequency characteristics. GPS data is analyzed in real time to capture location information and generate a map.

[0763] Data Matching

[0764] The server then matches the extracted features and the results of the voice analysis with information from shelter databases and online sources. The shelter database contains information registered when a missing pet is rescued. This allows the server to quickly notify owners if the same or similar pet is found.

[0765] Behavioral prediction

[0766] Furthermore, the server predicts pet behavior based on GPS data, taking into account the pet's breed, age, personality, surrounding terrain, etc., to estimate where the pet may head next. This behavior prediction helps to efficiently narrow the search area.

[0767] Notification and Discovery Assistance

[0768] Matching results and behavior prediction results are sent to the device in real time. Users can receive this notification and begin searching for their pet based on the provided information. If necessary, the system can also connect with a specialized pet rescue team to provide optimal support.

[0769] Specific examples

[0770] For example, if a user were to search for a missing dog, the process would be as follows: First, the user would use a dedicated application to upload the dog's photo, bark, and GPS data. The server receives this data, extracts the dog's characteristics from the photo, performs audio analysis of the bark, and analyzes the GPS data in real time. The server then matches the data with information from shelters and online to see if a similar dog is being cared for. At the same time, it predicts the dog's behavior based on the GPS data and determines the expected search area. The matching and behavior prediction results are notified to the device in real time, allowing the user to check them and search for the dog efficiently.

[0771] In this way, the present invention is a system that combines AI technology and data matching technology to enable the quick and effective discovery and protection of lost pets.

[0772] The processing flow will be explained below.

[0773] Step 1:

[0774] Users launch a dedicated application, capture photos of their missing pet, record its cries, and collect GPS data, which are then uploaded to a server via the application.

[0775] Step 2:

[0776] The server receives the uploaded photos, bird calls, and GPS data and stores them in a designated database, where they are prepared for analysis.

[0777] Step 3:

[0778] The server inputs the stored pet photos into an AI analysis module and uses image recognition technology to extract the pet's characteristics, such as fur color, body shape, and pattern.

[0779] Step 4:

[0780] The server inputs the saved bark data into a voice analysis module and extracts the characteristics of the pet's voice by analyzing the frequency characteristics and waveform patterns.

[0781] Step 5:

[0782] The server analyzes the GPS data in real time and plots the pet's current location on a map. The location information is used by the behavior prediction module.

[0783] Step 6:

[0784] The server compares the extracted photo characteristics and audio analysis results with information from shelter databases and online sources, and retrieves information about pets with similar characteristics.

[0785] Step 7:

[0786] The server predicts the pet's next move based on GPS data and its behavioral patterns, taking into account the pet's type, age, personality, and surrounding terrain information, for example.

[0787] Step 8:

[0788] The server generates a map of the search area based on the matching results and behavior prediction results and notifies the terminal in real time.

[0789] Step 9:

[0790] The device displays the notified information to the user, allowing the user to check the provided search area and pet characteristics. The user can then quickly start searching based on this information.

[0791] Step 10:

[0792] Users follow the instructions in the app to search for their pet within the provided search area, and if necessary, the app will connect them with a specialized pet rescue team to provide optimal support to help them find their pet.

[0793] Example 1

[0794] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0795] To quickly find and protect lost pets, it is necessary to accurately analyze various data such as photos, sounds, and GPS data, and efficiently search for matching information. It is also necessary to provide clues for the search by predicting the pet's current location and behavior.

[0796] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0797] In this invention, the server includes an information processing terminal means for uploading data on missing pets, an information processing device means for receiving the uploaded data and extracting features, an information processing device means for comparing the features and analysis results with records of the shelter, an information processing device means for predicting behavior based on the data, and an information processing terminal means for notifying the analysis results and prediction results, thereby enabling the rapid identification and behavior prediction of missing pets and further real-time notification.

[0798] The "information processing terminal means" is a device for collecting data about missing pets and transmitting it to an information processing device.

[0799] The "information processing device means" is a device for analyzing received data, extracting and collating necessary information, and predicting behavior.

[0800] The "location information acquisition device means" refers to a device for acquiring coordinate data of the pet's current location, such as a GPS device.

[0801] "Feature extraction" is the process of extracting specific attributes, such as shape or waveform, from uploaded photos or audio data.

[0802] "Audio analysis" is the process of analyzing uploaded bird call data and extracting its waveform, frequency characteristics, etc.

[0803] "Data matching" is the process of checking whether the extracted characteristics match the shelter's records or online information.

[0804] "Behavior prediction" is the process of estimating where your pet will go next, taking into account the pet's breed, age, personality, terrain, GPS data, and more.

[0805] "Real-time notification" is a notification function that instantly notifies users of analysis results and behavior prediction results.

[0806] System Overview

[0807] The present invention is an AI system designed to support the rapid discovery and protection of missing pets. This system is composed of an information processing terminal means, an information processing device means, and a location information acquisition device means. The information processing terminal means is a device that collects and transmits photos, audio, GPS data, etc. of pets, and the information processing device means analyzes, collates, and predicts the received data and transmits the results. The location information acquisition device means is a device that acquires pet location information in real time.

[0808] Hardware and software used

[0809] The server uses a computer with high-performance data processing capabilities. Deep learning frameworks such as TensorFlow and PyTorch are used to analyze the photo data. Audio analysis tools such as Librosa and WaveSurfer are used to analyze the audio data. Geographic information systems such as PostGIS are used to process GPS data. An SQL-based database system is used for database management.

[0810] Data collection and transmission

[0811] Users use a dedicated application to collect and upload data about their missing pets. This application runs on mobile platforms such as iOS and Android. For example, by entering data such as a photo of the missing dog, the dog's barking sound, and the current GPS location, users can easily send the data to the server by pressing the send button.

[0812] Data analysis and feature extraction

[0813] The server receives the uploaded data and performs the following analysis on each data. Photo data is input into a TensorFlow model to extract features such as the pet's shape and color. Audio data is analyzed using Librosa to extract waveforms and frequency characteristics. GPS data is processed using PostGIS and plotted on a map.

[0814] Data collation and behavior prediction

[0815] The analyzed feature data is compared with the shelter's database and data on the internet, for example by running a query such as "SELECT FROM shelter_pets WHERE shape = 'dog' AND color = 'brown'." The server then uses machine learning models such as random forests and neural networks to predict the pet's behavior and estimate its next likely destination.

[0816] Notification and Discovery Assistance

[0817] The server notifies the device of the matching results and behavior prediction results in real time. This notification is provided to the user in the form of a push notification or an in-app notification. For example, specific information such as "A similar pet has been found at shelter A" or "The next likely destination is park B" is provided.

[0818] Specific examples

[0819] For example, if a user is searching for a missing dog, they use a dedicated application to upload the following data: a photo of the dog (e.g., Fido_missing.jpg), the dog's barks (e.g., Fido_bark.wav), and current GPS data (e.g., GPS_location.csv), which are then sent directly to the server. The server receives this data and analyzes and extracts features using TensorFlow, Librosa, PostGIS, and other software. The results are compared with the shelter's database and information on the Internet to predict behavior. Finally, the results of the comparison and predicted behavior are sent to the user's device in real time, allowing for efficient searching for the pet.

[0820] The system combines AI and data analysis technologies to enable the fast and effective discovery and protection of lost pets.

[0821] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0822] Step 1: Upload your data

[0823] A user uses a dedicated application to collect photos, sounds, and GPS data related to a missing pet, and then sends this data to a server from an information processing device. For example, a user presses the "Add a new pet" button in the application, takes a photo of the pet, and uploads it. They also record the sound of the pet's sounds and upload that file. Next, the device's GPS function is used to obtain the pet's current location, which is also sent. The input data consists of a photo file (e.g., Fido_missing.jpg), an audio file (e.g., Fido_sound.wav), and GPS text data (e.g., GPS_location.csv). This data is sent to the server.

[0824] Step 2: Receiving and storing data

[0825] The server receives the data sent from the device and checks the format and consistency of each piece of data. After checking, it stores the received photo data in the "photos" table, the bird sound data in the "audio" table, and the GPS data in the "gps_logs" table. Specifically, the server stores each piece of data in temporary storage, verifies whether it is in the correct data format, and then inserts it into the corresponding database table. At this time, metadata (e.g., data timestamp, user ID) is also saved. If the data is successfully saved in the database, the server generates a success message and proceeds to the next step.

[0826] Step 3: Data analysis and feature extraction

[0827] The server analyzes the received and stored data and extracts its features. The photo data is input into a TensorFlow model to extract features such as the pet's shape and color. The input data is the stored photo file, and the output data is the extracted feature vector. Similarly, Librosa is used to analyze the sound waveform and frequency characteristics of the pet's bark data and extract a feature vector. The GPS data is plotted on a map based on the coordinate information, and the location information is confirmed. Specifically, the server feeds the photo data into a deep learning model and extracts features using a convolutional neural network (CNN). The analysis results are saved as a feature vector. Similarly, audio features such as power spectrograms and Mel-Frequency Cepstrum (MFCC) are extracted from the audio data and saved as a feature vector. PostGIS is used to convert the GPS data into geographic coordinates and plot them on a map.

[0828] Step 4: Collating the data

[0829] The server uses the extracted feature vectors to match the search results with the shelter's database and information on the internet. This process involves generating an SQL-based query to perform database matching, for example, "SELECT FROM shelter_pets WHERE shape = 'dog' AND color = 'brown'". The input data is the extracted feature vectors, and the output data is a list of matching database records. After the matching process, the server passes the details of any matching records to the next step.

[0830] Step 5: Predict behavior

[0831] The server comprehensively evaluates information such as the pet's species, age, personality, and GPS data to predict its behavior. It then uses machine learning algorithms such as random forests and neural networks to estimate the pet's next likely destination. The input data is the pet's characteristics and current GPS data, and the output data is the predicted range of movement and the next destination. Specifically, the server inputs known behavioral pattern data and current location information into a machine learning model to obtain a prediction result. For example, it generates a prediction result in the form of "The most likely next destination is a nearby park."

[0832] Step 6: Notification

[0833] The server notifies the user of the matching results and behavior prediction results in real time via push notifications or in-app notifications. The input data are the matching results and behavior prediction results obtained in the previous step, and the output data is the notification message. For example, messages such as "A similar pet is at shelter A" or "The next likely destination is park B" are sent to the user's device.

[0834] Step 7: Search support

[0835] The user receives the notification and quickly begins searching for the pet based on the provided information. For example, by heading to Park B based on the notification, the search can be carried out efficiently. If necessary, the system can also cooperate with a pet rescue team to carry out more effective search activities. When searching, the device's map function and GPS can be used to confirm the specific search location. This makes it possible to quickly find and protect the missing pet.

[0836] (Application example 1)

[0837] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0838] There is a lack of methods to quickly and efficiently find and protect missing animals. There is also a need for a system that can predict behavior in real time based on information provided by owners, thereby increasing the probability of finding animals.

[0839] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0840] In this invention, the server includes a terminal means for uploading photos of missing animals, an information processing means for receiving the uploaded photos and extracting features, an information processing means for performing information matching with an information storage means of a shelter based on the features, an information processing means for predicting behavior based on the results of the information matching, and a terminal means for notifying the results of the behavior prediction in real time, thereby enabling the efficient discovery and protection of missing animals.

[0841] A "missing animal" is a domestic animal that has become separated from its owner and whose whereabouts cannot be confirmed.

[0842] "Terminal means" refers to devices or equipment that can be operated by users, and specifically includes smartphones, tablets, and personal computers.

[0843] "Information processing means" refers to a device or system that analyzes and processes received data, and specifically includes a server or computer system.

[0844] "Photo feature extraction" is the process of using image processing technology to analyze features such as shape and color from uploaded photos and extract them as data.

[0845] A "shelter" is a facility where lost animals are rescued and temporarily housed, including animal shelters and rescue centers.

[0846] "Information storage means" refers to a device or system that stores and saves information in the form of a database or the like.

[0847] "Information matching" is the process of comparing and matching extracted data with other data sets.

[0848] "Behavioral prediction" is the process of predicting the future behavior and destinations of animals based on past data and current conditions.

[0849] "Notification means" refers to the means of communication used to inform users of analysis and prediction results, and specifically includes push notifications, email, SMS, etc.

[0850] The present invention is a comprehensive AI system for quickly locating and protecting missing animals. This system begins when a user uploads data about the missing animal (photos, sounds, GPS information) using a terminal. Specific embodiments of this system are described below.

[0851] Overall system description

[0852] The system mainly consists of the following components:

[0853] Terminal means

[0854] Server Means

[0855] Information Processing Means

[0856] Information storage means

[0857] Information matching method

[0858] Behavioral prediction methods

[0859] Notification means

[0860] Data collection

[0861] Users use a dedicated smartphone application to upload photos, sounds, and GPS data of missing animals from the terminal means, and all of this data is sent to the server means.

[0862] Data analysis and feature extraction

[0863] The server analyzes the received photo data using image analysis software called OpenCV to extract specific features (such as the animal's shape and body color). Similarly, audio analysis using Librosa analyzes the call data for characteristics such as frequency characteristics and sound spectrum. Additionally, geopy is used to analyze GPS data to determine the animal's current location and movement patterns.

[0864] Data matching and behavior prediction

[0865] The server compares the analyzed features with the shelter's database and information on the Internet. If the same or similar animal is found, the server extracts its information. Furthermore, the server predicts the animal's behavior based on the analyzed GPS data. This predicts the animal's next likely destination.

[0866] Notification and Discovery Assistance

[0867] The matching results and behavior prediction results are sent to the user's terminal in real time. Users can use this information to efficiently search for animals. If necessary, they can also work with a specialized rescue team to carry out search activities.

[0868] Specific examples

[0869] For example, if a user were to search for a missing dog, the process would go something like this: First, the user would use a dedicated application to upload a photo of the dog, its barks, and GPS data showing its recent movement route. The server receives the data, extracts the dog's characteristics from the photo, performs audio analysis of the barks, and analyzes the GPS data. The server then matches the data with information from shelters and online sources to see if a similar dog is being cared for. At the same time, it predicts the dog's behavior based on the GPS data and determines the expected search area. These results are then sent to the user's device in real time, allowing the user to efficiently search for the dog.

[0870] Prompt Sentence Examples

[0871] I have lost my dog ​​and would like some help finding him. I am providing the following information:

[0872] Photo: <Attached image file>

[0873] Sounds: <audio file>

[0874] GPS information: Latitude: 35.6895, Longitude: 139.6917

[0875] Please provide a location prediction, including recent movement patterns from the starting point.

[0876] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0877] Step 1: Upload your data

[0878] Description: A user uses a smartphone application to upload photos, sounds, and GPS data about missing animals.

[0879] Input: Photo files, audio files, GPS data (latitude and longitude)

[0880] Output: Photos, audio data, and GPS data uploaded to the server

[0881] Specific operation: The user accesses the input form within the app, selects and attaches each data file, and then presses the send button. This action sends the data from the device to the server.

[0882] Step 2: Save your data

[0883] Description: The server stores the received data in a database.

[0884] Input: Uploaded photos, audio data, GPS data

[0885] Output: Photos, audio data, and GPS data stored in the database

[0886] What happens: The server stores the received data in temporary storage and creates the appropriate database entries to store each file.

[0887] Step 3: Extracting image features

[0888] Description: The server extracts specific features from the stored photo data using OpenCV.

[0889] Input: Saved photo data

[0890] Output: Image feature data (specific patterns such as shape, color, etc.)

[0891] Specific operation: The server converts the image to grayscale using the OpenCV library, and detects key points and extracts feature vectors using the SIFT (Scale-Invariant Feature Transform) algorithm.

[0892] Step 4: Extract audio features

[0893] Description: The server analyzes the stored audio data using Librosa to extract certain audio features.

[0894] Input: Stored audio data

[0895] Output: Audio feature data (frequency characteristics, spectrum, etc.)

[0896] Specific operation: The server loads the audio data using the Librosa library and calculates MFCCs (Mel-Frequency Cepstrum Coefficients) to extract audio features.

[0897] Step 5: Analyzing the GPS data

[0898] Description: The server analyzes stored GPS data to determine the current location and movement patterns of animals.

[0899] Input: Stored GPS data

[0900] Output: Analyzed location information and movement patterns

[0901] What it does: The server uses the geopy library to analyze GPS data, identify movement patterns from past history, and predict possible next destinations.

[0902] Step 6: Data collation

[0903] Description: The server compares the extracted feature data with the shelter's database and online information.

[0904] Input: Image feature data, audio feature data, GPS analysis data

[0905] Output: Matched animal information

[0906] How it works: The server accesses shelter and online databases, applies a comparison and matching algorithm to the extracted feature data, calculates a similarity score, and identifies matching animals.

[0907] Step 7: Predict behavior

[0908] Description: The server makes animal behavior predictions based on the analyzed GPS data.

[0909] Input: Analyzed location and movement patterns

[0910] Output: Predictions of where the animal might go next

[0911] How it works: The server uses past movement data and current location information to apply an algorithm to predict where the animal will go next, along with other data such as the animal's species, age, and surrounding environment.

[0912] Step 8: Notification

[0913] Description: The server notifies the user of the matching results and behavior prediction results to the user's terminal means.

[0914] Input: Matched animal information, behavior prediction information

[0915] Output: A notification message that is displayed on the user's terminal.

[0916] Specific operation: The server compiles the matching results and behavior prediction information and sends them to the user's device via push notification, email, SMS, etc. This allows the user to know where the animal was found and potential locations to find it next.

[0917] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0918] Overall system description

[0919] This invention is an AI system that supports the rapid discovery and protection of lost pets, and improves the user experience by combining it with an emotion engine that recognizes the user's emotions. In this system, a server receives photos, sounds, GPS data, etc. of pets uploaded by owners using a terminal, analyzes and matches them, and adjusts search support based on the user's emotions recognized by the emotion engine.

[0920] Data collection and storage

[0921] Users use a dedicated application to upload photos, sounds, and GPS data of their pets. This data is sent from the device to a server, which then stores the received data in an internal database. In addition, the user's emotional data is also collected through an emotion engine.

[0922] Data analysis and feature extraction

[0923] The server inputs the saved pet photos into an AI analysis module to extract specific features (shape, color, etc.). The audio analysis module also analyzes the uploaded pet's cries for waveform and frequency characteristics. GPS data is analyzed in real time and displayed as a map. An emotion engine analyzes the user's emotional state and combines that data with other data.

[0924] Data Matching

[0925] The server then matches the extracted features and the results of the voice analysis with information from shelter databases and online sources. The shelter database contains information registered when a missing pet is rescued. If an identical or similar pet is found, the server can quickly notify the owner.

[0926] Behavioral prediction

[0927] The server predicts the pet's next likely location based on GPS data and its movement patterns. For example, it takes into account the pet's type, age, personality, and surrounding terrain information. This behavior prediction effectively narrows the search area.

[0928] Emotional Engine Adjustment

[0929] The emotion engine analyzes the user's emotional state and adjusts its search assistance based on its output. For example, if the user is extremely anxious, it will change the priority of notifications and provide urgent information immediately. It also provides customized messages and advice based on the user's emotions, reducing anxiety and stress.

[0930] Notification and Discovery Assistance

[0931] Matching results and behavior prediction results are sent to the device in real time. The user receives this notification and begins searching for a pet based on the provided information. If necessary, the system can also work with a specialized pet rescue team to provide optimal support. The analysis results of the emotion engine are also reflected in the notification, and support is provided according to the user's feelings.

[0932] Specific examples

[0933] For example, if a user were searching for a missing cat, the process would be as follows: First, the user would use a dedicated application to upload the cat's photo, meows, and GPS data. The server receives this data, extracts the cat's characteristics from the photo, performs audio analysis of the meows, and analyzes the GPS data in real time. The server then matches the data with information from shelters and online sources to see if a similar cat is being cared for. At the same time, it predicts the cat's behavior based on the GPS data and determines the expected search area. The matching and predicted behavior results are notified to the device in real time, allowing the user to check them and search for the cat efficiently. Furthermore, the emotion engine analyzes the user's emotions and provides messages and notifications according to the level of urgency, thereby reducing the user's anxiety and supporting the search process.

[0934] In this way, the present invention is a system that combines AI technology, data matching technology, and an emotion engine to quickly and effectively find and protect lost pets, while also providing support that takes into consideration the emotions of pet owners.

[0935] The processing flow will be explained below.

[0936] Step 1:

[0937] Users launch a dedicated application, capture photos of their missing pet, record its cries, and collect GPS data, which are then uploaded to a server via the application.

[0938] Step 2:

[0939] To collect the user's emotional state, the device uses an emotion engine to analyze the user's facial expressions, tone of voice, input text, etc. The analyzed emotional data is also sent to the server.

[0940] Step 3:

[0941] The server receives the uploaded pet photos, sounds, and GPS data and stores them in a designated database, as well as the received emotion data.

[0942] Step 4:

[0943] The server inputs the stored pet photos into an AI analysis module and uses image recognition technology to extract the pet's characteristics, such as fur color, body shape, and pattern.

[0944] Step 5:

[0945] The server inputs the saved bark data into a voice analysis module and extracts the characteristics of the pet's voice by analyzing the frequency characteristics and waveform patterns.

[0946] Step 6:

[0947] The server analyzes the GPS data in real time and plots the pet's current location on a map. The location information is used by the behavior prediction module.

[0948] Step 7:

[0949] The server compares the extracted photo characteristics and audio analysis results with information from shelter databases and online sources, and retrieves information about pets with similar characteristics.

[0950] Step 8:

[0951] The server predicts the pet's next likely location based on GPS data and its movement patterns, taking into account factors such as the pet's type, age, personality, and surrounding terrain.

[0952] Step 9:

[0953] The server generates a map of the search area based on the matching results and behavior prediction results, and notifies the device in real time. The content and priority of notifications are adjusted based on the output of the emotion engine.

[0954] Step 10:

[0955] The device displays the notified information to the user, allowing the user to check the provided search area and pet characteristics. The user can then quickly start searching based on this information.

[0956] Step 11:

[0957] Users follow the instructions in the app to search for their pet within the provided search area, and if necessary, the app will connect them with a specialized pet rescue team to provide optimal support to help them find their pet.

[0958] Step 12:

[0959] The server collects emotional data in real time as the user explores, analyzes the user's stress level and anxiety using an emotion engine, and provides customized messages and advice to the user via their device based on the analysis results.

[0960] Step 13:

[0961] The device displays customized messages and advice provided by the emotion engine to the user to support their exploration activities. If necessary, it re-analyzes the user's emotional state and makes further adjustments.

[0962] This allows for quick and effective detection and protection of pets, reducing the mental burden on owners.

[0963] Example 2

[0964] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0965] Conventional technologies have not been sufficient to quickly and effectively locate and protect lost pets. Furthermore, there is a lack of support that takes into account the owner's emotions. Therefore, there is a need for methods to improve the efficiency of searches and the user experience. In particular, there has been a challenge in developing a method to effectively locate pets using their appearance, sounds, and location information.

[0966] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0967] In this invention, the server includes a terminal means for uploading photos, sounds, and GPS data related to the missing pet, a server means for receiving the uploaded photos, sounds, and GPS data and extracting features including data derived from emotions, a server means for matching the features with data on shelters and information on the Internet, and an emotion engine means for analyzing the user's emotion data and adjusting the priority of notifications based on the output. This makes it possible to quickly and effectively find and protect missing pets and provide support that takes the user's emotions into consideration.

[0968] "Terminal means" refers to a device or application that allows a user to upload information about a missing pet.

[0969] "Server means" means a computer system for processing, storing and analyzing data received from the terminal.

[0970] The "emotion engine means" is a software module that analyzes the user's emotional state and adjusts the system's operation based on the results.

[0971] "Features" are identifying information such as shape, color, and sound characteristics extracted from photos and sounds of pets.

[0972] "Matching" is the process of comparing the extracted features with data from shelters and online sources.

[0973] "Shelter" means a placement facility where lost pets may be taken.

[0974] "GPS data" is geographic information that indicates your pet's current location and last known location.

[0975] "Audio analysis means" is a technology for analyzing uploaded bird call data and extracting its characteristics.

[0976] "Data analysis means" refers to technology for predicting pet behavior based on GPS data and past behavioral patterns.

[0977] "Notification means" is a system function for notifying users of matching results and behavior prediction results.

[0978] MODE FOR CARRYING OUT THE INVENTION

[0979] This invention is an AI system that supports the rapid discovery and protection of lost pets, and improves the user experience by combining it with an emotion engine that recognizes the user's emotions. In this system, the server receives, analyzes, and matches photos, sounds, and GPS data of pets uploaded by owners using their devices, and adjusts search support based on the user's emotions recognized by the emotion engine.

[0980] Terminal means:

[0981] Users use a dedicated application to upload photos of their pets, their pet's cries, and GPS data. This data is sent from the device to the server. Photos taken by the user are sent in JPEG format, recorded pet cries are sent in WAV format, and GPS data is sent in JSON format. In addition, the device captures the user's facial expressions with its camera and sends the analysis results to the server via an emotion engine.

[0982] Server means:

[0983] The server stores the received data in an internal database. The stored pet photos are input into an AI image analysis module (e.g., YOLO or ResNet) to extract specific features (shape, color, pattern, etc.). An audio analysis module (e.g., MFCC or WaveNet) is used to analyze the waveform and frequency characteristics of the pet's bark data. The server also analyzes GPS data in conjunction with a map API (e.g., Google Maps API) to determine location information in real time. The emotion engine analyzes the user's emotional state and stores the data on the server.

[0984] Feature data matching:

[0985] The server then matches the extracted features and the results of the voice analysis with information from shelter databases and on the Internet. The shelter database contains information about lost pets that are registered when they are rescued. The server compares this data and promptly notifies the user if the same or similar pet is found.

[0986] Behavioral prediction and emotional data adjustment:

[0987] The server predicts the pet's next likely location based on GPS data and its behavioral patterns. This prediction is performed using an AI behavioral prediction model (e.g., LSTM or RNN model). It also adjusts the priority of notification messages based on the analysis results of the emotion engine. It generates customized messages according to specific emotional states and provides them to the user.

[0988] Notifications and Support:

[0989] Matching results and behavior prediction results are sent to the device in real time. Users can receive this notification and begin searching for their pet based on the provided information. Furthermore, if necessary, they can work with a specialized pet rescue team to provide optimal support.

[0990] Specific examples

[0991] For example, if a user were searching for a missing cat, the process would be as follows: First, the user would use a dedicated application to upload the cat's photo, meows, and GPS data. The server receives this data, extracts the cat's characteristics from the photo, performs audio analysis of the meows, and analyzes the GPS data in real time. The server then matches the data with information from shelters and online sources to see if a similar cat is being cared for. At the same time, it predicts the cat's behavior based on the GPS data and determines the expected search area. These matching and behavior prediction results are notified to the device in real time, allowing the user to search for the cat efficiently. Furthermore, the emotion engine analyzes the user's emotions and provides messages and notifications according to the level of urgency, thereby reducing the user's anxiety and supporting the search process.

[0992] Prompt Sentence Examples

[0993] Please explain the specific process of how the server uses the cat photos, meows, and GPS data uploaded from the dedicated app.

[0994] In this way, the present invention combines AI technology, data matching technology, and an emotion engine to quickly and effectively locate and protect lost pets, and provide emotional support to pet owners.

[0995] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0996] Step 1: Data collection

[0997] Users use a dedicated application to upload photos, sounds, and GPS data of their pets.

[0998] Input: Pet photo (JPEG format), pet sound (WAV format), GPS data (JSON format)

[0999] Output: Collected data is temporarily stored on the device and ready to be sent.

[1000] Specific operation: The user takes a photo of the cat using a smartphone and saves it in the application. The user records the cat's meows and uses the smartphone's GPS function to collect location information.

[1001] Step 2: Send data

[1002] The terminal transmits the uploaded data from the user to a server via the Internet.

[1003] Input: Collected photos, calls, and GPS data

[1004] Output: Data is sent to the server.

[1005] How it works: The device uses Wi-Fi or mobile data to upload photos and bird call data to the server, and GPS data is also sent to the server at the same time.

[1006] Step 3: Save Data

[1007] The server stores the received data in an internal database.

[1008] Input: Photos, sounds, and GPS data sent to the server

[1009] Output: Photos, calls, and GPS data stored in a database

[1010] Specific operation: The server stores JPEG images in the "Pet Image" table, WAV audio data in the "Pet Audio" table, and JSON-formatted GPS data in the "Pet Location Information" table.

[1011] Step 4: Sentiment Data Analysis

[1012] The device analyzes the user's emotional state using a dedicated emotion engine and sends the data to a server.

[1013] Input: User's facial expression data

[1014] Output: Parsed emotional state data

[1015] Specific operation: The device uses an emotion engine to analyze the user's facial expressions and recognize emotional states such as "anxiety" or "impatience." The analysis results are sent to the server.

[1016] Step 5: Image analysis

[1017] The server uses an image analysis module to extract specific features from stored photos of pets.

[1018] Input: Saved pet photo

[1019] Output: Extracted feature data (shape, color, pattern, etc.)

[1020] How it works: The server uses an AI image analysis model (such as YOLO or ResNet) to scan the pet's image and extract features such as shape, color, and pattern. These feature data are stored in a "feature data" table.

[1021] Step 6: Audio analysis

[1022] The server uses an audio analysis module to analyze the stored bird call data.

[1023] Input: Stored call data

[1024] Output: Analyzed audio feature data (waveform, frequency characteristics, etc.)

[1025] Specific operation: The server uses an audio analysis module (such as MFCC or WaveNet) to analyze the waveform and frequency characteristics of the bird call data and generate audio feature data. This data is stored in the "audio feature data" table.

[1026] Step 7: GPS data analysis

[1027] The server analyzes the GPS data in real time to determine the location.

[1028] Input: Stored GPS data

[1029] Output: Parsed map information

[1030] What it does: The server connects the GPS data to the Google Maps API and visualizes the location information in real time. This information is stored in the "Map Data" table.

[1031] Step 8: Data Matching

[1032] The server matches the extracted features and the results of voice analysis with information from the shelter's database and online sources.

[1033] Input: Pet feature data, audio feature data

[1034] Output: Matching result data

[1035] Specific operation: The server accesses the shelter's database to search for pets with matching characteristics, and then compares them with online pet information to obtain matching results, which are then stored in a "matching results" table.

[1036] Step 9: Behavioral Prediction

[1037] The server uses GPS data and the pet's behavioral patterns to predict where the pet may move next.

[1038] Input: Stored GPS data, behavioral pattern data

[1039] Output: Predicted action location data

[1040] Specific operation: The server uses an AI behavior prediction model (LSTM or RNN model) based on past behavior data to predict the pet's movement pattern. The predicted location data is saved in the "Behavior prediction data" table.

[1041] Step 10: Emotional Data Adjustment

[1042] The server analyzes the user's emotional state and adjusts the search assistance based on its output.

[1043] Input: Parsed emotion data

[1044] Output: Reconciled notification data

[1045] Specific operation: The server adjusts the notification priority based on the analysis results of the emotion engine and generates messages that quickly provide the most important information to the user.

[1046] Step 11: Notification and Support

[1047] The server notifies the terminal of the matching results and behavior prediction results in real time and notifies the user.

[1048] Input: Matching result data, behavior prediction data

[1049] Output: Notification data to the device

[1050] Specific operation: The server compiles the matching results and behavior prediction results and sends them to the device as a push notification. The user can use this notification to continue searching for their pet. In addition, in the event of an emergency, the system can cooperate with a specialized pet rescue team to provide support.

[1051] (Application example 2)

[1052] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1053] Conventional food delivery systems have difficulty improving customer satisfaction and providing efficient services because they do not adequately consider users' emotional state and real-time feedback, which leads to problems such as user frustration and insufficient order optimization.

[1054] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means for uploading photos of missing pets, server means for receiving the uploaded photos and extracting features, server means for matching the photos with a database of shelters based on the features, server means including an emotion engine for analyzing the user's emotion data, and server means for adjusting notifications based on the emotion data. This makes it possible to grasp the user's emotional state in real time and provide optimal menus and notifications based on that.

[1055] "Terminal" refers to a device that allows a user to input information or send data to a server.

[1056] "Server" refers to a computer system that receives and stores uploaded data and performs various analyses.

[1057] "Features" refer to identifiable information or patterns extracted from image or audio data.

[1058] "Server means" refers to a combination of hardware and software that implements a particular function within a server.

[1059] An "emotion engine" refers to an artificial intelligence mechanism that analyzes the user's emotional state and outputs analytical results based on that data.

[1060] "Notification" refers to information or messages sent by the server to the user.

[1061] "Matching" refers to the process of searching for and matching relevant data and information based on characteristics and analysis results.

[1062] A "database" refers to a collection of information that stores specific information in an organized manner and is configured to facilitate search and retrieval.

[1063] In this invention, a system is constructed for analyzing the emotional state of a user and providing optimal menus and notifications in real time in the operation of a food delivery system. Specific embodiments of this system are described below.

[1064] Overall system description

[1065] A food delivery service application consists of the following main components:

[1066] Terminal means: A device such as a smartphone through which a user enters information and transmits the data to a server.

[1067] Server means: A computer system that receives, stores, and performs various analyses of data.

[1068] Emotion Engine: An AI module that analyzes the user's emotional state.

[1069] Notification vehicle: A mechanism for delivering information from a server to a user.

[1070] Data collection and storage

[1071] Users use a smartphone app to upload food photos, ratings, GPS data, and comments. This data is sent from the smartphone to a server, which stores the received data in an internal database. User emotion data is also collected through an emotion engine.

[1072] Data analysis and feature extraction

[1073] The server inputs the saved food photos into an AI analysis module to extract specific features (shape, color, presentation, etc.). It also performs sentiment analysis on uploaded comments using natural language processing technology. GPS data is analyzed in real time and converted into map information. The emotion engine analyzes the user's emotional state and combines that data with other data.

[1074] Data Matching

[1075] The server then matches the extracted features and sentiment analysis results with food delivery menus and store information, enabling users to quickly obtain menus and store information that meet their desired criteria.

[1076] Behavioral prediction

[1077] The server predicts users' behavior based on GPS data and their past order history. For example, it takes into account their preferences and rating history to predict what menu items they are likely to order next. This behavior prediction allows delivery companies and restaurants to efficiently manage and prepare inventory.

[1078] Emotional Engine Adjustment

[1079] The emotion engine analyzes the user's emotional state and tailors notifications based on its output. For example, if the user is feeling stressed, it may send relaxing menu recommendations or inspiring messages to reduce anxiety and stress.

[1080] Notifications and Support

[1081] Matching results and behavioral prediction results are sent to the user's smartphone in real time. Users can receive these notifications and place orders immediately. In addition, the analysis results of the emotion engine are reflected in the notifications, providing support that takes the user's emotions into consideration.

[1082] Specific examples

[1083] For example, if a user wants to eat delicious pasta, the process goes like this: First, the user uses a smartphone app to upload a photo of themselves along with a comment saying, "I want to eat delicious pasta." The server receives this data, analyzes the user's emotions from the photo, and analyzes the comment using natural language processing. Then, based on the extracted features, it matches relevant store and menu information and notifies the user's smartphone. The emotion engine analyzes the user's emotions and sends notifications recommending relaxing menu items, thereby reducing the user's anxiety and providing the optimal food delivery service.

[1084] Prompt Sentence Examples

[1085] "Implement an application that identifies emotions from a user's facial image, sends it to a server along with GPS data and comments, and then notifies a food delivery service of menu recommendations in real time based on that data."

[1086] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1087] Step 1:

[1088] Users use a smartphone app to upload photos of food, GPS data, and comments, along with a photo of their face, which the system receives as input. The uploaded data is then sent to a server via an internet connection.

[1089] Step 2:

[1090] The server stores the received data. The stored data includes the user's photos, GPS data, and comments. The server first sends the user's photos to the emotion engine and identifies emotions from the facial images. This emotion data is analyzed by the emotion engine's AI model.

[1091] Step 3:

[1092] The server receives the emotion data output by the emotion engine and sends it along with the comment data to a natural language processing (NLP) module. The NLP module analyzes the emotion data and comments to identify the user's current emotional state and the specific content of the comment. This data analysis clarifies the user's desires and emotional state.

[1093] Step 4:

[1094] Based on the analysis results, the server sends the stored food photo data to a feature extraction module, which extracts features such as shape, color, and presentation from the image data. This feature data is used to match menus and restaurant information.

[1095] Step 5:

[1096] The server searches internal databases and external related information based on the extracted feature data and emotion data to match the optimal menu and restaurant information. The matching results are selected based on the user's current emotional state and the content of their comments.

[1097] Step 6:

[1098] The server organizes the matching results and sends real-time notifications to the user's smartphone, including recommended menu items, restaurant information, and a customized message based on the results of the emotion engine's analysis.

[1099] Step 7:

[1100] The user receives a notification, reviews the information displayed in the application, and places an order if necessary, ensuring that the user selects the menu that best suits their emotional state and needs.

[1101] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1102] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1103] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1104] [Fourth embodiment]

[1105] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1106] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1107] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1108] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1109] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1110] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1111] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1112] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1113] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1114] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1115] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1116] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1117] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1118] Overall system description

[1119] This invention is an AI system that supports the rapid discovery and protection of missing pets. This system uses a server to receive and analyze photos, sounds, and GPS data of pets uploaded by owners using a terminal, and then identifies the pet's location and predicts its behavior by performing analysis and matching.

[1120] Data collection and storage

[1121] Users use a dedicated application to upload photos, sounds, and GPS data of their pets, which are then sent from the device to a server, which then stores the data in an internal database.

[1122] Data analysis and feature extraction

[1123] The server inputs the saved pet photos into the AI ​​model to extract specific features (shape, color, etc.). It also performs audio analysis on the uploaded pet's cries to extract their waveform and frequency characteristics. GPS data is analyzed in real time to capture location information and generate a map.

[1124] Data Matching

[1125] The server then matches the extracted features and the results of the voice analysis with information from shelter databases and online sources. The shelter database contains information registered when a missing pet is rescued. This allows the server to quickly notify owners if the same or similar pet is found.

[1126] Behavioral prediction

[1127] Furthermore, the server predicts pet behavior based on GPS data, taking into account the pet's breed, age, personality, surrounding terrain, etc., to estimate where the pet may head next. This behavior prediction helps to efficiently narrow the search area.

[1128] Notification and Discovery Assistance

[1129] Matching results and behavior prediction results are sent to the device in real time. Users can receive this notification and begin searching for their pet based on the provided information. If necessary, the system can also connect with a specialized pet rescue team to provide optimal support.

[1130] Specific examples

[1131] For example, if a user were to search for a missing dog, the process would be as follows: First, the user would use a dedicated application to upload the dog's photo, bark, and GPS data. The server receives this data, extracts the dog's characteristics from the photo, performs audio analysis of the bark, and analyzes the GPS data in real time. The server then matches the data with information from shelters and online to see if a similar dog is being cared for. At the same time, it predicts the dog's behavior based on the GPS data and determines the expected search area. The matching and behavior prediction results are notified to the device in real time, allowing the user to check them and search for the dog efficiently.

[1132] In this way, the present invention is a system that combines AI technology and data matching technology to enable the quick and effective discovery and protection of lost pets.

[1133] The processing flow will be explained below.

[1134] Step 1:

[1135] Users launch a dedicated application, capture photos of their missing pet, record its cries, and collect GPS data, which are then uploaded to a server via the application.

[1136] Step 2:

[1137] The server receives the uploaded photos, bird calls, and GPS data and stores them in a designated database, where they are prepared for analysis.

[1138] Step 3:

[1139] The server inputs the stored pet photos into an AI analysis module and uses image recognition technology to extract the pet's characteristics, such as fur color, body shape, and pattern.

[1140] Step 4:

[1141] The server inputs the saved bark data into a voice analysis module and extracts the characteristics of the pet's voice by analyzing the frequency characteristics and waveform patterns.

[1142] Step 5:

[1143] The server analyzes the GPS data in real time and plots the pet's current location on a map. The location information is used by the behavior prediction module.

[1144] Step 6:

[1145] The server compares the extracted photo characteristics and audio analysis results with information from shelter databases and online sources, and retrieves information about pets with similar characteristics.

[1146] Step 7:

[1147] The server predicts the pet's next move based on GPS data and its behavioral patterns, taking into account the pet's type, age, personality, and surrounding terrain information, for example.

[1148] Step 8:

[1149] The server generates a map of the search area based on the matching results and behavior prediction results and notifies the terminal in real time.

[1150] Step 9:

[1151] The device displays the notified information to the user, allowing the user to check the provided search area and pet characteristics. The user can then quickly start searching based on this information.

[1152] Step 10:

[1153] Users follow the instructions in the app to search for their pet within the provided search area, and if necessary, the app will connect them with a specialized pet rescue team to provide optimal support to help them find their pet.

[1154] Example 1

[1155] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1156] To quickly find and protect lost pets, it is necessary to accurately analyze various data such as photos, sounds, and GPS data, and efficiently search for matching information. It is also necessary to provide clues for the search by predicting the pet's current location and behavior.

[1157] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1158] In this invention, the server includes an information processing terminal means for uploading data on missing pets, an information processing device means for receiving the uploaded data and extracting features, an information processing device means for comparing the features and analysis results with records of the shelter, an information processing device means for predicting behavior based on the data, and an information processing terminal means for notifying the analysis results and prediction results, thereby enabling the rapid identification and behavior prediction of missing pets and further real-time notification.

[1159] The "information processing terminal means" is a device for collecting data about missing pets and transmitting it to an information processing device.

[1160] The "information processing device means" is a device for analyzing received data, extracting and collating necessary information, and predicting behavior.

[1161] The "location information acquisition device means" refers to a device for acquiring coordinate data of the pet's current location, such as a GPS device.

[1162] "Feature extraction" is the process of extracting specific attributes, such as shape or waveform, from uploaded photos or audio data.

[1163] "Audio analysis" is the process of analyzing uploaded bird call data and extracting its waveform, frequency characteristics, etc.

[1164] "Data matching" is the process of checking whether the extracted characteristics match the shelter's records or online information.

[1165] "Behavior prediction" is the process of estimating where your pet will go next, taking into account the pet's breed, age, personality, terrain, GPS data, and more.

[1166] "Real-time notification" is a notification function that instantly notifies users of analysis results and behavior prediction results.

[1167] System Overview

[1168] The present invention is an AI system designed to support the rapid discovery and protection of missing pets. This system is composed of an information processing terminal means, an information processing device means, and a location information acquisition device means. The information processing terminal means is a device that collects and transmits photos, audio, GPS data, etc. of pets, and the information processing device means analyzes, collates, and predicts the received data and transmits the results. The location information acquisition device means is a device that acquires pet location information in real time.

[1169] Hardware and software used

[1170] The server uses a computer with high-performance data processing capabilities. Deep learning frameworks such as TensorFlow and PyTorch are used to analyze the photo data. Audio analysis tools such as Librosa and WaveSurfer are used to analyze the audio data. Geographic information systems such as PostGIS are used to process GPS data. An SQL-based database system is used for database management.

[1171] Data collection and transmission

[1172] Users use a dedicated application to collect and upload data about their missing pets. This application runs on mobile platforms such as iOS and Android. For example, by entering data such as a photo of the missing dog, the dog's barking sound, and the current GPS location, users can easily send the data to the server by pressing the send button.

[1173] Data analysis and feature extraction

[1174] The server receives the uploaded data and performs the following analysis on each data. Photo data is input into a TensorFlow model to extract features such as the pet's shape and color. Audio data is analyzed using Librosa to extract waveforms and frequency characteristics. GPS data is processed using PostGIS and plotted on a map.

[1175] Data collation and behavior prediction

[1176] The analyzed feature data is compared with the shelter's database and data on the internet, for example by running a query such as "SELECT FROM shelter_pets WHERE shape = 'dog' AND color = 'brown'." The server then uses machine learning models such as random forests and neural networks to predict the pet's behavior and estimate its next likely destination.

[1177] Notification and Discovery Assistance

[1178] The server notifies the device of the matching results and behavior prediction results in real time. This notification is provided to the user in the form of a push notification or an in-app notification. For example, specific information such as "A similar pet has been found at shelter A" or "The next likely destination is park B" is provided.

[1179] Specific examples

[1180] For example, if a user is searching for a missing dog, they use a dedicated application to upload the following data: a photo of the dog (e.g., Fido_missing.jpg), the dog's barks (e.g., Fido_bark.wav), and current GPS data (e.g., GPS_location.csv), which are then sent directly to the server. The server receives this data and analyzes and extracts features using TensorFlow, Librosa, PostGIS, and other software. The results are compared with the shelter's database and information on the Internet to predict behavior. Finally, the results of the comparison and predicted behavior are sent to the user's device in real time, allowing for efficient searching for the pet.

[1181] The system combines AI and data analysis technologies to enable the fast and effective discovery and protection of lost pets.

[1182] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1183] Step 1: Upload your data

[1184] A user uses a dedicated application to collect photos, sounds, and GPS data related to a missing pet, and then sends this data to a server from an information processing device. For example, a user presses the "Add a new pet" button in the application, takes a photo of the pet, and uploads it. They also record the sound of the pet's sounds and upload that file. Next, the device's GPS function is used to obtain the pet's current location, which is also sent. The input data consists of a photo file (e.g., Fido_missing.jpg), an audio file (e.g., Fido_sound.wav), and GPS text data (e.g., GPS_location.csv). This data is sent to the server.

[1185] Step 2: Receiving and storing data

[1186] The server receives the data sent from the device and checks the format and consistency of each piece of data. After checking, it stores the received photo data in the "photos" table, the bird sound data in the "audio" table, and the GPS data in the "gps_logs" table. Specifically, the server stores each piece of data in temporary storage, verifies whether it is in the correct data format, and then inserts it into the corresponding database table. At this time, metadata (e.g., data timestamp, user ID) is also saved. If the data is successfully saved in the database, the server generates a success message and proceeds to the next step.

[1187] Step 3: Data analysis and feature extraction

[1188] The server analyzes the received and stored data and extracts its features. The photo data is input into a TensorFlow model to extract features such as the pet's shape and color. The input data is the stored photo file, and the output data is the extracted feature vector. Similarly, Librosa is used to analyze the sound waveform and frequency characteristics of the pet's bark data and extract a feature vector. The GPS data is plotted on a map based on the coordinate information, and the location information is confirmed. Specifically, the server feeds the photo data into a deep learning model and extracts features using a convolutional neural network (CNN). The analysis results are saved as a feature vector. Similarly, audio features such as power spectrograms and Mel-Frequency Cepstrum (MFCC) are extracted from the audio data and saved as a feature vector. PostGIS is used to convert the GPS data into geographic coordinates and plot them on a map.

[1189] Step 4: Collating the data

[1190] The server uses the extracted feature vectors to match the search results with the shelter's database and information on the internet. This process involves generating an SQL-based query to perform database matching, for example, "SELECT FROM shelter_pets WHERE shape = 'dog' AND color = 'brown'". The input data is the extracted feature vectors, and the output data is a list of matching database records. After the matching process, the server passes the details of any matching records to the next step.

[1191] Step 5: Predict behavior

[1192] The server comprehensively evaluates information such as the pet's species, age, personality, and GPS data to predict its behavior. It then uses machine learning algorithms such as random forests and neural networks to estimate the pet's next likely destination. The input data is the pet's characteristics and current GPS data, and the output data is the predicted range of movement and the next destination. Specifically, the server inputs known behavioral pattern data and current location information into a machine learning model to obtain a prediction result. For example, it generates a prediction result in the form of "The most likely next destination is a nearby park."

[1193] Step 6: Notification

[1194] The server notifies the user of the matching results and behavior prediction results in real time via push notifications or in-app notifications. The input data are the matching results and behavior prediction results obtained in the previous step, and the output data is the notification message. For example, messages such as "A similar pet is at shelter A" or "The next likely destination is park B" are sent to the user's device.

[1195] Step 7: Search support

[1196] The user receives the notification and quickly begins searching for the pet based on the provided information. For example, by heading to Park B based on the notification, the search can be carried out efficiently. If necessary, the system can also cooperate with a pet rescue team to carry out more effective search activities. When searching, the device's map function and GPS can be used to confirm the specific search location. This makes it possible to quickly find and protect the missing pet.

[1197] (Application example 1)

[1198] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1199] There is a lack of methods to quickly and efficiently find and protect missing animals. There is also a need for a system that can predict behavior in real time based on information provided by owners, thereby increasing the probability of finding animals.

[1200] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1201] In this invention, the server includes a terminal means for uploading photos of missing animals, an information processing means for receiving the uploaded photos and extracting features, an information processing means for performing information matching with an information storage means of a shelter based on the features, an information processing means for predicting behavior based on the results of the information matching, and a terminal means for notifying the results of the behavior prediction in real time, thereby enabling the efficient discovery and protection of missing animals.

[1202] A "missing animal" is a domestic animal that has become separated from its owner and whose whereabouts cannot be confirmed.

[1203] "Terminal means" refers to devices or equipment that can be operated by users, and specifically includes smartphones, tablets, and personal computers.

[1204] "Information processing means" refers to a device or system that analyzes and processes received data, and specifically includes a server or computer system.

[1205] "Photo feature extraction" is the process of using image processing technology to analyze features such as shape and color from uploaded photos and extract them as data.

[1206] A "shelter" is a facility where lost animals are rescued and temporarily housed, including animal shelters and rescue centers.

[1207] "Information storage means" refers to a device or system that stores and saves information in the form of a database or the like.

[1208] "Information matching" is the process of comparing and matching extracted data with other data sets.

[1209] "Behavioral prediction" is the process of predicting the future behavior and destinations of animals based on past data and current conditions.

[1210] "Notification means" refers to the means of communication used to inform users of analysis and prediction results, and specifically includes push notifications, email, SMS, etc.

[1211] The present invention is a comprehensive AI system for quickly locating and protecting missing animals. This system begins when a user uploads data about the missing animal (photos, sounds, GPS information) using a terminal. Specific embodiments of this system are described below.

[1212] Overall system description

[1213] The system mainly consists of the following components:

[1214] Terminal means

[1215] Server Means

[1216] Information Processing Means

[1217] Information storage means

[1218] Information matching method

[1219] Behavioral prediction methods

[1220] Notification means

[1221] Data collection

[1222] Users use a dedicated smartphone application to upload photos, sounds, and GPS data of missing animals from the terminal means, and all of this data is sent to the server means.

[1223] Data analysis and feature extraction

[1224] The server analyzes the received photo data using image analysis software called OpenCV to extract specific features (such as the animal's shape and body color). Similarly, audio analysis using Librosa analyzes the call data for characteristics such as frequency characteristics and sound spectrum. Additionally, geopy is used to analyze GPS data to determine the animal's current location and movement patterns.

[1225] Data matching and behavior prediction

[1226] The server compares the analyzed features with the shelter's database and information on the Internet. If the same or similar animal is found, the server extracts its information. Furthermore, the server predicts the animal's behavior based on the analyzed GPS data. This predicts the animal's next likely destination.

[1227] Notification and Discovery Assistance

[1228] The matching results and behavior prediction results are sent to the user's terminal in real time. Users can use this information to efficiently search for animals. If necessary, they can also work with a specialized rescue team to carry out search activities.

[1229] Specific examples

[1230] For example, if a user were to search for a missing dog, the process would go something like this: First, the user would use a dedicated application to upload a photo of the dog, its barks, and GPS data showing its recent movement route. The server receives the data, extracts the dog's characteristics from the photo, performs audio analysis of the barks, and analyzes the GPS data. The server then matches the data with information from shelters and online sources to see if a similar dog is being cared for. At the same time, it predicts the dog's behavior based on the GPS data and determines the expected search area. These results are then sent to the user's device in real time, allowing the user to efficiently search for the dog.

[1231] Prompt Sentence Examples

[1232] I have lost my dog ​​and would like some help finding him. I am providing the following information:

[1233] Photo: <Attached image file>

[1234] Sounds: <audio file>

[1235] GPS information: Latitude: 35.6895, Longitude: 139.6917

[1236] Please provide a location prediction, including recent movement patterns from the starting point.

[1237] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1238] Step 1: Upload your data

[1239] Description: A user uses a smartphone application to upload photos, sounds, and GPS data about missing animals.

[1240] Input: Photo files, audio files, GPS data (latitude and longitude)

[1241] Output: Photos, audio data, and GPS data uploaded to the server

[1242] Specific operation: The user accesses the input form within the app, selects and attaches each data file, and then presses the send button. This action sends the data from the device to the server.

[1243] Step 2: Save your data

[1244] Description: The server stores the received data in a database.

[1245] Input: Uploaded photos, audio data, GPS data

[1246] Output: Photos, audio data, and GPS data stored in the database

[1247] What happens: The server stores the received data in temporary storage and creates the appropriate database entries to store each file.

[1248] Step 3: Extracting image features

[1249] Description: The server extracts specific features from the stored photo data using OpenCV.

[1250] Input: Saved photo data

[1251] Output: Image feature data (specific patterns such as shape, color, etc.)

[1252] Specific operation: The server converts the image to grayscale using the OpenCV library, and detects key points and extracts feature vectors using the SIFT (Scale-Invariant Feature Transform) algorithm.

[1253] Step 4: Extract audio features

[1254] Description: The server analyzes the stored audio data using Librosa to extract certain audio features.

[1255] Input: Stored audio data

[1256] Output: Audio feature data (frequency characteristics, spectrum, etc.)

[1257] Specific operation: The server loads the audio data using the Librosa library and calculates MFCCs (Mel-Frequency Cepstrum Coefficients) to extract audio features.

[1258] Step 5: Analyzing the GPS data

[1259] Description: The server analyzes stored GPS data to determine the current location and movement patterns of animals.

[1260] Input: Stored GPS data

[1261] Output: Analyzed location information and movement patterns

[1262] What it does: The server uses the geopy library to analyze GPS data, identify movement patterns from past history, and predict possible next destinations.

[1263] Step 6: Data collation

[1264] Description: The server compares the extracted feature data with the shelter's database and online information.

[1265] Input: Image feature data, audio feature data, GPS analysis data

[1266] Output: Matched animal information

[1267] How it works: The server accesses shelter and online databases, applies a comparison and matching algorithm to the extracted feature data, calculates a similarity score, and identifies matching animals.

[1268] Step 7: Predict behavior

[1269] Description: The server makes animal behavior predictions based on the analyzed GPS data.

[1270] Input: Analyzed location and movement patterns

[1271] Output: Predictions of where the animal might go next

[1272] How it works: The server uses past movement data and current location information to apply an algorithm to predict where the animal will go next, along with other data such as the animal's species, age, and surrounding environment.

[1273] Step 8: Notification

[1274] Description: The server notifies the user of the matching results and behavior prediction results to the user's terminal means.

[1275] Input: Matched animal information, behavior prediction information

[1276] Output: A notification message that is displayed on the user's terminal.

[1277] Specific operation: The server compiles the matching results and behavior prediction information and sends them to the user's device via push notification, email, SMS, etc. This allows the user to know where the animal was found and potential locations to find it next.

[1278] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1279] Overall system description

[1280] This invention is an AI system that supports the rapid discovery and protection of lost pets, and improves the user experience by combining it with an emotion engine that recognizes the user's emotions. In this system, a server receives photos, sounds, GPS data, etc. of pets uploaded by owners using a terminal, analyzes and matches them, and adjusts search support based on the user's emotions recognized by the emotion engine.

[1281] Data collection and storage

[1282] Users use a dedicated application to upload photos, sounds, and GPS data of their pets. This data is sent from the device to a server, which then stores the received data in an internal database. In addition, the user's emotional data is also collected through an emotion engine.

[1283] Data analysis and feature extraction

[1284] The server inputs the saved pet photos into an AI analysis module to extract specific features (shape, color, etc.). The audio analysis module also analyzes the uploaded pet's cries for waveform and frequency characteristics. GPS data is analyzed in real time and displayed as a map. An emotion engine analyzes the user's emotional state and combines that data with other data.

[1285] Data Matching

[1286] The server then matches the extracted features and the results of the voice analysis with information from shelter databases and online sources. The shelter database contains information registered when a missing pet is rescued. If an identical or similar pet is found, the server can quickly notify the owner.

[1287] Behavioral prediction

[1288] The server predicts the pet's next likely location based on GPS data and its movement patterns. For example, it takes into account the pet's type, age, personality, and surrounding terrain information. This behavior prediction effectively narrows the search area.

[1289] Emotional Engine Adjustment

[1290] The emotion engine analyzes the user's emotional state and adjusts its search assistance based on its output. For example, if the user is extremely anxious, it will change the priority of notifications and provide urgent information immediately. It also provides customized messages and advice based on the user's emotions, reducing anxiety and stress.

[1291] Notification and Discovery Assistance

[1292] Matching results and behavior prediction results are sent to the device in real time. The user receives this notification and begins searching for a pet based on the provided information. If necessary, the system can also work with a specialized pet rescue team to provide optimal support. The analysis results of the emotion engine are also reflected in the notification, and support is provided according to the user's feelings.

[1293] Specific examples

[1294] For example, if a user were searching for a missing cat, the process would be as follows: First, the user would use a dedicated application to upload the cat's photo, meows, and GPS data. The server receives this data, extracts the cat's characteristics from the photo, performs audio analysis of the meows, and analyzes the GPS data in real time. The server then matches the data with information from shelters and online sources to see if a similar cat is being cared for. At the same time, it predicts the cat's behavior based on the GPS data and determines the expected search area. The matching and predicted behavior results are notified to the device in real time, allowing the user to check them and search for the cat efficiently. Furthermore, the emotion engine analyzes the user's emotions and provides messages and notifications according to the level of urgency, thereby reducing the user's anxiety and supporting the search process.

[1295] In this way, the present invention is a system that combines AI technology, data matching technology, and an emotion engine to quickly and effectively find and protect lost pets, while also providing support that takes into consideration the emotions of pet owners.

[1296] The processing flow will be explained below.

[1297] Step 1:

[1298] Users launch a dedicated application, capture photos of their missing pet, record its cries, and collect GPS data, which are then uploaded to a server via the application.

[1299] Step 2:

[1300] To collect the user's emotional state, the device uses an emotion engine to analyze the user's facial expressions, tone of voice, input text, etc. The analyzed emotional data is also sent to the server.

[1301] Step 3:

[1302] The server receives the uploaded pet photos, sounds, and GPS data and stores them in a designated database, as well as the received emotion data.

[1303] Step 4:

[1304] The server inputs the stored pet photos into an AI analysis module and uses image recognition technology to extract the pet's characteristics, such as fur color, body shape, and pattern.

[1305] Step 5:

[1306] The server inputs the saved bark data into a voice analysis module and extracts the characteristics of the pet's voice by analyzing the frequency characteristics and waveform patterns.

[1307] Step 6:

[1308] The server analyzes the GPS data in real time and plots the pet's current location on a map. The location information is used by the behavior prediction module.

[1309] Step 7:

[1310] The server compares the extracted photo characteristics and audio analysis results with information from shelter databases and online sources, and retrieves information about pets with similar characteristics.

[1311] Step 8:

[1312] The server predicts the pet's next likely location based on GPS data and its movement patterns, taking into account factors such as the pet's type, age, personality, and surrounding terrain.

[1313] Step 9:

[1314] The server generates a map of the search area based on the matching results and behavior prediction results, and notifies the device in real time. The content and priority of notifications are adjusted based on the output of the emotion engine.

[1315] Step 10:

[1316] The device displays the notified information to the user, allowing the user to check the provided search area and pet characteristics. The user can then quickly start searching based on this information.

[1317] Step 11:

[1318] Users follow the instructions in the app to search for their pet within the provided search area, and if necessary, the app will connect them with a specialized pet rescue team to provide optimal support to help them find their pet.

[1319] Step 12:

[1320] The server collects emotional data in real time as the user explores, analyzes the user's stress level and anxiety using an emotion engine, and provides customized messages and advice to the user via their device based on the analysis results.

[1321] Step 13:

[1322] The device displays customized messages and advice provided by the emotion engine to the user to support their exploration activities. If necessary, it re-analyzes the user's emotional state and makes further adjustments.

[1323] This allows for quick and effective detection and protection of pets, reducing the mental burden on owners.

[1324] Example 2

[1325] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1326] Conventional technologies have not been sufficient to quickly and effectively locate and protect lost pets. Furthermore, there is a lack of support that takes into account the owner's emotions. Therefore, there is a need for methods to improve the efficiency of searches and the user experience. In particular, there has been a challenge in developing a method to effectively locate pets using their appearance, sounds, and location information.

[1327] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1328] In this invention, the server includes a terminal means for uploading photos, sounds, and GPS data related to the missing pet, a server means for receiving the uploaded photos, sounds, and GPS data and extracting features including data derived from emotions, a server means for matching the features with data on shelters and information on the Internet, and an emotion engine means for analyzing the user's emotion data and adjusting the priority of notifications based on the output. This makes it possible to quickly and effectively find and protect missing pets and provide support that takes the user's emotions into consideration.

[1329] "Terminal means" refers to a device or application that allows a user to upload information about a missing pet.

[1330] "Server means" means a computer system for processing, storing and analyzing data received from the terminal.

[1331] The "emotion engine means" is a software module that analyzes the user's emotional state and adjusts the system's operation based on the results.

[1332] "Features" are identifying information such as shape, color, and sound characteristics extracted from photos and sounds of pets.

[1333] "Matching" is the process of comparing the extracted features with data from shelters and online sources.

[1334] "Shelter" means a placement facility where lost pets may be taken.

[1335] "GPS data" is geographic information that indicates your pet's current location and last known location.

[1336] "Audio analysis means" is a technology for analyzing uploaded bird call data and extracting its characteristics.

[1337] "Data analysis means" refers to technology for predicting pet behavior based on GPS data and past behavioral patterns.

[1338] "Notification means" is a system function for notifying users of matching results and behavior prediction results.

[1339] MODE FOR CARRYING OUT THE INVENTION

[1340] This invention is an AI system that supports the rapid discovery and protection of lost pets, and improves the user experience by combining it with an emotion engine that recognizes the user's emotions. In this system, the server receives, analyzes, and matches photos, sounds, and GPS data of pets uploaded by owners using their devices, and adjusts search support based on the user's emotions recognized by the emotion engine.

[1341] Terminal means:

[1342] Users use a dedicated application to upload photos of their pets, their pet's cries, and GPS data. This data is sent from the device to the server. Photos taken by the user are sent in JPEG format, recorded pet cries are sent in WAV format, and GPS data is sent in JSON format. In addition, the device captures the user's facial expressions with its camera and sends the analysis results to the server via an emotion engine.

[1343] Server means:

[1344] The server stores the received data in an internal database. The stored pet photos are input into an AI image analysis module (e.g., YOLO or ResNet) to extract specific features (shape, color, pattern, etc.). An audio analysis module (e.g., MFCC or WaveNet) is used to analyze the waveform and frequency characteristics of the pet's bark data. The server also analyzes GPS data in conjunction with a map API (e.g., Google Maps API) to determine location information in real time. The emotion engine analyzes the user's emotional state and stores the data on the server.

[1345] Feature data matching:

[1346] The server then matches the extracted features and the results of the voice analysis with information from shelter databases and on the Internet. The shelter database contains information about lost pets that are registered when they are rescued. The server compares this data and promptly notifies the user if the same or similar pet is found.

[1347] Behavioral prediction and emotional data adjustment:

[1348] The server predicts the pet's next likely location based on GPS data and its behavioral patterns. This prediction is performed using an AI behavioral prediction model (e.g., LSTM or RNN model). It also adjusts the priority of notification messages based on the analysis results of the emotion engine. It generates customized messages according to specific emotional states and provides them to the user.

[1349] Notifications and Support:

[1350] Matching results and behavior prediction results are sent to the device in real time. Users can receive this notification and begin searching for their pet based on the provided information. Furthermore, if necessary, they can work with a specialized pet rescue team to provide optimal support.

[1351] Specific examples

[1352] For example, if a user were searching for a missing cat, the process would be as follows: First, the user would use a dedicated application to upload the cat's photo, meows, and GPS data. The server receives this data, extracts the cat's characteristics from the photo, performs audio analysis of the meows, and analyzes the GPS data in real time. The server then matches the data with information from shelters and online sources to see if a similar cat is being cared for. At the same time, it predicts the cat's behavior based on the GPS data and determines the expected search area. These matching and behavior prediction results are notified to the device in real time, allowing the user to search for the cat efficiently. Furthermore, the emotion engine analyzes the user's emotions and provides messages and notifications according to the level of urgency, thereby reducing the user's anxiety and supporting the search process.

[1353] Prompt Sentence Examples

[1354] Please explain the specific process of how the server uses the cat photos, meows, and GPS data uploaded from the dedicated app.

[1355] In this way, the present invention combines AI technology, data matching technology, and an emotion engine to quickly and effectively locate and protect lost pets, and provide emotional support to pet owners.

[1356] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1357] Step 1: Data collection

[1358] Users use a dedicated application to upload photos, sounds, and GPS data of their pets.

[1359] Input: Pet photo (JPEG format), pet sound (WAV format), GPS data (JSON format)

[1360] Output: Collected data is temporarily stored on the device and ready to be sent.

[1361] Specific operation: The user takes a photo of the cat using a smartphone and saves it in the application. The user records the cat's meows and uses the smartphone's GPS function to collect location information.

[1362] Step 2: Send data

[1363] The terminal transmits the uploaded data from the user to a server via the Internet.

[1364] Input: Collected photos, calls, and GPS data

[1365] Output: Data is sent to the server.

[1366] How it works: The device uses Wi-Fi or mobile data to upload photos and bird call data to the server, and GPS data is also sent to the server at the same time.

[1367] Step 3: Save Data

[1368] The server stores the received data in an internal database.

[1369] Input: Photos, sounds, and GPS data sent to the server

[1370] Output: Photos, calls, and GPS data stored in a database

[1371] Specific operation: The server stores JPEG images in the "Pet Image" table, WAV audio data in the "Pet Audio" table, and JSON-formatted GPS data in the "Pet Location Information" table.

[1372] Step 4: Sentiment Data Analysis

[1373] The device analyzes the user's emotional state using a dedicated emotion engine and sends the data to a server.

[1374] Input: User's facial expression data

[1375] Output: Parsed emotional state data

[1376] Specific operation: The device uses an emotion engine to analyze the user's facial expressions and recognize emotional states such as "anxiety" or "impatience." The analysis results are sent to the server.

[1377] Step 5: Image analysis

[1378] The server uses an image analysis module to extract specific features from stored photos of pets.

[1379] Input: Saved pet photo

[1380] Output: Extracted feature data (shape, color, pattern, etc.)

[1381] How it works: The server uses an AI image analysis model (such as YOLO or ResNet) to scan the pet's image and extract features such as shape, color, and pattern. These feature data are stored in a "feature data" table.

[1382] Step 6: Audio analysis

[1383] The server uses an audio analysis module to analyze the stored bird call data.

[1384] Input: Stored call data

[1385] Output: Analyzed audio feature data (waveform, frequency characteristics, etc.)

[1386] Specific operation: The server uses an audio analysis module (such as MFCC or WaveNet) to analyze the waveform and frequency characteristics of the bird call data and generate audio feature data. This data is stored in the "audio feature data" table.

[1387] Step 7: GPS data analysis

[1388] The server analyzes the GPS data in real time to determine the location.

[1389] Input: Stored GPS data

[1390] Output: Parsed map information

[1391] What it does: The server connects the GPS data to the Google Maps API and visualizes the location information in real time. This information is stored in the "Map Data" table.

[1392] Step 8: Data Matching

[1393] The server matches the extracted features and the results of voice analysis with information from the shelter's database and online sources.

[1394] Input: Pet feature data, audio feature data

[1395] Output: Matching result data

[1396] Specific operation: The server accesses the shelter's database to search for pets with matching characteristics, and then compares them with online pet information to obtain matching results, which are then stored in a "matching results" table.

[1397] Step 9: Behavioral Prediction

[1398] The server uses GPS data and the pet's behavioral patterns to predict where the pet may move next.

[1399] Input: Stored GPS data, behavioral pattern data

[1400] Output: Predicted action location data

[1401] Specific operation: The server uses an AI behavior prediction model (LSTM or RNN model) based on past behavior data to predict the pet's movement pattern. The predicted location data is saved in the "Behavior prediction data" table.

[1402] Step 10: Emotional Data Adjustment

[1403] The server analyzes the user's emotional state and adjusts the search assistance based on its output.

[1404] Input: Parsed emotion data

[1405] Output: Reconciled notification data

[1406] Specific operation: The server adjusts the notification priority based on the analysis results of the emotion engine and generates messages that quickly provide the most important information to the user.

[1407] Step 11: Notification and Support

[1408] The server notifies the terminal of the matching results and behavior prediction results in real time and notifies the user.

[1409] Input: Matching result data, behavior prediction data

[1410] Output: Notification data to the device

[1411] Specific operation: The server compiles the matching results and behavior prediction results and sends them to the device as a push notification. The user can use this notification to continue searching for their pet. In addition, in the event of an emergency, the system can cooperate with a specialized pet rescue team to provide support.

[1412] (Application example 2)

[1413] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1414] Conventional food delivery systems have difficulty improving customer satisfaction and providing efficient services because they do not adequately consider users' emotional state and real-time feedback, which leads to problems such as user frustration and insufficient order optimization.

[1415] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means for uploading photos of missing pets, server means for receiving the uploaded photos and extracting features, server means for matching the photos with a database of shelters based on the features, server means including an emotion engine for analyzing the user's emotion data, and server means for adjusting notifications based on the emotion data. This makes it possible to grasp the user's emotional state in real time and provide optimal menus and notifications based on that.

[1416] "Terminal" refers to a device that allows a user to input information or send data to a server.

[1417] "Server" refers to a computer system that receives and stores uploaded data and performs various analyses.

[1418] "Features" refer to identifiable information or patterns extracted from image or audio data.

[1419] "Server means" refers to a combination of hardware and software that implements a particular function within a server.

[1420] An "emotion engine" refers to an artificial intelligence mechanism that analyzes the user's emotional state and outputs analytical results based on that data.

[1421] "Notification" refers to information or messages sent by the server to the user.

[1422] "Matching" refers to the process of searching for and matching relevant data and information based on characteristics and analysis results.

[1423] A "database" refers to a collection of information that stores specific information in an organized manner and is configured to facilitate search and retrieval.

[1424] In this invention, a system is constructed for analyzing the emotional state of a user and providing optimal menus and notifications in real time in the operation of a food delivery system. Specific embodiments of this system are described below.

[1425] Overall system description

[1426] A food delivery service application consists of the following main components:

[1427] Terminal means: A device such as a smartphone through which a user enters information and transmits the data to a server.

[1428] Server means: A computer system that receives, stores, and performs various analyses of data.

[1429] Emotion Engine: An AI module that analyzes the user's emotional state.

[1430] Notification vehicle: A mechanism for delivering information from a server to a user.

[1431] Data collection and storage

[1432] Users use a smartphone app to upload food photos, ratings, GPS data, and comments. This data is sent from the smartphone to a server, which stores the received data in an internal database. User emotion data is also collected through an emotion engine.

[1433] Data analysis and feature extraction

[1434] The server inputs the saved food photos into an AI analysis module to extract specific features (shape, color, presentation, etc.). It also performs sentiment analysis on uploaded comments using natural language processing technology. GPS data is analyzed in real time and converted into map information. The emotion engine analyzes the user's emotional state and combines that data with other data.

[1435] Data Matching

[1436] The server then matches the extracted features and sentiment analysis results with food delivery menus and store information, enabling users to quickly obtain menus and store information that meet their desired criteria.

[1437] Behavioral prediction

[1438] The server predicts users' behavior based on GPS data and their past order history. For example, it takes into account their preferences and rating history to predict what menu items they are likely to order next. This behavior prediction allows delivery companies and restaurants to efficiently manage and prepare inventory.

[1439] Emotional Engine Adjustment

[1440] The emotion engine analyzes the user's emotional state and tailors notifications based on its output. For example, if the user is feeling stressed, it may send relaxing menu recommendations or inspiring messages to reduce anxiety and stress.

[1441] Notifications and Support

[1442] Matching results and behavioral prediction results are sent to the user's smartphone in real time. Users can receive these notifications and place orders immediately. In addition, the analysis results of the emotion engine are reflected in the notifications, providing support that takes the user's emotions into consideration.

[1443] Specific examples

[1444] For example, if a user wants to eat delicious pasta, the process goes like this: First, the user uses a smartphone app to upload a photo of themselves along with a comment saying, "I want to eat delicious pasta." The server receives this data, analyzes the user's emotions from the photo, and analyzes the comment using natural language processing. Then, based on the extracted features, it matches relevant store and menu information and notifies the user's smartphone. The emotion engine analyzes the user's emotions and sends notifications recommending relaxing menu items, thereby reducing the user's anxiety and providing the optimal food delivery service.

[1445] Prompt Sentence Examples

[1446] "Implement an application that identifies emotions from a user's facial image, sends it to a server along with GPS data and comments, and then notifies a food delivery service of menu recommendations in real time based on that data."

[1447] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1448] Step 1:

[1449] Users use a smartphone app to upload photos of food, GPS data, and comments, along with a photo of their face, which the system receives as input. The uploaded data is then sent to a server via an internet connection.

[1450] Step 2:

[1451] The server stores the received data. The stored data includes the user's photos, GPS data, and comments. The server first sends the user's photos to the emotion engine and identifies emotions from the facial images. This emotion data is analyzed by the emotion engine's AI model.

[1452] Step 3:

[1453] The server receives the emotion data output by the emotion engine and sends it along with the comment data to a natural language processing (NLP) module. The NLP module analyzes the emotion data and comments to identify the user's current emotional state and the specific content of the comment. This data analysis clarifies the user's desires and emotional state.

[1454] Step 4:

[1455] Based on the analysis results, the server sends the stored food photo data to a feature extraction module, which extracts features such as shape, color, and presentation from the image data. This feature data is used to match menus and restaurant information.

[1456] Step 5:

[1457] The server searches internal databases and external related information based on the extracted feature data and emotion data to match the optimal menu and restaurant information. The matching results are selected based on the user's current emotional state and the content of their comments.

[1458] Step 6:

[1459] The server organizes the matching results and sends real-time notifications to the user's smartphone, including recommended menu items, restaurant information, and a customized message based on the results of the emotion engine's analysis.

[1460] Step 7:

[1461] The user receives a notification, reviews the information displayed in the application, and places an order if necessary, ensuring that the user selects the menu that best suits their emotional state and needs.

[1462] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1463] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1464] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1465] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1466] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1467] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1468] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1469] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1470] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1471] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1472] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1473] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1474] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1475] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1476] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1477] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1478] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1479] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1480] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1481] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1482] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1483] The following is further disclosed regarding the above embodiment.

[1484] (Claim 1)

[1485] a terminal means for uploading photographs relating to the missing pet;

[1486] a server means for receiving the uploaded photograph and extracting features;

[1487] A server means for matching the characteristics with a database of shelters;

[1488] A system including:

[1489] (Claim 2)

[1490] A terminal means for uploading the sounds of the missing pet;

[1491] a server means for receiving the uploaded bird calls and performing audio analysis;

[1492] a server means for matching the results of the voice analysis with information on the Internet;

[1493] 10. The system of claim 1, comprising:

[1494] (Claim 3)

[1495] a GPS device means for tracking the location of the missing pet;

[1496] a server means for receiving the location information and grasping the location in real time;

[1497] a server means for predicting behavior based on the location information;

[1498] 10. The system of claim 1, comprising:

[1499] "Example 1"

[1500] (Claim 1)

[1501] an information processing terminal means for uploading data relating to the missing pet;

[1502] an information processing device that receives the uploaded data and extracts features from the data;

[1503] an information processing device for collating records of the shelter based on the characteristics and analysis results;

[1504] an information processing device that performs behavior prediction based on the data;

[1505] an information processing terminal means for notifying the analysis results and prediction results;

[1506] A system including:

[1507] (Claim 2)

[1508] an information processing terminal means for uploading the voice of the missing pet;

[1509] an information processing device that receives the uploaded voice and performs voice analysis;

[1510] an information processing device that compares data based on the results of the voice analysis;

[1511] 10. The system of claim 1, comprising:

[1512] (Claim 3)

[1513] a location information acquisition device means for tracking the location of a missing pet;

[1514] an information processing device means for receiving the location information and displaying the location in real time;

[1515] an information processing device that performs behavior prediction based on the location information;

[1516] 10. The system of claim 1, comprising:

[1517] "Application Example 1"

[1518] (Claim 1)

[1519] a terminal means for uploading photographs relating to the missing animal;

[1520] an information processing means for receiving the uploaded photograph and extracting features;

[1521] an information processing means for performing information matching with information storage means of the protection facility based on the characteristics;

[1522] an information processing means for predicting behavior based on the information matching result;

[1523] a terminal means for notifying the behavior prediction result in real time;

[1524] A system including:

[1525] (Claim 2)

[1526] a terminal means for uploading the sounds of the missing animal;

[1527] an information processing means for receiving the uploaded bird calls and performing audio analysis;

[1528] an information processing means for performing information matching with information on the Internet based on the results of the voice analysis;

[1529] an information processing means for predicting behavior based on the information matching result;

[1530] a terminal means for notifying the behavior prediction result in real time;

[1531] 10. The system of claim 1, comprising:

[1532] (Claim 3)

[1533] a location determination means for tracking the location of the missing animal;

[1534] an information processing means for receiving the location information and grasping the location in real time;

[1535] an information processing means for predicting behavior based on the location information;

[1536] a terminal means for notifying the behavior prediction result in real time;

[1537] 10. The system of claim 1, comprising:

[1538] "Example 2: Combining Emotion Engines"

[1539] (Claim 1)

[1540] A terminal means for uploading photos, sounds, and GPS data related to missing pets;

[1541] A server means for receiving the uploaded photos, sounds, and GPS data and extracting features including emotion-derived data;

[1542] A server means for matching data on shelters and information on the Internet based on the characteristics;

[1543] emotion engine means for analyzing the emotion data of the user and adjusting the priority of notifications based on the output of the emotion engine means;

[1544] A system including:

[1545] (Claim 2)

[1546] A sound analysis method for analyzing the cries of missing pets;

[1547] A server means for analyzing GPS data in real time and grasping location information;

[1548] 2. The system according to claim 1, further comprising a server means for matching with information on shelters and the Internet.

[1549] (Claim 3)

[1550] a data analysis means for predicting the location of a missing pet;

[1551] a server means for predicting behavior based on the analysis;

[1552] A notification method that instantly provides high-urgency notifications based on the analysis results of the emotion engine, and

[1553] 10. The system of claim 1, comprising:

[1554] "Application example 2 when combining emotion engines"

[1555] (Claim 1)

[1556] a terminal means for uploading photographs relating to the missing pet;

[1557] a server means for receiving the uploaded photograph and extracting features;

[1558] A server means for matching the characteristics with a database of shelters;

[1559] a server means including an emotion engine for analyzing emotion data of a user;

[1560] a server means for adjusting a notification based on the emotion data;

[1561] A system including:

[1562] (Claim 2)

[1563] A terminal means for uploading the sounds of the missing pet;

[1564] a server means for receiving the uploaded bird calls and performing audio analysis;

[1565] a server means for matching the results of the voice analysis with information on the Internet;

[1566] a server means including an emotion engine for analyzing emotion data of a user;

[1567] server means for adjusting notifications based on said emotion data;

[1568] 10. The system of claim 1.

[1569] (Claim 3)

[1570] a GPS device means for tracking the location of the missing pet;

[1571] a server means for receiving the location information and grasping the location in real time;

[1572] a server means for predicting behavior based on the location information;

[1573] a server means including an emotion engine for analyzing emotion data of a user;

[1574] server means for adjusting notifications based on said emotion data;

[1575] 10. The system of claim 1. [Explanation of symbols]

[1576] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a terminal means for uploading photographs relating to the missing pet; a server means for receiving the uploaded photograph and extracting features; A server means for matching the characteristics with a database of shelters; A system including:

2. A terminal means for uploading the sounds of the missing pet; a server means for receiving the uploaded bird calls and performing audio analysis; a server means for matching the results of the voice analysis with information on the Internet; The system of claim 1 , comprising:

3. a GPS device means for tracking the location of the missing pet; a server means for receiving the location information and grasping the location in real time; a server means for predicting behavior based on the location information; The system of claim 1 , comprising:

Images