system

Abstract

A system is provided.SOLUTION: A system, comprising: means for collecting pictures and post contents related to a tourist spot from an SNS platform; means for performing image analysis on the collected pictures to determine a progress state of cherry blossoms or autumnal leaves; means for analyzing text of the collected post contents to determine a progress state of cherry blossoms or autumnal leaves; means for evaluating authenticity of the images and whether or not the images have been processed; means for integrating results of the image analysis and the text analysis to estimate the progress state of cherry blossoms or autumnal leaves; and means for providing the estimated progress state to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Today's social media platforms often contain edited photos, which often lead to discrepancies between expectations and reality when visiting tourist destinations. Furthermore, users seek accurate, real-time information to avoid missing the best viewing times for seasonal events like cherry blossoms and autumn leaves, but current information sources have limitations in their accuracy. This often makes it difficult for users to plan their trips. The present invention aims to solve these problems. [Means for solving the problem]

[0005] The present invention solves the above problem with a system that includes: means for collecting photos and posted content related to tourist destinations from social media platforms; means for performing image analysis on the collected photos to determine the progress of cherry blossoms and autumn leaves; means for analyzing the text of the collected posted content to determine the progress of cherry blossoms and autumn leaves; means for evaluating the authenticity of the image and whether it has been edited and restoring the image to its original state; means for integrating the results of the image and text analysis to estimate the progress of cherry blossoms and autumn leaves at tourist destinations; and means for providing the estimated progress to users. Furthermore, the system provides more accurate information by including means for identifying tourist destinations and estimating their progress based on the location information and posting date and time of the photos and posted content collected from social media platforms, and means for generating near-real-time images and providing them to users.

[0006] "SNS Platform" means a website or application that provides social networking services and enables users to post and share photos and text.

[0007] "Tourist destinations" refer to places and facilities that are primarily intended for tourists to visit, and often have specific attractions for each season.

[0008] "Photos" refer to still images shared on social media platforms whose content reflects the current state of a tourist destination.

[0009] "Post content" refers to the text information that users share along with photos on social media platforms, including descriptions and impressions of the local situation.

[0010] "Image analysis" refers to the technical means of applying algorithms to collected photographic data to evaluate and judge its content.

[0011] "Text analysis" refers to the use of natural language processing technology to analyze the text data of posted content and understand and judge its content.

[0012] "Authenticity" refers to the evaluation criteria used to determine whether collected photos and posted content accurately reflect reality.

[0013] "Whether or not a photograph has been edited" refers to the criteria for determining whether or not a photograph has been digitally edited.

[0014] "Original" refers to the original state of the photograph before it has been edited, and is necessary to enhance its authenticity.

[0015] "Progress" refers to the progress of seasonal natural phenomena at tourist spots, such as the state of cherry blossom blooming or the progress of autumn leaves.

[0016] "Real-time" refers to information being provided immediately without delay, allowing users to quickly understand the current situation. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention relates to a system that collects photos and posts related to tourist spots from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves. The following specific embodiments are conceivable for carrying out the present invention.

[0039] Data collection

[0040] The server periodically collects posts with hashtags such as "cherry blossoms" and "autumn leaves" via the SNS platform API. From the posts, it extracts photos, text, posting date and time, and location information and temporarily stores them in a database.

[0041] Data analysis

[0042] Image analysis

[0043] The server then sends the photos stored in the database to an AI image analysis module, which analyzes the image features to determine the state of cherry blossom blooming and the progress of autumn leaves. The analysis results are then sent back to the server along with metadata.

[0044] Text analytics

[0045] The server sends the text of the post to a natural language processing (NLP) module, which analyzes the context and keywords. The NLP module extracts keywords (e.g., "full bloom" or "beginning to change color") that predict the state of the cherry blossoms and autumn leaves from the post and returns the results to the server.

[0046] Image authenticity assessment and manipulation determination

[0047] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally altered. If manipulation is confirmed, an algorithm is applied to return the photo to its original state and the results are sent back to the server.

[0048] Estimating progress

[0049] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0050] Provision to users

[0051] Users use a smartphone app to search for the current cherry blossom and autumn foliage conditions at tourist spots. The device sends the user's request to a server, which retrieves the latest information from a database. The server then generates analytical information and related images and provides them to the user. Users can then use the app to plan their trip based on real-time information.

[0052] Specific examples

[0053] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, the user opens the smartphone app and searches for "Ueno Park cherry blossoms." The device sends this request to the server, which retrieves the latest cherry blossom information for Ueno Park from an existing database.

[0054] As a result of photo analysis, the information "full bloom" was obtained, so the server provides the user with the latest information, "The cherry blossoms in Ueno Park are currently in full bloom." Furthermore, the server generates near real-time images and displays them on the app, allowing the user to visually check the local situation.

[0055] As described above, the present invention makes it easier for users to plan their travel destinations based on highly accurate, real-time information.

[0056] The processing flow will be explained below.

[0057] Step 1:

[0058] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts contain photos, text, posting date and time, and location information.

[0059] Step 2:

[0060] The server extracts photo data from the collected posts and sends it to an image analysis module, which detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color.

[0061] Step 3:

[0062] The server sends the text data of the post to a natural language processing (NLP) module, which extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the text and analyzes the context.

[0063] Step 4:

[0064] The server evaluates the authenticity of the photo based on the results from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo and sends the result back to the server.

[0065] Step 5:

[0066] The server sends the photo to the image manipulation determination module to check whether the image has been manipulated. If manipulation is confirmed, the image manipulation determination module processes the photo to return it to its original state and sends the converted image to the server.

[0067] Step 6:

[0068] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0069] Step 7:

[0070] A user uses a smartphone app to search for the current cherry blossom or autumn foliage status of a tourist spot, and the device sends this request to the server.

[0071] Step 8:

[0072] The server retrieves the latest information on tourist spots from the database in response to user requests. The retrieved information includes estimated cherry blossom and autumn foliage progress and related images.

[0073] Step 9:

[0074] The server responds to users with images related to the current cherry blossom and autumn foliage conditions at tourist destinations, allowing them to use a smartphone app to check the local conditions based on real-time information and plan their trip.

[0075] Example 1

[0076] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0077] In today's world, there is a need to effectively utilize the vast amount of information available on social media to understand the progress of cherry blossoms and autumn leaves at tourist spots in real time. However, there are issues such as the effort required for users to manually search social media to gather information, the difficulty of assessing the credibility of the information, and the risk of photo manipulation. The challenge is to solve these issues and provide more accurate and reliable information in real time.

[0078] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0079] In this invention, the server includes means for collecting photos and posts related to tourist destinations from SNS platforms, means for determining the progress of cherry blossoms and autumn leaves in the collected photos using an AI image analysis module, means for analyzing the text of the collected posts using a natural language processing module to determine the progress of cherry blossoms and autumn leaves, means for evaluating the authenticity of the images and whether they have been edited using an AI image analysis module and an image manipulation determination module and restoring the images to their original state if necessary, means for integrating the results of the image analysis and text analysis and using an AI model to estimate the progress of cherry blossoms and autumn leaves at tourist destinations, and means for providing the estimated progress to users. This allows users to easily obtain accurate and reliable progress information on cherry blossoms and autumn leaves in real time, enabling them to efficiently plan their sightseeing trips.

[0080] "SNS Platform" means an online service that enables users to post photos and text and share those posts with other users.

[0081] The "image analysis module" is a program that uses AI technology to analyze the characteristics of a photo and automatically determine its content.

[0082] A "natural language processing module" is a technology for understanding and analyzing text data, and is a program capable of extracting specific keywords and context.

[0083] "Authenticity assessment" is the process of determining whether a photograph or text accurately reflects a real-world situation.

[0084] The "image manipulation determination module" is a technology for determining whether an image has been digitally manipulated.

[0085] An "AI model" is an artificial intelligence algorithm and architecture that is trained to perform a specific task based on specified input data.

[0086] "Real-time" means providing data or information with a very short time delay, so that it is nearly simultaneous with real time.

[0087] A "tourist destination" is a place or area where visitors gather to see natural elements such as cherry blossoms or autumn leaves.

[0088] "Progress" refers to the current stage of development of cherry blossoms or autumn leaves and the degree of their peak viewing.

[0089] "Text analysis" is the process by which a computer analyzes the content of a sentence or document to extract specific information.

[0090] A "post" is content such as a photo or text that a user publishes on a social media platform.

[0091] This invention is a system that collects photos and posts related to tourist spots from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves.To implement this system, a combination of multiple hardware and software components is used.

[0092] The main components of the system are:

[0093] 1. Data Collection

[0094] The server periodically collects posts tagged with hashtags such as "cherry blossoms" and "autumn leaves" using the APIs of social media platforms. Specifically, it does this through the Twitter API and Instagram API. The server extracts photos, text, posting date and time, and location information from the collected posts and temporarily stores this data in a database. For example, it manages the data using MySQL or PostgreSQL.

[0095] 2. Data Analysis

[0096] Image analysis:

[0097] The server sends the photos stored in the database to an AI image analysis module, which uses deep learning libraries such as TensorFlow and PyTorch, to identify the state of cherry blossom blooming and the progress of autumn leaves and generate corresponding metadata.

[0098] Text analysis:

[0099] The server sends the text of the post to a natural language processing (NLP) module that analyzes the context and keywords. The NLP module uses Hugging Face's Transformers library to extract keywords such as "full bloom" and "beginning to change color" from the text and returns the results to the server.

[0100] 3. Image authenticity assessment and manipulation determination

[0101] The server evaluates the authenticity of the photo based on the analysis results sent from the AI ​​image analysis module. This process uses technology that compares it with previously posted data and evaluates the degree of pattern matching. The image manipulation detection module also determines whether the photo has been digitally altered. Specifically, it checks for unnatural color changes and pixel consistency, and if any abnormalities are found, it applies an algorithm to correct the image.

[0102] 4. Estimating progress

[0103] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at specific tourist spots. It uses AI models such as Scikit-learn and TensorFlow to quantify the condition of each tourist spot and determine the best time to see them.

[0104] 5. Provision to Users

[0105] A user uses a smartphone application to search for the current cherry blossom or autumn foliage conditions at a tourist spot. The device sends the user's request to a server, which retrieves the latest information from a database. The server generates analytical information and related images and provides them to the user. The user can then plan their trip based on the information displayed within the app.

[0106] Specific examples

[0107] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, they open the smartphone app and search for "Ueno Park cherry blossoms." The device then sends this search request to the server. The server retrieves the latest cherry blossom information for Ueno Park from an existing database, analyzes the information, and generates near-real-time images to provide to the user. Based on this, users can visually check the local conditions and efficiently plan their trip.

[0108] Prompt Sentence Examples

[0109] For example, a possible prompt for a generative AI model might be:

[0110] "Please tell me the steps to analyze the progress of cherry blossoms and autumn leaves at tourist spots from social media posts."

[0111] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0112] Step 1:

[0113] Collecting social media posts

[0114] The server periodically collects posts tagged with specific hashtags, such as "cherry blossoms" or "autumn leaves," using the API of the social media platform. Specifically, it uses the Twitter API or Instagram API. The server extracts photos, text, posting date and time, and location information from the posts. The input is the raw posting data obtained from the social media API, and the output is a database entry with the extracted photos, text, posting date and time, and location information. This data is temporarily stored in a database. For example, it includes operations to retrieve data in JSON format and save each field to an SQL database.

[0115] Step 2:

[0116] Image analysis

[0117] The server sends the photos stored in the database to an AI image analysis module. Specifically, it uses a deep learning model using TensorFlow and PyTorch. The input is photo data retrieved from the database, and the output generates metadata indicating the state of cherry blossom blooming and the progress of autumn leaves. For example, it converts images into feature vectors and classifies them as "blooming" or "full bloom" based on those vectors.

[0118] Step 3:

[0119] Text analytics

[0120] The server sends the text of the post to a natural language processing (NLP) module, which analyzes keywords and context. The NLP module uses Hugging Face's Transformers library. The input is the post text, and the output is extracted keywords that indicate the state of the cherry blossoms and autumn leaves (e.g., "in full bloom" and "beginning to change color"). For example, this includes tokenizing the text and identifying keywords by capturing the context using the BERT model.

[0121] Step 4:

[0122] Credibility assessment and manipulation judgment

[0123] The server evaluates the authenticity of the photo based on the analysis results sent from the AI ​​image analysis module. First, the authenticity evaluation evaluates the degree of match with previously posted data and assigns an authenticity score. Next, the image manipulation determination module determines whether or not the photo has been digitally altered. Specifically, the image's pixel data is analyzed using a deep learning model. The input is the analyzed photo data, and the output is an authenticity score and an alteration determination result. If alteration is confirmed, the process also applies an algorithm to return the photo to its original state.

[0124] Step 5:

[0125] Estimating progress

[0126] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at specific tourist spots. The input is the result data of image and text analysis, and the output is a quantified result of the progress of cherry blossoms and autumn leaves at each tourist spot. Specifically, this involves using AI models such as Scikit-learn and TensorFlow to aggregate data points and estimate the progress.

[0127] Step 6:

[0128] Provision to users

[0129] A user uses a smartphone app to search for the current progress of cherry blossoms or autumn leaves at a tourist spot. The device sends the user's request to a server. The server retrieves the latest information from a database, generates analysis results and related images, and provides them to the user. The input is the user's search query, and the output is analysis information and near-real-time images. For example, this includes the actions of an app developed with Flutter or React Native sending a request to a server, receiving a response from the server, and displaying it.

[0130] (Application example 1)

[0131] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0132] In recent years, the spread of social media platforms has increased the number of ways to obtain real-time seasonal landscape information for tourist destinations. However, this information is scattered, making it difficult to centrally collect and provide accurate and reliable information to users. Furthermore, when planning a trip, it is necessary to simultaneously obtain information on seasonal food and beverages and related services, but no such system exists. Given this background, this invention aims to provide a system that accurately obtains the progress of cherry blossoms and autumn leaves at tourist destinations in real time and provides information on seasonal food and beverages based on that information.

[0133] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0134] In this invention, the server includes means for collecting images and posts related to tourist destinations from SNS platforms, means for performing image analysis on the collected images to determine the progress of cherry blossoms and autumn leaves, means for analyzing the text of the collected posts to determine the progress of cherry blossoms and autumn leaves, means for evaluating the authenticity of the images and whether they have been edited and restoring the images to their original state, means for integrating the results of the image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at the tourist destination, means for providing information related to seasonal food and drink based on the estimated progress, and means for providing the estimated progress and information on related food and drink to the user. This allows users to grasp the progress of cherry blossoms and autumn leaves at tourist destinations in real time, make optimal sightseeing plans based on that information, and simultaneously obtain information on seasonal food delivery and food and drink.

[0135] "SNS Platform" means an online social networking service that enables users to share posts such as photos and text.

[0136] A "tourist destination" is a specific place or area that tourists visit.

[0137] "Image analysis" is the process of analyzing image data using computer vision techniques to identify image content and features.

[0138] "Text analytics" is the process of analyzing text data using natural language processing techniques to understand its content and meaning.

[0139] "Credibility" is an indicator of the accuracy and reliability of information or data.

[0140] "Whether or not an image or data has been altered" refers to determining whether or not the image or data has been digitally altered.

[0141] "Estimation" is the prediction of unobserved or future states based on data analysis.

[0142] "Seasonal food and beverages" are foods and beverages that are only available during a particular season.

[0143] "Information provision" means providing useful information to users in a timely manner.

[0144] "User" refers to the general user of this system.

[0145] This invention is a system that provides information on seasonal scenery and food and drink options at tourist destinations. This system collects related images and posts from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves at tourist destinations. It also aims to improve convenience for tourists by simultaneously providing information on seasonal food and drink options.

[0146] Overall system flow:

[0147] 1. Data Collection

[0148] The server periodically uses the API of the social media platform to collect posts with hashtags such as "cherry blossoms" and "autumn leaves."

[0149] Images, text, posting date and time, and location information are extracted from posts and temporarily stored in a database.

[0150] 2. Image Analysis

[0151] The server sends the stored images to an AI image analysis module, which uses deep learning frameworks such as TensorFlow and Keras to determine the progression of cherry blossoms and autumn leaves.

[0152] 3. Text Analysis

[0153] The server sends the text of the post to a natural language processing (NLP) module, which extracts context and keywords to infer the progress of the cherry blossoms and autumn leaves.

[0154] 4. Credibility assessment and manipulation judgment

[0155] The server evaluates the authenticity of the image based on the output of the AI ​​image analysis module, and then the image manipulation determination module determines whether the image has been digitally manipulated, and if so, returns it to its original state.

[0156] 5. Estimating progress

[0157] The server integrates the results of image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot.

[0158] The estimated results are provided to the user in real time, along with information related to seasonal food and drink options for specific tourist destinations.

[0159] 6. Provision to Users

[0160] Users use a smartphone app to search for information on the progress of cherry blossoms and autumn leaves at tourist spots and related food and drink options.

[0161] The terminal sends the user's request to the server, which retrieves the latest information from the database.

[0162] The server generates and provides the user with information about food and drink related to the progress.

[0163] Specific hardware and software

[0164] Hardware: Using cloud platforms (e.g., AWS, Google Cloud), servers perform large-scale data processing.

[0165] Software: TensorFlow and Keras are used for image analysis, and SpaCy and Transformers are used for NLP.

[0166] Specific examples

[0167] For example, if a user wants to know about spring tourist spots in a certain area, the user opens a smartphone app and voice-inputs "tourist spot name cherry blossom food delivery." This prompt is sent to the server, which analyzes the collected SNS data and estimates the latest status of the cherry blossoms at that tourist spot. The analysis results in the information that the cherry blossoms are in full bloom, and related seasonal foods and drinks such as "cherry blossom bento" are recommended. The user can receive a notification that "The cherry blossoms at tourist spot name are currently in full bloom. The recommended food and drink is the cherry blossom bento."

[0168] Prompt Sentence Examples

[0169] "Ueno Park Cherry Blossom Food Delivery"

[0170] "Kyoto Autumn Leaves Recommended Menu"

[0171] As described above, the present invention enables users to plan trips based on real-time information on tourist spots, and also allows users to obtain information on seasonal food and drink all at once.

[0172] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0173] Step 1:

[0174] The server periodically collects posts tagged with hashtags such as "cherry blossoms" and "autumn leaves" via the API of the social media platform. At this time, the server extracts images, text, posting date and time, and location information, and temporarily stores them in a database. The input is the posting information from the social media platform, and the output is the raw data stored in the database.

[0175] Step 2:

[0176] The server sends the images stored in the database to an AI image analysis module. The AI ​​image analysis module (using TensorFlow and Keras) analyzes the image features and determines the progress of cherry blossoms and autumn leaves. The input is images collected from social media, and the output is the progress (e.g., full bloom, beginning to change color) as a result of analysis.

[0177] Step 3:

[0178] The server sends the collected text data to a natural language processing (NLP) module, which analyzes the text's context and keywords. The NLP module uses SpaCy and Transformers to extract keywords that describe the state of the cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color"). The input is the text data collected from social media, and the output is the extracted keywords and their analysis results.

[0179] Step 4:

[0180] The server integrates the results obtained from the AI ​​image analysis module and the NLP module. Specifically, it associates the image analysis results with the text analysis results and estimates the progress of cherry blossoms and autumn leaves at tourist spots based on location information and posting date and time. The inputs are the image analysis results and text analysis results, and the output is the integrated progress data.

[0181] Step 5:

[0182] The server evaluates the authenticity of images and determines whether they have been altered. The authenticity evaluation module assigns an authenticity score to the image, and the image alteration determination module determines whether the image has been digitally altered. If alteration is confirmed, the server applies an image correction algorithm to return the image to its original state. The input is image data collected from SNS, and the output is the authenticity score and the altered image.

[0183] Step 6:

[0184] The server provides information related to seasonal food and drink based on the estimated cherry blossom and autumn foliage progress. This information is linked to the progress of each tourist spot. The input is the integrated progress data, and the output is seasonal food and drink information.

[0185] Step 7:

[0186] A user uses a smartphone app to search for information on the progress of cherry blossoms and autumn leaves at tourist spots and related food and drink options. The device sends the user's request to a server, which retrieves the latest information from a database and provides it. The input is the user's search query, and the output is a notification to the user containing the progress of the tourist spot and related food and drink options.

[0187] This allows users to plan their trips based on real-time information on tourist spots, and also allows them to obtain seasonal food delivery information at the same time.The operation of this system realizes real-time and highly accurate information provision, and provides services that meet user demand.

[0188] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0189] The present invention combines a system that collects photos and posts related to tourist spots from SNS platforms, analyzes them, and estimates and provides the progress of cherry blossoms and autumn leaves in real time, with an emotion engine that recognizes the user's emotions. The following specific forms are conceivable for carrying out the present invention.

[0190] Data collection

[0191] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts include photos, text, posting date and time, and location information, which are temporarily stored in a database.

[0192] Data analysis

[0193] Image analysis

[0194] The server sends the photos stored in the database to an AI image analysis module, which detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color. The analysis results are then sent back to the server along with metadata.

[0195] Text analytics

[0196] The server sends the text of the post to a natural language processing (NLP) module, which analyzes the context and keywords. The NLP module extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the post and analyzes the context.

[0197] Image authenticity assessment and manipulation determination

[0198] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally altered. If manipulation is confirmed, an algorithm is applied to return the photo to its original state and the results are sent back to the server.

[0199] Estimating progress

[0200] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0201] Provision to users

[0202] Users use a smartphone app to search for the current cherry blossom or autumn foliage conditions at tourist spots. The device sends this request to the server, which retrieves the latest information from the database. The server then generates analytical information and related images and provides them to the user. Users can use the app to check the local conditions based on real-time information and plan their trip.

[0203] Sentiment analysis and its applications

[0204] Emotion Engine

[0205] The server sends the user's posted content and past posted data to the emotion engine. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server. This makes it possible to understand in real time what emotions the user is currently feeling.

[0206] Emotion-based tourist destination recommendation

[0207] The server recommends customized tourist spots based on the user's emotional data. For example, if a user feels like "I want to relax," it can recommend tourist spots with a calm atmosphere. Similarly, if a user feels like "I want to get excited," it can recommend active tourist spots.

[0208] Specific examples

[0209] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, the user opens a smartphone app and searches for "Ueno Park cherry blossoms." The device sends this request to the server, which retrieves the latest cherry blossom information for Ueno Park from an existing database.

[0210] As a result of photo analysis, the information "full bloom" was obtained, so the server provides the user with the latest information, "The cherry blossoms in Ueno Park are currently in full bloom." Furthermore, the server generates near real-time images and displays them on the app, allowing the user to visually check the local situation.

[0211] In addition, if the user's emotional data is analyzed and the user feels like "I want to relax," the server can recommend quiet tourist spots and places where they can relax. This allows the user to plan a trip that suits their state of mind.

[0212] As described above, the present invention not only makes it easier for users to plan their travel destinations based on highly accurate, real-time information, but also allows users to choose tourist destinations that suit their own emotions, resulting in a more satisfying travel experience.

[0213] The processing flow will be explained below.

[0214] Step 1:

[0215] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts include photos, text, posting date and time, and location information.

[0216] Step 2:

[0217] The server extracts photo data from the collected posts and sends it to an AI image analysis module. The AI ​​image analysis module detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color. The analysis results are then sent back to the server along with metadata.

[0218] Step 3:

[0219] The server sends the text data of the post to a natural language processing (NLP) module. The NLP module extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the text and analyzes the context. The analysis results are sent back to the server.

[0220] Step 4:

[0221] The server evaluates the authenticity of the photo based on the results from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo and sends the result back to the server.

[0222] Step 5:

[0223] The server sends the photo to the image manipulation determination module to check whether the image has been manipulated. If it is found to have been manipulated, the image manipulation determination module processes the photo to return it to its original state and sends the converted image to the server.

[0224] Step 6:

[0225] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0226] Step 7:

[0227] A user uses a smartphone app to search for the current cherry blossom or autumn foliage status of a tourist spot, and the device sends this request to the server.

[0228] Step 8:

[0229] The server retrieves the latest information on tourist spots from the database in response to user requests, and generates images related to the progress of cherry blossoms and autumn leaves based on the retrieved information.

[0230] Step 9:

[0231] The server responds to the user with the current cherry blossom and autumn foliage conditions at the tourist destination and the generated images. The user can then use the smartphone app to check the local conditions based on real-time information and plan their trip.

[0232] Step 10:

[0233] The server sends data to the emotion engine to analyze the user's past posts and current emotions. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server.

[0234] Step 11:

[0235] The server recommends customized tourist spots based on the user's emotional data. For example, if the user feels like "I want to relax," the server will recommend quiet tourist spots and places where they can relax. Information about tourist spots that match the user's emotions is sent to the server.

[0236] Step 12:

[0237] Users can check tourist destination information provided through the smartphone app and create travel plans that suit their own preferences, resulting in a more satisfying travel experience.

[0238] Example 2

[0239] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0240] Conventional tourist information systems have had difficulty accurately grasping the progress of local plants (e.g., cherry blossoms and autumn leaves) in tourist destinations in real time. Furthermore, since tourist destination recommendations are not tailored to the user's emotions, satisfaction with travel plans can decrease. The purpose of this invention is to solve these problems.

[0241] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data related to tourist destinations from SNS platforms, means for performing image analysis on the collected data to determine the progress of plants, means for analyzing text in the collected data to determine the progress of plants, means for evaluating the authenticity of the image and whether it has been edited and restoring the image to its original state, means for integrating the results of the image analysis and text analysis to estimate the progress of plants at the tourist destination, means for providing the estimated progress to the user, and means for analyzing the user's emotions and recommending tourist destinations based on the results. This makes it possible to accurately grasp the progress of tourist destinations in real time and to recommend tourist destinations customized based on the user's emotions.

[0242] "SNS platform" refers to an online service that allows a large number of users to post and interact with each other.

[0243] "Data" includes information collected from social media platforms, such as photos, text, posting dates and times, and location information.

[0244] "Image analysis" refers to the process of detecting specific features from collected photographic data and determining the condition based on those features.

[0245] "Plant progress" refers to the extent to which plants such as cherry blossoms and autumn leaves have flowered or changed color.

[0246] "Text analysis" refers to the process of analyzing the context and keywords of collected text data to extract specific information.

[0247] "Authenticity" refers to the degree of confidence that a photograph or text accurately reflects the actual situation.

[0248] "Presence or absence of manipulation" refers to the state of determining whether a photograph or text has been digitally altered.

[0249] "Reverting" refers to the process of restoring digitally altered photographs or text to their original state.

[0250] "Integration" refers to combining the results of image analysis and text analysis into a single comprehensive piece of information.

[0251] "Tourist destination" refers to an area or facility that users visit for sightseeing.

[0252] "Estimation" refers to predicting information such as the progress of plants based on collected data.

[0253] "User" refers to the end user who uses this system to obtain tourist information.

[0254] "Providing" refers to displaying or notifying the user of the estimated information.

[0255] "Sentiment analysis" refers to the process of analyzing a user's posts and past data to understand their emotional state at the time.

[0256] "Recommendation" refers to selecting and presenting appropriate tourist destinations based on the user's emotional state.

[0257] The present invention combines a system that collects data related to tourist destinations from a social media platform, analyzes the data, and estimates and provides the progress of plants in real time, with an emotion engine that recognizes the user's emotions. The following specific forms are conceivable for carrying out the present invention.

[0258] Data collection

[0259] The server periodically sends requests to the API of the social media platform to collect posts containing hashtags such as "cherry blossoms" or "autumn leaves." The collected data includes photos, text, posting date and time, and location information, which is temporarily stored in a database.

[0260] Image analysis

[0261] The server sends the photos stored in the database to an AI image analysis module, which uses deep learning technology to detect plant characteristics in the photos and determine their state, such as full bloom or color. The analysis results, along with metadata, are then sent back to the server.

[0262] Text analytics

[0263] The server sends the text of the post to a natural language processing (NLP) module, which analyzes the context and keywords. The NLP module extracts keywords related to the plant's progress (e.g., "full bloom" or "beginning to change color") from the post and analyzes the context.

[0264] Image authenticity assessment and manipulation determination

[0265] The server evaluates the authenticity of the photo based on the results sent from the image analysis module. The authenticity assessment module assigns an authenticity score to the photo, and then the image manipulation determination module determines whether the photo has been digitally manipulated. If manipulation is confirmed, an algorithm is applied to restore the image to its original state and the result is sent back to the server.

[0266] Estimating progress

[0267] The server integrates the results of image and text analysis to estimate the progress of plants at each tourist spot, and uses an AI model to evaluate the current state of the tourist spot in real time, quantifying the progress, and determining the best viewing conditions.

[0268] Provision to users

[0269] Users use a smartphone app to search for the current plant status of a tourist attraction. The device sends this request to the server, which retrieves the latest information from the database. The server generates analytical information and related images and provides them to the user. Users can use the app to check the local situation based on real-time information and plan their trip.

[0270] Sentiment analysis and its applications

[0271] Emotion Engine

[0272] The server sends the user's posted content and past posted data to the emotion engine. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server. This makes it possible to understand the user's current emotions in real time.

[0273] Emotion-based tourist destination recommendation

[0274] The server recommends customized tourist spots based on the user's emotional data. For example, if a user feels like "I want to relax," the server can recommend tourist spots with a calm atmosphere. Similarly, if a user feels like "I want to get excited," the server can recommend active tourist spots.

[0275] Specific examples

[0276] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, they open a smartphone app and search for "Ueno Park cherry blossoms." The device sends this request to the server, which retrieves the latest cherry blossom information for Ueno Park from its existing database. If the photo analysis results indicate that the cherry blossoms are in full bloom, the server provides the user with the latest information: "The cherry blossoms in Ueno Park are currently in full bloom." Furthermore, the server generates near-real-time images and displays them in the app, allowing the user to visually check the current situation. Furthermore, by analyzing the user's emotional data, if the user feels like "relaxing," the server can recommend quiet tourist spots and relaxing places. This allows users to create travel plans that suit their individual needs.

[0277] The specific hardware and software used

[0278] Hardware: Servers, devices (smartphones, tablets), etc.

[0279] Software: Social media platform API, AI image analysis module, natural language processing (NLP) module, emotion engine

[0280] Other specific technology examples include TensorFlow, Google Cloud Natural Language API, MySQL, and PostgreSQL.

[0281] Example prompts for generative AI models

[0282] "I would like to know the current state of the cherry blossoms in Ueno Park in Tokyo."

[0283] Please recommend some relaxing tourist spots.

[0284] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0285] Step 1: Data collection

[0286] The server periodically sends requests to the API of the social media platform. As input data, it collects posts containing specific hashtags (e.g., "cherry blossoms" or "autumn leaves"). The collected data includes photos, text, posting date and time, and location information. The server temporarily stores the collected data in a database. Specifically, it uses the API of the social media platform to search for related posts, obtains the data in JSON format, and stores it in the database.

[0287] Step 2: Image analysis

[0288] The server sends the photo data stored in the database to the AI ​​image analysis module. By providing the photo as input data, the image analysis module uses deep learning technology to detect the characteristics of the plants in the photo. As output, the state of full bloom and color is returned along with metadata. Specifically, an image analysis model using TensorFlow is used to determine the full bloom state of the cherry blossoms, and metadata such as "75% full bloom" is returned.

[0289] Step 3: Text analysis

[0290] The server sends the collected text data of posts to a natural language processing (NLP) module. By providing the text as input data, the NLP module analyzes the context and keywords and extracts keywords related to the plant's progress (e.g., "in full bloom" or "beginning to change color"). The extracted keywords and their analysis results are returned as output. Specifically, the Google Cloud Natural Language API is used to extract the keyword "in full bloom" from the text and understand its context.

[0291] Step 4: Assess the authenticity of the image and determine whether it has been edited

[0292] The server evaluates the authenticity of the photo based on the results of image analysis. The image analysis results are provided as input data, and the authenticity assessment module assigns an authenticity score to the photo. The image manipulation assessment module then determines whether the photo has been digitally manipulated. The output provides an authenticity score and a manipulation assessment result. If manipulation is confirmed, an algorithm is applied to return the photo to its original state. Specifically, the Adobe Photoshop API is used to evaluate the authenticity of the image at 87%, detect traces of manipulation, and restore it to its original state if necessary.

[0293] Step 5: Estimating progress

[0294] The server combines the results of image analysis and text analysis. It provides both analysis results as input data and uses an AI model to evaluate the progress of the plants at the tourist destination in real time. As an output, the progress is quantified and the best viewing state is determined. Specifically, the AI ​​model combines past and present data to predict the progress of the plants as "85% full bloom."

[0295] Step 6: Inform users

[0296] A user uses a smartphone app to search for the current plant progress at a tourist spot. The device sends this request to the server. The server retrieves the latest information from the database, generates analysis information and related images, and provides them to the user. The system receives the user's request as input data and provides the analysis results and images as output. Specifically, when a user searches for "Ueno Park cherry blossoms," the server responds with "Currently 85% in full bloom," and also displays an image of the actual location.

[0297] Step 7: Sentiment analysis and its applications

[0298] The server sends the user's posted content and past data to the emotion engine. Text data is provided as input, and the emotion engine uses NLP technology to analyze the user's emotions. The analysis results, or emotional state, are sent back to the server as output. Based on the results, customized tourist destination recommendations are made. For example, if the user feels like "relaxing," tranquil tourist destinations are recommended. Specifically, if the user's emotion is determined to be "relaxed," the server will recommend quiet tourist destinations.

[0299] (Application example 2)

[0300] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0301] Conventional tourist destination information systems have limitations in terms of real-timeness and accuracy, making it particularly difficult to grasp seasonal information such as the progress of cherry blossoms and autumn leaves. Furthermore, they lacked the ability to recommend tourist destinations based on user emotions and to link with real-world store information, preventing user satisfaction. There is a need for a system that can solve these problems, provide users with highly accurate tourist information in real time, and also recommend tourist destinations based on emotions and provide information on nearby real-world stores.

[0302] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting photos and posted content related to tourist destinations from SNS platforms, means for performing image analysis on the collected photos to determine the progress of cherry blossoms and autumn leaves, means for analyzing the text of the collected posted content to determine the progress of cherry blossoms and autumn leaves, means for evaluating the authenticity of the image and whether it has been edited and restoring the image to its original state, means for integrating the results of the image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at the tourist destination, means for providing the estimated progress to the user, means for analyzing the user's emotions and recommending tourist destinations based on the emotions, and means for providing information on brick-and-mortar stores near the tourist destination. This allows users to not only obtain highly accurate tourist information in real time, but also receive recommendations for tourist destinations and information on brick-and-mortar stores in the vicinity that match their emotions at any given time.

[0303] "SNS Platform" means an online platform that provides social networking services, where users can post content such as photos and text to share with other users.

[0304] A "tourist destination" is a specific geographical location or region that tourists visit, and includes natural scenery, cultural assets, leisure facilities, etc.

[0305] A "photograph" is a still image recorded using a photographic device such as a camera, which visually captures the state and characteristics of the tourist destination.

[0306] "Posted content" refers to all content, including text, images, and videos, posted by users on social media platforms.

[0307] "Image analysis" is the process of analyzing the content of an image using computer vision techniques to detect and classify specific objects or features.

[0308] "Progression of cherry blossoms and autumn leaves" refers to the state and stage of progress of plants that change with the seasons, such as cherry blossoms blooming and reaching full bloom, and the color of autumn leaves.

[0309] "Text analysis" is the process of analyzing text data using natural language processing technology to evaluate and extract its content, context, sentiment, etc.

[0310] "Credibility" is the concept of assessing whether collected data or information is accurate and truthful.

[0311] "Manifold" refers to whether an image or text has been digitally altered or modified.

[0312] "Progress estimation" is the process of numerically or qualitatively assessing the current state of cherry blossoms and autumn leaves at tourist spots based on analyzed image and text data.

[0313] "Providing to the user" refers to the act of sending the analyzed information to the user's device and displaying it in visual or text format.

[0314] "Analyzing user emotions" is the process of assessing users' emotions and moods from social media and other digital content using natural language processing.

[0315] "Tourist destination recommendation" is the act of selecting and suggesting appropriate tourist destinations and spots to visit based on the user's current emotions and preferences.

[0316] "Information about nearby physical stores" refers to information that includes details about commercial facilities and service providers (e.g., cafes, souvenir shops, etc.) located around tourist destinations.

[0317] This invention is a system that collects photos and posts related to tourist spots from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves.It also recognizes the user's emotions and provides recommendations on tourist spots and brick-and-mortar store information based on those emotions.

[0318] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts include photos, text, posting date and time, and location information, which are temporarily stored in a database.

[0319] The server then sends the photos stored in the database to an AI image analysis module, which detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color. The analysis results are then sent back to the server along with metadata.

[0320] The server then sends the text of the post to a natural language processing (NLP) module for context and keyword analysis. The NLP module extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the post and analyzes the context.

[0321] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally altered. If manipulation is confirmed, an algorithm is applied to return the photo to its original state and the results are sent back to the server.

[0322] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0323] Users use a smartphone app to search for the current cherry blossom or autumn foliage conditions at tourist spots. The device sends this request to the server, which retrieves the latest information from the database. The server then generates analytical information and related images and provides them to the user. Users can use the app to check the local conditions based on real-time information and plan their trip.

[0324] The server sends the user's posted content and past posted data to the emotion engine. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server. This makes it possible to understand in real time what emotions the user is currently feeling.

[0325] The server recommends customized tourist spots based on the user's emotional data. For example, if a user feels like "I want to relax," it can recommend tourist spots with a calm atmosphere. Similarly, if a user feels like "I want to get excited," it can recommend active tourist spots.

[0326] This will provide users with highly accurate, real-time information, making it easier to plan trips, and will also enable recommendations of tourist spots based on the user's emotions. Furthermore, real-time information on nearby brick-and-mortar stores will also be provided, improving the overall tourist experience.

[0327] For example, if a user searches for "Ueno Park cherry blossoms," the system will provide real-time information on the current cherry blossom conditions in Ueno Park. Along with information such as "They're in full bloom," it will also display information on nearby cafes and souvenir shops. If the user feels like "I want to relax," the system will recommend quiet tourist spots and cafes.

[0328] Examples of prompts include:

[0329] "Please tell me the latest information about the cherry blossoms in Ueno Park and places to relax in the area."

[0330] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0331] Step 1:

[0332] The server sends a request to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The input is the social media platform's API URL and a specific hashtag, and the output is post data including photos, text, posting date and time, and location information. This post data is temporarily stored in a database.

[0333] Step 2:

[0334] The server sends the photos stored in the database to the AI ​​image analysis module. The input is the photo data, and the output is metadata including the results of detecting the characteristics of cherry blossoms and autumn leaves and their status (e.g., full bloom, color change). The AI ​​image analysis module analyzes the photo and sends the results back to the server.

[0335] Step 3:

[0336] The server sends the text of the post to a natural language processing (NLP) module to analyze the context and keywords. The input is the text of the SNS post, and the output is keywords and context analysis results related to the progress of cherry blossoms and autumn leaves (e.g., "in full bloom" and "beginning to change color").

[0337] Step 4:

[0338] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. The input is the result of the AI ​​image analysis, and the output is an authenticity score. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally manipulated. The input is the photo data for manipulation determination, and the output is a manipulation score. If manipulation is confirmed, an algorithm is applied to return the photo to its original state, and the photo is sent back to the server.

[0339] Step 5:

[0340] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The input is the results of image analysis and text analysis, and the output is evaluation data that quantifies the progress of each tourist spot. The AI ​​model evaluates the current state of tourist spots in real time and determines the best viewing conditions.

[0341] Step 6:

[0342] A user uses a smartphone app to search for the current cherry blossom and autumn foliage conditions at a tourist spot. The input is the user's search query (e.g., "Ueno Park cherry blossoms"), and the output is the latest information about the tourist spot (e.g., "In full bloom" and related images). The device sends the request to the server, which retrieves the latest information from the database, generates analytical information and related images, and provides them to the user.

[0343] Step 7:

[0344] The server sends the user's posted content and past posted data to the emotion engine. The input is the user's posted data, and the output is the result of analyzing the user's emotions. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server.

[0345] Step 8:

[0346] The server recommends customized tourist spots based on the user's emotional data. The input is the user's emotional data, and the output is a list of recommended tourist spots. For example, if the user feels like "I want to relax," tourist spots with a calm atmosphere will be recommended. Similarly, if the user feels like "I want to get excited," tourist spots with an active atmosphere will be recommended.

[0347] Step 9:

[0348] The server provides information about brick-and-mortar stores around tourist spots. The input is the location information of the tourist spot and filtering conditions based on the user's interests, and the output is information about brick-and-mortar stores such as nearby cafes and souvenir shops. Users can obtain this information through the app and check it in real time.

[0349] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0350] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0351] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0352] [Second embodiment]

[0353] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0354] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0355] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0356] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0357] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0358] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0359] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0360] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0361] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0362] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0363] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0364] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0365] The present invention relates to a system that collects photos and posts related to tourist spots from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves. The following specific embodiments are conceivable for carrying out the present invention.

[0366] Data collection

[0367] The server periodically collects posts with hashtags such as "cherry blossoms" and "autumn leaves" via the SNS platform API. Photos, text, posting date and time, and location information are extracted from the posts and temporarily stored in a database.

[0368] Data analysis

[0369] Image analysis

[0370] The server then sends the photos stored in the database to an AI image analysis module, which analyzes the image features to determine the state of cherry blossom blooming and the progress of autumn leaves. The analysis results are then sent back to the server along with metadata.

[0371] Text analytics

[0372] The server sends the text of the post to a natural language processing (NLP) module, which analyzes the context and keywords. The NLP module extracts keywords (e.g., "full bloom" or "beginning to change color") that predict the state of the cherry blossoms and autumn leaves from the post and returns the results to the server.

[0373] Image authenticity assessment and manipulation determination

[0374] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally altered. If manipulation is confirmed, an algorithm is applied to return the photo to its original state and the results are sent back to the server.

[0375] Estimating progress

[0376] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0377] Provision to users

[0378] Users use a smartphone app to search for the current cherry blossom and autumn foliage conditions at tourist spots. The device sends the user's request to a server, which retrieves the latest information from a database. The server then generates analytical information and related images and provides them to the user. Users can then use the app to plan their trip based on real-time information.

[0379] Specific examples

[0380] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, the user opens the smartphone app and searches for "Ueno Park cherry blossoms." The device sends this request to the server, which retrieves the latest cherry blossom information for Ueno Park from an existing database.

[0381] As a result of photo analysis, the information "full bloom" was obtained, so the server provides the user with the latest information, "The cherry blossoms in Ueno Park are currently in full bloom." Furthermore, the server generates near real-time images and displays them on the app, allowing the user to visually check the local situation.

[0382] As described above, the present invention makes it easier for users to plan their travel destinations based on highly accurate, real-time information.

[0383] The processing flow will be explained below.

[0384] Step 1:

[0385] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts contain photos, text, posting date and time, and location information.

[0386] Step 2:

[0387] The server extracts photo data from the collected posts and sends it to an image analysis module, which detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color.

[0388] Step 3:

[0389] The server sends the text data of the post to a natural language processing (NLP) module, which extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the text and analyzes the context.

[0390] Step 4:

[0391] The server evaluates the authenticity of the photo based on the results from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo and sends the result back to the server.

[0392] Step 5:

[0393] The server sends the photo to the image manipulation determination module to check whether the image has been manipulated. If manipulation is confirmed, the image manipulation determination module processes the photo to return it to its original state and sends the converted image to the server.

[0394] Step 6:

[0395] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0396] Step 7:

[0397] A user uses a smartphone app to search for the current cherry blossom or autumn foliage status of a tourist spot, and the device sends this request to the server.

[0398] Step 8:

[0399] The server retrieves the latest information on tourist spots from the database in response to user requests. The retrieved information includes estimated cherry blossom and autumn foliage progress and related images.

[0400] Step 9:

[0401] The server responds to users with images related to the current cherry blossom and autumn foliage conditions at tourist destinations, allowing them to use a smartphone app to check the local conditions based on real-time information and plan their trip.

[0402] Example 1

[0403] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0404] In today's world, there is a need to effectively utilize the vast amount of information available on social media to understand the progress of cherry blossoms and autumn leaves at tourist spots in real time. However, there are issues such as the effort required for users to manually search social media to gather information, the difficulty of assessing the credibility of the information, and the risk of photo manipulation. The challenge is to solve these issues and provide more accurate and reliable information in real time.

[0405] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0406] In this invention, the server includes means for collecting photos and posts related to tourist destinations from SNS platforms, means for determining the progress of cherry blossoms and autumn leaves in the collected photos using an AI image analysis module, means for analyzing the text of the collected posts using a natural language processing module to determine the progress of cherry blossoms and autumn leaves, means for evaluating the authenticity of the images and whether they have been edited using an AI image analysis module and an image manipulation determination module and restoring the images to their original state if necessary, means for integrating the results of the image analysis and text analysis and using an AI model to estimate the progress of cherry blossoms and autumn leaves at tourist destinations, and means for providing the estimated progress to users. This allows users to easily obtain accurate and reliable progress information on cherry blossoms and autumn leaves in real time, enabling them to efficiently plan their sightseeing trips.

[0407] "SNS Platform" means an online service that enables users to post photos and text and share those posts with other users.

[0408] The "image analysis module" is a program that uses AI technology to analyze the characteristics of a photo and automatically determine its content.

[0409] A "natural language processing module" is a technology for understanding and analyzing text data, and is a program capable of extracting specific keywords and context.

[0410] "Authenticity assessment" is the process of determining whether a photograph or text accurately reflects a real-world situation.

[0411] The "image manipulation determination module" is a technology for determining whether an image has been digitally manipulated.

[0412] An "AI model" is an artificial intelligence algorithm and architecture that is trained to perform a specific task based on specified input data.

[0413] "Real-time" means providing data or information with a very short time delay, so that it is nearly simultaneous with real time.

[0414] A "tourist destination" is a place or area where visitors gather to see natural elements such as cherry blossoms or autumn leaves.

[0415] "Progress" refers to the current stage of development of cherry blossoms or autumn leaves and the degree of their peak viewing.

[0416] "Text analysis" is the process by which a computer analyzes the content of a sentence or document to extract specific information.

[0417] A "post" is content such as a photo or text that a user publishes on a social media platform.

[0418] This invention is a system that collects photos and posts related to tourist spots from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves.To implement this system, a combination of multiple hardware and software components is used.

[0419] The main components of the system are:

[0420] 1. Data Collection

[0421] The server periodically collects posts tagged with hashtags such as "cherry blossoms" and "autumn leaves" using the APIs of social media platforms. Specifically, it does this through the Twitter API and Instagram API. The server extracts photos, text, posting date and time, and location information from the collected posts and temporarily stores this data in a database. For example, it manages the data using MySQL or PostgreSQL.

[0422] 2. Data Analysis

[0423] Image analysis:

[0424] The server sends the photos stored in the database to an AI image analysis module, which uses deep learning libraries such as TensorFlow and PyTorch, to identify the state of cherry blossom blooming and the progress of autumn leaves and generate corresponding metadata.

[0425] Text analysis:

[0426] The server sends the text of the post to a natural language processing (NLP) module that analyzes the context and keywords. The NLP module uses Hugging Face's Transformers library to extract keywords such as "full bloom" and "beginning to change color" from the text and returns the results to the server.

[0427] 3. Image authenticity assessment and manipulation determination

[0428] The server evaluates the authenticity of the photo based on the analysis results sent from the AI ​​image analysis module. This process uses technology that compares it with previously posted data and evaluates the degree of pattern matching. The image manipulation detection module also determines whether the photo has been digitally altered. Specifically, it checks for unnatural color changes and pixel consistency, and if any abnormalities are found, it applies an algorithm to correct the image.

[0429] 4. Estimating progress

[0430] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at specific tourist spots. It uses AI models such as Scikit-learn and TensorFlow to quantify the condition of each tourist spot and determine the best time to see them.

[0431] 5. Provision to Users

[0432] A user uses a smartphone application to search for the current cherry blossom or autumn foliage conditions at a tourist spot. The device sends the user's request to a server, which retrieves the latest information from a database. The server generates analytical information and related images and provides them to the user. The user can then plan their trip based on the information displayed within the app.

[0433] Specific examples

[0434] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, they open the smartphone app and search for "Ueno Park cherry blossoms." The device then sends this search request to the server. The server retrieves the latest cherry blossom information for Ueno Park from an existing database, analyzes the information, and generates near-real-time images to provide to the user. Based on this, users can visually check the local conditions and efficiently plan their trip.

[0435] Prompt Sentence Examples

[0436] For example, a possible prompt for a generative AI model might be:

[0437] "Please tell me the steps to analyze the progress of cherry blossoms and autumn leaves at tourist spots from social media posts."

[0438] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0439] Step 1:

[0440] Collecting social media posts

[0441] The server periodically collects posts tagged with specific hashtags, such as "cherry blossoms" or "autumn leaves," using the API of the social media platform. Specifically, it uses the Twitter API or Instagram API. The server extracts photos, text, posting date and time, and location information from the posts. The input is the raw posting data obtained from the social media API, and the output is a database entry with the extracted photos, text, posting date and time, and location information. This data is temporarily stored in a database. For example, it includes operations to retrieve data in JSON format and save each field to an SQL database.

[0442] Step 2:

[0443] Image analysis

[0444] The server sends the photos stored in the database to an AI image analysis module. Specifically, it uses a deep learning model using TensorFlow and PyTorch. The input is photo data retrieved from the database, and the output generates metadata indicating the state of cherry blossom blooming and the progress of autumn leaves. For example, it converts images into feature vectors and classifies them as "blooming" or "full bloom" based on those vectors.

[0445] Step 3:

[0446] Text analytics

[0447] The server sends the text of the post to a natural language processing (NLP) module, which analyzes keywords and context. The NLP module uses Hugging Face's Transformers library. The input is the post text, and the output is extracted keywords that indicate the state of the cherry blossoms and autumn leaves (e.g., "in full bloom" and "beginning to change color"). For example, this includes tokenizing the text and identifying keywords by capturing the context using the BERT model.

[0448] Step 4:

[0449] Credibility assessment and manipulation judgment

[0450] The server evaluates the authenticity of the photo based on the analysis results sent from the AI ​​image analysis module. First, the authenticity evaluation evaluates the degree of match with previously posted data and assigns an authenticity score. Next, the image manipulation determination module determines whether or not the photo has been digitally altered. Specifically, the image's pixel data is analyzed using a deep learning model. The input is the analyzed photo data, and the output is an authenticity score and an alteration determination result. If alteration is confirmed, the process also applies an algorithm to return the photo to its original state.

[0451] Step 5:

[0452] Estimating progress

[0453] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at specific tourist spots. The input is the result data of image and text analysis, and the output is a quantified result of the progress of cherry blossoms and autumn leaves at each tourist spot. Specifically, this involves using AI models such as Scikit-learn and TensorFlow to aggregate data points and estimate the progress.

[0454] Step 6:

[0455] Provision to users

[0456] A user uses a smartphone app to search for the current progress of cherry blossoms or autumn leaves at a tourist spot. The device sends the user's request to a server. The server retrieves the latest information from a database, generates analysis results and related images, and provides them to the user. The input is the user's search query, and the output is analysis information and near-real-time images. For example, this includes the actions of an app developed with Flutter or React Native sending a request to a server, receiving a response from the server, and displaying it.

[0457] (Application example 1)

[0458] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0459] In recent years, the spread of social media platforms has increased the number of ways to obtain real-time seasonal landscape information for tourist destinations. However, this information is scattered, making it difficult to centrally collect and provide accurate and reliable information to users. Furthermore, when planning a trip, it is necessary to simultaneously obtain information on seasonal food and beverages and related services, but no such system exists. Given this background, this invention aims to provide a system that accurately obtains the progress of cherry blossoms and autumn leaves at tourist destinations in real time and provides information on seasonal food and beverages based on that information.

[0460] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0461] In this invention, the server includes means for collecting images and posts related to tourist destinations from SNS platforms, means for performing image analysis on the collected images to determine the progress of cherry blossoms and autumn leaves, means for analyzing the text of the collected posts to determine the progress of cherry blossoms and autumn leaves, means for evaluating the authenticity of the images and whether they have been edited and restoring the images to their original state, means for integrating the results of the image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at the tourist destination, means for providing information related to seasonal food and drink based on the estimated progress, and means for providing the estimated progress and information on related food and drink to the user. This allows users to grasp the progress of cherry blossoms and autumn leaves at tourist destinations in real time, make optimal sightseeing plans based on that information, and simultaneously obtain information on seasonal food delivery and food and drink.

[0462] "SNS Platform" means an online social networking service that enables users to share posts such as photos and text.

[0463] A "tourist destination" is a specific place or area that tourists visit.

[0464] "Image analysis" is the process of analyzing image data using computer vision techniques to identify image content and features.

[0465] "Text analytics" is the process of analyzing text data using natural language processing techniques to understand its content and meaning.

[0466] "Credibility" is an indicator of the accuracy and reliability of information or data.

[0467] "Whether or not an image or data has been altered" refers to determining whether or not the image or data has been digitally altered.

[0468] "Estimation" is the prediction of unobserved or future states based on data analysis.

[0469] "Seasonal food and beverages" are foods and beverages that are only available during a particular season.

[0470] "Information provision" means providing useful information to users in a timely manner.

[0471] "User" refers to the general user of this system.

[0472] This invention is a system that provides information on seasonal scenery and food and drink options at tourist destinations. This system collects related images and posts from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves at tourist destinations. It also aims to improve convenience for tourists by simultaneously providing information on seasonal food and drink options.

[0473] Overall system flow:

[0474] 1. Data Collection

[0475] The server periodically uses the API of the social media platform to collect posts with hashtags such as "cherry blossoms" and "autumn leaves."

[0476] Images, text, posting date and time, and location information are extracted from posts and temporarily stored in a database.

[0477] 2. Image Analysis

[0478] The server sends the stored images to an AI image analysis module, which uses deep learning frameworks such as TensorFlow and Keras to determine the progression of cherry blossoms and autumn leaves.

[0479] 3. Text Analysis

[0480] The server sends the text of the post to a natural language processing (NLP) module, which extracts context and keywords to infer the progress of the cherry blossoms and autumn leaves.

[0481] 4. Credibility assessment and manipulation judgment

[0482] The server evaluates the authenticity of the image based on the output of the AI ​​image analysis module, and then the image manipulation determination module determines whether the image has been digitally manipulated, and if so, returns it to its original state.

[0483] 5. Estimating progress

[0484] The server integrates the results of image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot.

[0485] The estimated results are provided to the user in real time, along with information related to seasonal food and drink options for specific tourist destinations.

[0486] 6. Provision to Users

[0487] Users use a smartphone app to search for information on the progress of cherry blossoms and autumn leaves at tourist spots and related food and drink options.

[0488] The terminal sends the user's request to the server, which retrieves the latest information from the database.

[0489] The server generates and provides the user with information about food and drink related to the progress.

[0490] Specific hardware and software

[0491] Hardware: Using cloud platforms (e.g., AWS, Google Cloud), servers perform large-scale data processing.

[0492] Software: TensorFlow and Keras are used for image analysis, and SpaCy and Transformers are used for NLP.

[0493] Specific examples

[0494] For example, if a user wants to know about spring tourist spots in a certain area, the user opens a smartphone app and voice-inputs "tourist spot name cherry blossom food delivery." This prompt is sent to the server, which analyzes the collected SNS data and estimates the latest status of the cherry blossoms at that tourist spot. The analysis results in the information that the cherry blossoms are in full bloom, and related seasonal foods and drinks such as "cherry blossom bento" are recommended. The user can receive a notification that "The cherry blossoms at tourist spot name are currently in full bloom. The recommended food and drink is the cherry blossom bento."

[0495] Prompt Sentence Examples

[0496] "Ueno Park Cherry Blossom Food Delivery"

[0497] "Kyoto Autumn Leaves Recommended Menu"

[0498] As described above, the present invention enables users to plan trips based on real-time information on tourist spots, and also allows users to obtain information on seasonal food and drink all at once.

[0499] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0500] Step 1:

[0501] The server periodically collects posts tagged with hashtags such as "cherry blossoms" and "autumn leaves" via the API of the social media platform. At this time, the server extracts images, text, posting date and time, and location information, and temporarily stores them in a database. The input is the posting information from the social media platform, and the output is the raw data stored in the database.

[0502] Step 2:

[0503] The server sends the images stored in the database to an AI image analysis module. The AI ​​image analysis module (using TensorFlow and Keras) analyzes the image features and determines the progress of cherry blossoms and autumn leaves. The input is images collected from social media, and the output is the progress (e.g., full bloom, beginning to change color) as a result of analysis.

[0504] Step 3:

[0505] The server sends the collected text data to a natural language processing (NLP) module, which analyzes the text's context and keywords. The NLP module uses SpaCy and Transformers to extract keywords that describe the state of the cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color"). The input is the text data collected from social media, and the output is the extracted keywords and their analysis results.

[0506] Step 4:

[0507] The server integrates the results obtained from the AI ​​image analysis module and the NLP module. Specifically, it associates the image analysis results with the text analysis results and estimates the progress of cherry blossoms and autumn leaves at tourist spots based on location information and posting date and time. The inputs are the image analysis results and text analysis results, and the output is the integrated progress data.

[0508] Step 5:

[0509] The server evaluates the authenticity of images and determines whether they have been altered. The authenticity evaluation module assigns an authenticity score to the image, and the image alteration determination module determines whether the image has been digitally altered. If alteration is confirmed, the server applies an image correction algorithm to return the image to its original state. The input is image data collected from SNS, and the output is the authenticity score and the altered image.

[0510] Step 6:

[0511] The server provides information related to seasonal food and drink based on the estimated cherry blossom and autumn foliage progress. This information is linked to the progress of each tourist spot. The input is the integrated progress data, and the output is seasonal food and drink information.

[0512] Step 7:

[0513] A user uses a smartphone app to search for information on the progress of cherry blossoms and autumn leaves at tourist spots and related food and drink options. The device sends the user's request to a server, which retrieves the latest information from a database and provides it. The input is the user's search query, and the output is a notification to the user containing the progress of the tourist spot and related food and drink options.

[0514] This allows users to plan their trips based on real-time information on tourist spots, and also allows them to obtain seasonal food delivery information at the same time.The operation of this system realizes real-time and highly accurate information provision, and provides services that meet user demand.

[0515] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0516] The present invention combines a system that collects photos and posts related to tourist spots from SNS platforms, analyzes them, and estimates and provides the progress of cherry blossoms and autumn leaves in real time, with an emotion engine that recognizes the user's emotions. The following specific forms are conceivable for carrying out the present invention.

[0517] Data collection

[0518] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts include photos, text, posting date and time, and location information, which are temporarily stored in a database.

[0519] Data analysis

[0520] Image analysis

[0521] The server sends the photos stored in the database to an AI image analysis module, which detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color. The analysis results are then sent back to the server along with metadata.

[0522] Text analytics

[0523] The server sends the text of the post to a natural language processing (NLP) module, which analyzes the context and keywords. The NLP module extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the post and analyzes the context.

[0524] Image authenticity assessment and manipulation determination

[0525] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally altered. If manipulation is confirmed, an algorithm is applied to return the photo to its original state and the results are sent back to the server.

[0526] Estimating progress

[0527] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0528] Provision to users

[0529] Users use a smartphone app to search for the current cherry blossom or autumn foliage conditions at tourist spots. The device sends this request to the server, which retrieves the latest information from the database. The server then generates analytical information and related images and provides them to the user. Users can use the app to check the local conditions based on real-time information and plan their trip.

[0530] Sentiment analysis and its applications

[0531] Emotion Engine

[0532] The server sends the user's posted content and past posted data to the emotion engine. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server. This makes it possible to understand in real time what emotions the user is currently feeling.

[0533] Emotion-based tourist destination recommendation

[0534] The server recommends customized tourist spots based on the user's emotional data. For example, if a user feels like "I want to relax," it can recommend tourist spots with a calm atmosphere. Similarly, if a user feels like "I want to get excited," it can recommend active tourist spots.

[0535] Specific examples

[0536] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, the user opens a smartphone app and searches for "Ueno Park cherry blossoms." The device sends this request to the server, which retrieves the latest cherry blossom information for Ueno Park from an existing database.

[0537] As a result of photo analysis, the information "full bloom" was obtained, so the server provides the user with the latest information, "The cherry blossoms in Ueno Park are currently in full bloom." Furthermore, the server generates near real-time images and displays them on the app, allowing the user to visually check the local situation.

[0538] In addition, if the user's emotional data is analyzed and the user feels like "I want to relax," the server can recommend quiet tourist spots and places where they can relax. This allows the user to plan a trip that suits their state of mind.

[0539] As described above, the present invention not only makes it easier for users to plan their travel destinations based on highly accurate, real-time information, but also allows users to choose tourist destinations that suit their own emotions, resulting in a more satisfying travel experience.

[0540] The processing flow will be explained below.

[0541] Step 1:

[0542] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts include photos, text, posting date and time, and location information.

[0543] Step 2:

[0544] The server extracts photo data from the collected posts and sends it to an AI image analysis module. The AI ​​image analysis module detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color. The analysis results are then sent back to the server along with metadata.

[0545] Step 3:

[0546] The server sends the text data of the post to a natural language processing (NLP) module. The NLP module extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the text and analyzes the context. The analysis results are sent back to the server.

[0547] Step 4:

[0548] The server evaluates the authenticity of the photo based on the results from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo and sends the result back to the server.

[0549] Step 5:

[0550] The server sends the photo to the image manipulation determination module to check whether the image has been manipulated. If it is found to have been manipulated, the image manipulation determination module processes the photo to return it to its original state and sends the converted image to the server.

[0551] Step 6:

[0552] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0553] Step 7:

[0554] A user uses a smartphone app to search for the current cherry blossom or autumn foliage status of a tourist spot, and the device sends this request to the server.

[0555] Step 8:

[0556] The server retrieves the latest information on tourist spots from the database in response to user requests, and generates images related to the progress of cherry blossoms and autumn leaves based on the retrieved information.

[0557] Step 9:

[0558] The server responds to the user with the current cherry blossom and autumn foliage conditions at the tourist destination and the generated images. The user can then use the smartphone app to check the local conditions based on real-time information and plan their trip.

[0559] Step 10:

[0560] The server sends data to the emotion engine to analyze the user's past posts and current emotions. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server.

[0561] Step 11:

[0562] The server recommends customized tourist spots based on the user's emotional data. For example, if the user feels like "I want to relax," the server will recommend quiet tourist spots and places where they can relax. Information about tourist spots that match the user's emotions is sent to the server.

[0563] Step 12:

[0564] Users can check tourist destination information provided through the smartphone app and create travel plans that suit their own preferences, resulting in a more satisfying travel experience.

[0565] Example 2

[0566] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0567] Conventional tourist information systems have had difficulty accurately grasping the progress of local plants (e.g., cherry blossoms and autumn leaves) in tourist destinations in real time. Furthermore, since tourist destination recommendations are not tailored to the user's emotions, satisfaction with travel plans can decrease. The purpose of this invention is to solve these problems.

[0568] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data related to tourist destinations from SNS platforms, means for performing image analysis on the collected data to determine the progress of plants, means for analyzing text in the collected data to determine the progress of plants, means for evaluating the authenticity of the image and whether it has been edited and restoring the image to its original state, means for integrating the results of the image analysis and text analysis to estimate the progress of plants at the tourist destination, means for providing the estimated progress to the user, and means for analyzing the user's emotions and recommending tourist destinations based on the results. This makes it possible to accurately grasp the progress of tourist destinations in real time and to recommend tourist destinations customized based on the user's emotions.

[0569] "SNS platform" refers to an online service that allows a large number of users to post and interact with each other.

[0570] "Data" includes information collected from social media platforms, such as photos, text, posting dates and times, and location information.

[0571] "Image analysis" refers to the process of detecting specific features from collected photographic data and determining the condition based on those features.

[0572] "Plant progress" refers to the extent to which plants such as cherry blossoms and autumn leaves have flowered or changed color.

[0573] "Text analysis" refers to the process of analyzing the context and keywords of collected text data to extract specific information.

[0574] "Authenticity" refers to the degree of confidence that a photograph or text accurately reflects the actual situation.

[0575] "Presence or absence of manipulation" refers to the state of determining whether a photograph or text has been digitally altered.

[0576] "Reverting" refers to the process of restoring digitally altered photographs or text to their original state.

[0577] "Integration" refers to combining the results of image analysis and text analysis into a single comprehensive piece of information.

[0578] "Tourist destination" refers to an area or facility that users visit for sightseeing.

[0579] "Estimation" refers to predicting information such as the progress of plants based on collected data.

[0580] "User" refers to the end user who uses this system to obtain tourist information.

[0581] "Providing" refers to displaying or notifying the user of the estimated information.

[0582] "Sentiment analysis" refers to the process of analyzing a user's posts and past data to understand their emotional state at the time.

[0583] "Recommendation" refers to selecting and presenting appropriate tourist destinations based on the user's emotional state.

[0584] The present invention combines a system that collects data related to tourist destinations from SNS platforms, analyzes the data, and estimates and provides the progress of plants in real time, with an emotion engine that recognizes the user's emotions. The following specific forms are conceivable for carrying out the present invention.

[0585] Data collection

[0586] The server periodically sends requests to the API of the social media platform to collect posts containing hashtags such as "cherry blossoms" or "autumn leaves." The collected data includes photos, text, posting date and time, and location information, which is temporarily stored in a database.

[0587] Image analysis

[0588] The server sends the photos stored in the database to an AI image analysis module, which uses deep learning technology to detect plant characteristics in the photos and determine their condition, such as full bloom or color. The analysis results, along with metadata, are then sent back to the server.

[0589] Text analytics

[0590] The server sends the text of the post to a natural language processing (NLP) module, which analyzes the context and keywords. The NLP module extracts keywords related to the plant's progress (e.g., "full bloom" or "beginning to change color") from the post and analyzes the context.

[0591] Image authenticity assessment and manipulation determination

[0592] The server evaluates the authenticity of the photo based on the results sent from the image analysis module. The authenticity assessment module assigns an authenticity score to the photo, and then the image manipulation determination module determines whether the photo has been digitally manipulated. If manipulation is confirmed, an algorithm is applied to restore the image to its original state and the result is sent back to the server.

[0593] Estimating progress

[0594] The server integrates the results of image and text analysis to estimate the progress of plants at each tourist spot, and uses an AI model to evaluate the current state of the tourist spot in real time, quantifying the progress, and determining the best viewing conditions.

[0595] Provision to users

[0596] Users use a smartphone app to search for the current plant status of a tourist attraction. The device sends this request to the server, which retrieves the latest information from the database. The server generates analytical information and related images and provides them to the user. Users can use the app to check the local situation based on real-time information and plan their trip.

[0597] Sentiment analysis and its applications

[0598] Emotion Engine

[0599] The server sends the user's posted content and past posted data to the emotion engine. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server. This makes it possible to understand the user's current emotions in real time.

[0600] Emotion-based tourist destination recommendation

[0601] The server recommends customized tourist spots based on the user's emotional data. For example, if a user feels like "I want to relax," the server can recommend tourist spots with a calm atmosphere. Similarly, if a user feels like "I want to get excited," the server can recommend active tourist spots.

[0602] Specific examples

[0603] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, they open a smartphone app and search for "Ueno Park cherry blossoms." The device sends this request to the server, which retrieves the latest cherry blossom information for Ueno Park from its existing database. If the photo analysis results indicate that the cherry blossoms are in full bloom, the server provides the user with the latest information: "The cherry blossoms in Ueno Park are currently in full bloom." Furthermore, the server generates near-real-time images and displays them in the app, allowing the user to visually check the current situation. Furthermore, by analyzing the user's emotional data, if the user feels like "relaxing," the server can recommend quiet tourist spots and relaxing places. This allows users to create travel plans that suit their individual needs.

[0604] The specific hardware and software used

[0605] Hardware: Servers, devices (smartphones, tablets), etc.

[0606] Software: Social media platform API, AI image analysis module, natural language processing (NLP) module, emotion engine

[0607] Other specific technology examples include TensorFlow, Google Cloud Natural Language API, MySQL, and PostgreSQL.

[0608] Example prompts for generative AI models

[0609] "I would like to know the current state of the cherry blossoms in Ueno Park in Tokyo."

[0610] Please recommend some relaxing tourist spots.

[0611] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0612] Step 1: Data collection

[0613] The server periodically sends requests to the API of the social media platform. As input data, it collects posts containing specific hashtags (e.g., "cherry blossoms" or "autumn leaves"). The collected data includes photos, text, posting date and time, and location information. The server temporarily stores the collected data in a database. Specifically, it uses the API of the social media platform to search for related posts, obtains the data in JSON format, and stores it in the database.

[0614] Step 2: Image analysis

[0615] The server sends the photo data stored in the database to the AI ​​image analysis module. By providing the photo as input data, the image analysis module uses deep learning technology to detect the characteristics of the plants in the photo. As output, the state of full bloom and color is returned along with metadata. Specifically, an image analysis model using TensorFlow is used to determine the full bloom state of the cherry blossoms, and metadata such as "75% full bloom" is returned.

[0616] Step 3: Text analysis

[0617] The server sends the collected text data of posts to a natural language processing (NLP) module. By providing the text as input data, the NLP module analyzes the context and keywords and extracts keywords related to the plant's progress (e.g., "in full bloom" or "beginning to change color"). The extracted keywords and their analysis results are returned as output. Specifically, the Google Cloud Natural Language API is used to extract the keyword "in full bloom" from the text and understand its context.

[0618] Step 4: Assess the authenticity of the image and determine whether it has been edited

[0619] The server evaluates the authenticity of the photo based on the results of image analysis. The image analysis results are provided as input data, and the authenticity assessment module assigns an authenticity score to the photo. The image manipulation determination module then determines whether the photo has been digitally manipulated. The output provides an authenticity score and a manipulation determination result. If manipulation is confirmed, an algorithm is applied to return the photo to its original state. Specifically, the Adobe Photoshop API is used to evaluate the authenticity of the image at 87%, detect traces of manipulation, and restore it to its original state if necessary.

[0620] Step 5: Estimating progress

[0621] The server combines the results of image analysis and text analysis. It provides both analysis results as input data and uses an AI model to evaluate the progress of the plants at the tourist destination in real time. As an output, the progress is quantified and the best viewing state is determined. Specifically, the AI ​​model combines past and present data to predict the progress of the plants as "85% full bloom."

[0622] Step 6: Inform users

[0623] A user uses a smartphone app to search for the current plant progress at a tourist spot. The device sends this request to the server. The server retrieves the latest information from the database, generates analysis information and related images, and provides them to the user. The system receives the user's request as input data and provides the analysis results and images as output. Specifically, when a user searches for "Ueno Park cherry blossoms," the server responds with "Currently 85% in full bloom," and also displays an image of the actual location.

[0624] Step 7: Sentiment analysis and its applications

[0625] The server sends the user's posted content and past data to the emotion engine. Text data is provided as input, and the emotion engine uses NLP technology to analyze the user's emotions. The analysis results, or emotional state, are sent back to the server as output. Based on the results, customized tourist destination recommendations are made. For example, if the user feels like "relaxing," tranquil tourist destinations are recommended. Specifically, if the user's emotion is determined to be "relaxed," the server will recommend quiet tourist destinations.

[0626] (Application example 2)

[0627] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0628] Conventional tourist destination information systems have limitations in terms of real-timeness and accuracy, making it particularly difficult to grasp seasonal information such as the progress of cherry blossoms and autumn leaves. Furthermore, they lacked the ability to recommend tourist destinations based on user emotions and to link with real-world store information, preventing user satisfaction. There is a need for a system that can solve these problems, provide users with highly accurate tourist information in real time, and also recommend tourist destinations based on emotions and provide information on nearby real-world stores.

[0629] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting photos and posted content related to tourist destinations from SNS platforms, means for performing image analysis on the collected photos to determine the progress of cherry blossoms and autumn leaves, means for analyzing the text of the collected posted content to determine the progress of cherry blossoms and autumn leaves, means for evaluating the authenticity of the image and whether it has been edited and restoring the image to its original state, means for integrating the results of the image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at the tourist destination, means for providing the estimated progress to the user, means for analyzing the user's emotions and recommending tourist destinations based on the emotions, and means for providing information on brick-and-mortar stores near the tourist destination. This allows users to not only obtain highly accurate tourist information in real time, but also receive recommendations for tourist destinations and information on brick-and-mortar stores in the vicinity that match their emotions at any given time.

[0630] "SNS Platform" means an online platform that provides social networking services, where users can post content such as photos and text to share with other users.

[0631] A "tourist destination" is a specific geographical location or region that tourists visit, and includes natural scenery, cultural assets, leisure facilities, etc.

[0632] A "photograph" is a still image recorded using a photographic device such as a camera, which visually captures the state and characteristics of the tourist destination.

[0633] "Posted content" refers to all content, including text, images, and videos, posted by users on social media platforms.

[0634] "Image analysis" is the process of analyzing the content of an image using computer vision techniques to detect and classify specific objects or features.

[0635] "Progression of cherry blossoms and autumn leaves" refers to the state and stage of progress of plants that change with the seasons, such as cherry blossoms blooming and reaching full bloom, and the color of autumn leaves.

[0636] "Text analysis" is the process of analyzing text data using natural language processing technology to evaluate and extract its content, context, sentiment, etc.

[0637] "Credibility" is the concept of assessing whether collected data or information is accurate and truthful.

[0638] "Manifold" refers to whether an image or text has been digitally altered or modified.

[0639] "Progress estimation" is the process of numerically or qualitatively assessing the current state of cherry blossoms and autumn leaves at tourist spots based on analyzed image and text data.

[0640] "Providing to the user" refers to the act of sending the analyzed information to the user's device and displaying it in visual or text format.

[0641] "Analyzing user emotions" is the process of assessing users' emotions and moods from social media and other digital content using natural language processing.

[0642] "Tourist destination recommendation" is the act of selecting and suggesting appropriate tourist destinations and spots to visit based on the user's current emotions and preferences.

[0643] "Information about nearby physical stores" refers to information that includes details about commercial facilities and service providers (e.g., cafes, souvenir shops, etc.) located around tourist destinations.

[0644] This invention is a system that collects photos and posts related to tourist spots from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves.It also recognizes the user's emotions and provides recommendations on tourist spots and brick-and-mortar store information based on those emotions.

[0645] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts include photos, text, posting date and time, and location information, which are temporarily stored in a database.

[0646] The server then sends the photos stored in the database to an AI image analysis module, which detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color. The analysis results are then sent back to the server along with metadata.

[0647] The server then sends the text of the post to a natural language processing (NLP) module for context and keyword analysis. The NLP module extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the post and analyzes the context.

[0648] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally altered. If manipulation is confirmed, an algorithm is applied to return the photo to its original state and the results are sent back to the server.

[0649] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0650] Users use a smartphone app to search for the current cherry blossom or autumn foliage conditions at tourist spots. The device sends this request to the server, which retrieves the latest information from the database. The server then generates analytical information and related images and provides them to the user. Users can use the app to check the local conditions based on real-time information and plan their trip.

[0651] The server sends the user's posted content and past posted data to the emotion engine. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server. This makes it possible to understand in real time what emotions the user is currently feeling.

[0652] The server recommends customized tourist spots based on the user's emotional data. For example, if a user feels like "I want to relax," it can recommend tourist spots with a calm atmosphere. Similarly, if a user feels like "I want to get excited," it can recommend active tourist spots.

[0653] This will provide users with highly accurate, real-time information, making it easier to plan trips, and will also enable recommendations of tourist spots based on the user's emotions. Furthermore, real-time information on nearby brick-and-mortar stores will also be provided, improving the overall tourist experience.

[0654] For example, if a user searches for "Ueno Park cherry blossoms," the system will provide real-time information on the current cherry blossom conditions in Ueno Park. Along with information such as "They're in full bloom," it will also display information on nearby cafes and souvenir shops. If the user feels like "I want to relax," the system will recommend quiet tourist spots and cafes.

[0655] Examples of prompts include:

[0656] "Please tell me the latest information about the cherry blossoms in Ueno Park and places to relax in the area."

[0657] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0658] Step 1:

[0659] The server sends a request to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The input is the social media platform's API URL and a specific hashtag, and the output is post data including photos, text, posting date and time, and location information. This post data is temporarily stored in a database.

[0660] Step 2:

[0661] The server sends the photos stored in the database to the AI ​​image analysis module. The input is the photo data, and the output is metadata including the results of detecting the characteristics of cherry blossoms and autumn leaves and their status (e.g., full bloom, color change). The AI ​​image analysis module analyzes the photo and sends the results back to the server.

[0662] Step 3:

[0663] The server sends the text of the post to a natural language processing (NLP) module to analyze the context and keywords. The input is the text of the SNS post, and the output is keywords and context analysis results related to the progress of cherry blossoms and autumn leaves (e.g., "in full bloom" and "beginning to change color").

[0664] Step 4:

[0665] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. The input is the result of the AI ​​image analysis, and the output is an authenticity score. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally manipulated. The input is the photo data for manipulation determination, and the output is a manipulation score. If manipulation is confirmed, an algorithm is applied to return the photo to its original state, and the photo is sent back to the server.

[0666] Step 5:

[0667] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The input is the results of image analysis and text analysis, and the output is evaluation data that quantifies the progress of each tourist spot. The AI ​​model evaluates the current state of tourist spots in real time and determines the best viewing conditions.

[0668] Step 6:

[0669] A user uses a smartphone app to search for the current cherry blossom and autumn foliage conditions at a tourist spot. The input is the user's search query (e.g., "Ueno Park cherry blossoms"), and the output is the latest information about the tourist spot (e.g., "In full bloom" and related images). The device sends the request to the server, which retrieves the latest information from the database, generates analytical information and related images, and provides them to the user.

[0670] Step 7:

[0671] The server sends the user's posted content and past posted data to the emotion engine. The input is the user's posted data, and the output is the result of analyzing the user's emotions. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server.

[0672] Step 8:

[0673] The server recommends customized tourist spots based on the user's emotional data. The input is the user's emotional data, and the output is a list of recommended tourist spots. For example, if the user feels like "I want to relax," tourist spots with a calm atmosphere will be recommended. Similarly, if the user feels like "I want to get excited," tourist spots with an active atmosphere will be recommended.

[0674] Step 9:

[0675] The server provides information about brick-and-mortar stores around tourist spots. The input is the location information of the tourist spot and filtering conditions based on the user's interests, and the output is information about brick-and-mortar stores such as nearby cafes and souvenir shops. Users can obtain this information through the app and check it in real time.

[0676] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0677] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0678] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0679] [Third embodiment]

[0680] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0681] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0682] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0683] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0684] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0685] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0686] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0687] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0688] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0689] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0690] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0691] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0692] The present invention relates to a system that collects photos and posts related to tourist spots from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves. The following specific embodiments are conceivable for carrying out the present invention.

[0693] Data collection

[0694] The server periodically collects posts with hashtags such as "cherry blossoms" and "autumn leaves" via the SNS platform API. Photos, text, posting date and time, and location information are extracted from the posts and temporarily stored in a database.

[0695] Data analysis

[0696] Image analysis

[0697] The server then sends the photos stored in the database to an AI image analysis module, which analyzes the image features to determine the state of cherry blossom blooming and the progress of autumn leaves. The analysis results are then sent back to the server along with metadata.

[0698] Text analytics

[0699] The server sends the text of the post to a natural language processing (NLP) module, which analyzes the context and keywords. The NLP module extracts keywords (e.g., "full bloom" or "beginning to change color") that predict the state of the cherry blossoms and autumn leaves from the post and returns the results to the server.

[0700] Image authenticity assessment and manipulation determination

[0701] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally altered. If manipulation is confirmed, an algorithm is applied to return the photo to its original state and the results are sent back to the server.

[0702] Estimating progress

[0703] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0704] Provision to users

[0705] Users use a smartphone app to search for the current cherry blossom and autumn foliage conditions at tourist spots. The device sends the user's request to a server, which retrieves the latest information from a database. The server then generates analytical information and related images and provides them to the user. Users can then use the app to plan their trip based on real-time information.

[0706] Specific examples

[0707] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, the user opens the smartphone app and searches for "Ueno Park cherry blossoms." The device sends this request to the server, which retrieves the latest cherry blossom information for Ueno Park from an existing database.

[0708] As a result of photo analysis, the information "full bloom" was obtained, so the server provides the user with the latest information, "The cherry blossoms in Ueno Park are currently in full bloom." Furthermore, the server generates near real-time images and displays them on the app, allowing the user to visually check the local situation.

[0709] As described above, the present invention makes it easier for users to plan their travel destinations based on highly accurate, real-time information.

[0710] The processing flow will be explained below.

[0711] Step 1:

[0712] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts contain photos, text, posting date and time, and location information.

[0713] Step 2:

[0714] The server extracts photo data from the collected posts and sends it to an image analysis module, which detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color.

[0715] Step 3:

[0716] The server sends the text data of the post to a natural language processing (NLP) module, which extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the text and analyzes the context.

[0717] Step 4:

[0718] The server evaluates the authenticity of the photo based on the results from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo and sends the result back to the server.

[0719] Step 5:

[0720] The server sends the photo to the image manipulation determination module to check whether the image has been manipulated. If manipulation is confirmed, the image manipulation determination module processes the photo to return it to its original state and sends the converted image to the server.

[0721] Step 6:

[0722] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0723] Step 7:

[0724] A user uses a smartphone app to search for the current cherry blossom or autumn foliage status of a tourist spot, and the device sends this request to the server.

[0725] Step 8:

[0726] The server retrieves the latest information on tourist spots from the database in response to user requests. The retrieved information includes estimated cherry blossom and autumn foliage progress and related images.

[0727] Step 9:

[0728] The server responds to users with images related to the current cherry blossom and autumn foliage conditions at tourist destinations, allowing them to use a smartphone app to check the local conditions based on real-time information and plan their trip.

[0729] Example 1

[0730] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0731] In today's world, there is a need to effectively utilize the vast amount of information available on social media to understand the progress of cherry blossoms and autumn leaves at tourist spots in real time. However, there are issues such as the effort required for users to manually search social media to gather information, the difficulty of assessing the credibility of the information, and the risk of photo manipulation. The challenge is to solve these issues and provide more accurate and reliable information in real time.

[0732] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0733] In this invention, the server includes means for collecting photos and posts related to tourist destinations from SNS platforms, means for determining the progress of cherry blossoms and autumn leaves in the collected photos using an AI image analysis module, means for analyzing the text of the collected posts using a natural language processing module to determine the progress of cherry blossoms and autumn leaves, means for evaluating the authenticity of the images and whether they have been edited using an AI image analysis module and an image manipulation determination module and restoring the images to their original state if necessary, means for integrating the results of the image analysis and text analysis and using an AI model to estimate the progress of cherry blossoms and autumn leaves at tourist destinations, and means for providing the estimated progress to users. This allows users to easily obtain accurate and reliable progress information on cherry blossoms and autumn leaves in real time, enabling them to efficiently plan their sightseeing trips.

[0734] "SNS Platform" means an online service that enables users to post photos and text and share those posts with other users.

[0735] The "image analysis module" is a program that uses AI technology to analyze the characteristics of a photo and automatically determine its content.

[0736] A "natural language processing module" is a technology for understanding and analyzing text data, and is a program capable of extracting specific keywords and context.

[0737] "Authenticity assessment" is the process of determining whether a photograph or text accurately reflects a real-world situation.

[0738] The "image manipulation determination module" is a technology for determining whether an image has been digitally manipulated.

[0739] An "AI model" is an artificial intelligence algorithm and architecture that is trained to perform a specific task based on specified input data.

[0740] "Real-time" means providing data or information with a very short time delay, so that it is nearly simultaneous with real time.

[0741] A "tourist destination" is a place or area where visitors gather to see natural elements such as cherry blossoms or autumn leaves.

[0742] "Progress" refers to the current stage of development of cherry blossoms or autumn leaves and the degree of their peak viewing.

[0743] "Text analysis" is the process by which a computer analyzes the content of a sentence or document to extract specific information.

[0744] A "post" is content such as a photo or text that a user publishes on a social media platform.

[0745] This invention is a system that collects photos and posts related to tourist spots from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves.To implement this system, a combination of multiple hardware and software components is used.

[0746] The main components of the system are:

[0747] 1. Data Collection

[0748] The server periodically collects posts tagged with hashtags such as "cherry blossoms" and "autumn leaves" using the APIs of social media platforms. Specifically, it does this through the Twitter API and Instagram API. The server extracts photos, text, posting date and time, and location information from the collected posts and temporarily stores this data in a database. For example, it manages the data using MySQL or PostgreSQL.

[0749] 2. Data Analysis

[0750] Image analysis:

[0751] The server sends the photos stored in the database to an AI image analysis module, which uses deep learning libraries such as TensorFlow and PyTorch, to identify the state of cherry blossom blooming and the progress of autumn leaves and generate corresponding metadata.

[0752] Text analysis:

[0753] The server sends the text of the post to a natural language processing (NLP) module that analyzes the context and keywords. The NLP module uses Hugging Face's Transformers library to extract keywords such as "full bloom" and "beginning to change color" from the text and returns the results to the server.

[0754] 3. Image authenticity assessment and manipulation determination

[0755] The server evaluates the authenticity of the photo based on the analysis results sent from the AI ​​image analysis module. This process uses technology that compares it with previously posted data and evaluates the degree of pattern matching. The image manipulation detection module also determines whether the photo has been digitally altered. Specifically, it checks for unnatural color changes and pixel consistency, and if any abnormalities are found, it applies an algorithm to correct the image.

[0756] 4. Estimating progress

[0757] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at specific tourist spots. It uses AI models such as Scikit-learn and TensorFlow to quantify the condition of each tourist spot and determine the best time to see them.

[0758] 5. Provision to Users

[0759] A user uses a smartphone application to search for the current cherry blossom or autumn foliage conditions at a tourist spot. The device sends the user's request to a server, which retrieves the latest information from a database. The server generates analytical information and related images and provides them to the user. The user can then plan their trip based on the information displayed within the app.

[0760] Specific examples

[0761] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, they open the smartphone app and search for "Ueno Park cherry blossoms." The device then sends this search request to the server. The server retrieves the latest cherry blossom information for Ueno Park from an existing database, analyzes the information, and generates near-real-time images to provide to the user. Based on this, users can visually check the local conditions and efficiently plan their trip.

[0762] Prompt Sentence Examples

[0763] For example, a possible prompt for a generative AI model might be:

[0764] "Please tell me the steps to analyze the progress of cherry blossoms and autumn leaves at tourist spots from social media posts."

[0765] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0766] Step 1:

[0767] Collecting social media posts

[0768] The server periodically collects posts tagged with specific hashtags, such as "cherry blossoms" or "autumn leaves," using the API of the social media platform. Specifically, it uses the Twitter API or Instagram API. The server extracts photos, text, posting date and time, and location information from the posts. The input is the raw posting data obtained from the social media API, and the output is a database entry with the extracted photos, text, posting date and time, and location information. This data is temporarily stored in a database. For example, it includes operations to retrieve data in JSON format and save each field to an SQL database.

[0769] Step 2:

[0770] Image analysis

[0771] The server sends the photos stored in the database to an AI image analysis module. Specifically, it uses a deep learning model using TensorFlow and PyTorch. The input is photo data retrieved from the database, and the output generates metadata indicating the state of cherry blossom blooming and the progress of autumn leaves. For example, it converts images into feature vectors and classifies them as "blooming" or "full bloom" based on those vectors.

[0772] Step 3:

[0773] Text analytics

[0774] The server sends the text of the post to a natural language processing (NLP) module, which analyzes keywords and context. The NLP module uses Hugging Face's Transformers library. The input is the post text, and the output is extracted keywords that indicate the state of the cherry blossoms and autumn leaves (e.g., "in full bloom" and "beginning to change color"). For example, this includes tokenizing the text and identifying keywords by capturing the context using the BERT model.

[0775] Step 4:

[0776] Credibility assessment and manipulation judgment

[0777] The server evaluates the authenticity of the photo based on the analysis results sent from the AI ​​image analysis module. First, the authenticity evaluation evaluates the degree of match with previously posted data and assigns an authenticity score. Next, the image manipulation determination module determines whether or not the photo has been digitally altered. Specifically, the image's pixel data is analyzed using a deep learning model. The input is the analyzed photo data, and the output is an authenticity score and an alteration determination result. If alteration is confirmed, the process also applies an algorithm to return the photo to its original state.

[0778] Step 5:

[0779] Estimating progress

[0780] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at specific tourist spots. The input is the result data of image and text analysis, and the output is a quantified result of the progress of cherry blossoms and autumn leaves at each tourist spot. Specifically, this involves using AI models such as Scikit-learn and TensorFlow to aggregate data points and estimate the progress.

[0781] Step 6:

[0782] Provision to users

[0783] A user uses a smartphone app to search for the current progress of cherry blossoms or autumn leaves at a tourist spot. The device sends the user's request to a server. The server retrieves the latest information from a database, generates analysis results and related images, and provides them to the user. The input is the user's search query, and the output is analysis information and near-real-time images. For example, this includes the actions of an app developed with Flutter or React Native sending a request to a server, receiving a response from the server, and displaying it.

[0784] (Application example 1)

[0785] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0786] In recent years, the spread of social media platforms has increased the number of ways to obtain real-time seasonal landscape information for tourist destinations. However, this information is scattered, making it difficult to centrally collect and provide accurate and reliable information to users. Furthermore, when planning a trip, it is necessary to simultaneously obtain information on seasonal food and beverages and related services, but no such system exists. Given this background, this invention aims to provide a system that accurately obtains the progress of cherry blossoms and autumn leaves at tourist destinations in real time and provides information on seasonal food and beverages based on that information.

[0787] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0788] In this invention, the server includes means for collecting images and posts related to tourist destinations from SNS platforms, means for performing image analysis on the collected images to determine the progress of cherry blossoms and autumn leaves, means for analyzing the text of the collected posts to determine the progress of cherry blossoms and autumn leaves, means for evaluating the authenticity of the images and whether they have been edited and restoring the images to their original state, means for integrating the results of the image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at the tourist destination, means for providing information related to seasonal food and drink based on the estimated progress, and means for providing the estimated progress and information on related food and drink to the user. This allows users to grasp the progress of cherry blossoms and autumn leaves at tourist destinations in real time, make optimal sightseeing plans based on that information, and simultaneously obtain information on seasonal food delivery and food and drink.

[0789] "SNS Platform" means an online social networking service that enables users to share posts such as photos and text.

[0790] A "tourist destination" is a specific place or area that tourists visit.

[0791] "Image analysis" is the process of analyzing image data using computer vision techniques to identify image content and features.

[0792] "Text analytics" is the process of analyzing text data using natural language processing techniques to understand its content and meaning.

[0793] "Credibility" is an indicator of the accuracy and reliability of information or data.

[0794] "Whether or not an image or data has been altered" refers to determining whether or not the image or data has been digitally altered.

[0795] "Estimation" is the prediction of unobserved or future states based on data analysis.

[0796] "Seasonal food and beverages" are foods and beverages that are only available during a particular season.

[0797] "Information provision" means providing useful information to users in a timely manner.

[0798] "User" refers to the general user of this system.

[0799] This invention is a system that provides information on seasonal scenery and food and drink options at tourist destinations. This system collects related images and posts from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves at tourist destinations. It also aims to improve convenience for tourists by simultaneously providing information on seasonal food and drink options.

[0800] Overall system flow:

[0801] 1. Data Collection

[0802] The server periodically uses the API of the social media platform to collect posts with hashtags such as "cherry blossoms" and "autumn leaves."

[0803] Images, text, posting date and time, and location information are extracted from posts and temporarily stored in a database.

[0804] 2. Image Analysis

[0805] The server sends the stored images to an AI image analysis module, which uses deep learning frameworks such as TensorFlow and Keras to determine the progression of cherry blossoms and autumn leaves.

[0806] 3. Text Analysis

[0807] The server sends the text of the post to a natural language processing (NLP) module, which extracts context and keywords to infer the progress of the cherry blossoms and autumn leaves.

[0808] 4. Credibility assessment and manipulation judgment

[0809] The server evaluates the authenticity of the image based on the output of the AI ​​image analysis module, and then the image manipulation determination module determines whether the image has been digitally manipulated, and if so, returns it to its original state.

[0810] 5. Estimating progress

[0811] The server integrates the results of image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot.

[0812] The estimated results are provided to the user in real time, along with information related to seasonal food and drink options for specific tourist destinations.

[0813] 6. Provision to Users

[0814] Users use a smartphone app to search for information on the progress of cherry blossoms and autumn leaves at tourist spots and related food and drink options.

[0815] The terminal sends the user's request to the server, which retrieves the latest information from the database.

[0816] The server generates and provides the user with information about food and drink related to the progress.

[0817] Specific hardware and software

[0818] Hardware: Using cloud platforms (e.g., AWS, Google Cloud), servers perform large-scale data processing.

[0819] Software: TensorFlow and Keras are used for image analysis, and SpaCy and Transformers are used for NLP.

[0820] Specific examples

[0821] For example, if a user wants to know about spring tourist spots in a certain area, the user opens a smartphone app and voice-inputs "tourist spot name cherry blossom food delivery." This prompt is sent to the server, which analyzes the collected SNS data and estimates the latest status of the cherry blossoms at that tourist spot. The analysis results in the information that the cherry blossoms are in full bloom, and related seasonal foods and drinks such as "cherry blossom bento" are recommended. The user can receive a notification that "The cherry blossoms at tourist spot name are currently in full bloom. The recommended food and drink is the cherry blossom bento."

[0822] Prompt Sentence Examples

[0823] "Ueno Park Cherry Blossom Food Delivery"

[0824] "Kyoto Autumn Leaves Recommended Menu"

[0825] As described above, the present invention enables users to plan trips based on real-time information on tourist spots, and also allows users to obtain information on seasonal food and drink all at once.

[0826] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0827] Step 1:

[0828] The server periodically collects posts tagged with hashtags such as "cherry blossoms" and "autumn leaves" via the API of the social media platform. At this time, the server extracts images, text, posting date and time, and location information, and temporarily stores them in a database. The input is the posting information from the social media platform, and the output is the raw data stored in the database.

[0829] Step 2:

[0830] The server sends the images stored in the database to an AI image analysis module. The AI ​​image analysis module (using TensorFlow and Keras) analyzes the image features and determines the progress of cherry blossoms and autumn leaves. The input is images collected from social media, and the output is the progress (e.g., full bloom, beginning to change color) as a result of analysis.

[0831] Step 3:

[0832] The server sends the collected text data to a natural language processing (NLP) module, which analyzes the text's context and keywords. The NLP module uses SpaCy and Transformers to extract keywords that describe the state of the cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color"). The input is the text data collected from social media, and the output is the extracted keywords and their analysis results.

[0833] Step 4:

[0834] The server integrates the results obtained from the AI ​​image analysis module and the NLP module. Specifically, it associates the image analysis results with the text analysis results and estimates the progress of cherry blossoms and autumn leaves at tourist spots based on location information and posting date and time. The inputs are the image analysis results and text analysis results, and the output is the integrated progress data.

[0835] Step 5:

[0836] The server evaluates the authenticity of images and determines whether they have been altered. The authenticity evaluation module assigns an authenticity score to the image, and the image alteration determination module determines whether the image has been digitally altered. If alteration is confirmed, the server applies an image correction algorithm to return the image to its original state. The input is image data collected from SNS, and the output is the authenticity score and the altered image.

[0837] Step 6:

[0838] The server provides information related to seasonal food and drink based on the estimated cherry blossom and autumn foliage progress. This information is linked to the progress of each tourist spot. The input is the integrated progress data, and the output is seasonal food and drink information.

[0839] Step 7:

[0840] A user uses a smartphone app to search for information on the progress of cherry blossoms and autumn leaves at tourist spots and related food and drink options. The device sends the user's request to a server, which retrieves the latest information from a database and provides it. The input is the user's search query, and the output is a notification to the user containing the progress of the tourist spot and related food and drink options.

[0841] This allows users to plan their trips based on real-time information on tourist spots, and also allows them to obtain seasonal food delivery information at the same time.The operation of this system realizes real-time and highly accurate information provision, and provides services that meet user demand.

[0842] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0843] The present invention combines a system that collects photos and posts related to tourist spots from SNS platforms, analyzes them, and estimates and provides the progress of cherry blossoms and autumn leaves in real time, with an emotion engine that recognizes the user's emotions. The following specific forms are conceivable for carrying out the present invention.

[0844] Data collection

[0845] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts include photos, text, posting date and time, and location information, which are temporarily stored in a database.

[0846] Data analysis

[0847] Image analysis

[0848] The server sends the photos stored in the database to an AI image analysis module, which detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color. The analysis results are then sent back to the server along with metadata.

[0849] Text analytics

[0850] The server sends the text of the post to a natural language processing (NLP) module, which analyzes the context and keywords. The NLP module extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the post and analyzes the context.

[0851] Image authenticity assessment and manipulation determination

[0852] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally altered. If manipulation is confirmed, an algorithm is applied to return the photo to its original state and the results are sent back to the server.

[0853] Estimating progress

[0854] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0855] Provision to users

[0856] Users use a smartphone app to search for the current cherry blossom or autumn foliage conditions at tourist spots. The device sends this request to the server, which retrieves the latest information from the database. The server then generates analytical information and related images and provides them to the user. Users can use the app to check the local conditions based on real-time information and plan their trip.

[0857] Sentiment analysis and its applications

[0858] Emotion Engine

[0859] The server sends the user's posted content and past posted data to the emotion engine. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server. This makes it possible to understand in real time what emotions the user is currently feeling.

[0860] Emotion-based tourist destination recommendation

[0861] The server recommends customized tourist spots based on the user's emotional data. For example, if a user feels like "I want to relax," it can recommend tourist spots with a calm atmosphere. Similarly, if a user feels like "I want to get excited," it can recommend active tourist spots.

[0862] Specific examples

[0863] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, the user opens a smartphone app and searches for "Ueno Park cherry blossoms." The device sends this request to the server, which retrieves the latest cherry blossom information for Ueno Park from an existing database.

[0864] As a result of photo analysis, the information "full bloom" was obtained, so the server provides the user with the latest information, "The cherry blossoms in Ueno Park are currently in full bloom." Furthermore, the server generates near real-time images and displays them on the app, allowing the user to visually check the local situation.

[0865] In addition, if the user's emotional data is analyzed and the user feels like "I want to relax," the server can recommend quiet tourist spots and places where they can relax. This allows the user to plan a trip that suits their state of mind.

[0866] As described above, the present invention not only makes it easier for users to plan their travel destinations based on highly accurate, real-time information, but also allows users to choose tourist destinations that suit their own emotions, resulting in a more satisfying travel experience.

[0867] The processing flow will be explained below.

[0868] Step 1:

[0869] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts include photos, text, posting date and time, and location information.

[0870] Step 2:

[0871] The server extracts photo data from the collected posts and sends it to an AI image analysis module. The AI ​​image analysis module detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color. The analysis results are then sent back to the server along with metadata.

[0872] Step 3:

[0873] The server sends the text data of the post to a natural language processing (NLP) module. The NLP module extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the text and analyzes the context. The analysis results are sent back to the server.

[0874] Step 4:

[0875] The server evaluates the authenticity of the photo based on the results from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo and sends the result back to the server.

[0876] Step 5:

[0877] The server sends the photo to the image manipulation determination module to check whether the image has been manipulated. If it is found to have been manipulated, the image manipulation determination module processes the photo to return it to its original state and sends the converted image to the server.

[0878] Step 6:

[0879] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0880] Step 7:

[0881] A user uses a smartphone app to search for the current cherry blossom or autumn foliage status of a tourist spot, and the device sends this request to the server.

[0882] Step 8:

[0883] The server retrieves the latest information on tourist spots from the database in response to user requests, and generates images related to the progress of cherry blossoms and autumn leaves based on the retrieved information.

[0884] Step 9:

[0885] The server responds to the user with the current cherry blossom and autumn foliage conditions at the tourist destination and the generated images. The user can then use the smartphone app to check the local conditions based on real-time information and plan their trip.

[0886] Step 10:

[0887] The server sends data to the emotion engine to analyze the user's past posts and current emotions. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server.

[0888] Step 11:

[0889] The server recommends customized tourist spots based on the user's emotional data. For example, if the user feels like "I want to relax," the server will recommend quiet tourist spots and places where they can relax. Information about tourist spots that match the user's emotions is sent to the server.

[0890] Step 12:

[0891] Users can check tourist destination information provided through the smartphone app and create travel plans that suit their own preferences, resulting in a more satisfying travel experience.

[0892] Example 2

[0893] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0894] Conventional tourist information systems have had difficulty accurately grasping the progress of local plants (e.g., cherry blossoms and autumn leaves) in tourist destinations in real time. Furthermore, since tourist destination recommendations are not tailored to the user's emotions, satisfaction with travel plans can decrease. The purpose of this invention is to solve these problems.

[0895] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data related to tourist destinations from SNS platforms, means for performing image analysis on the collected data to determine the progress of plants, means for analyzing text in the collected data to determine the progress of plants, means for evaluating the authenticity of the image and whether it has been edited and restoring the image to its original state, means for integrating the results of the image analysis and text analysis to estimate the progress of plants at the tourist destination, means for providing the estimated progress to the user, and means for analyzing the user's emotions and recommending tourist destinations based on the results. This makes it possible to accurately grasp the progress of tourist destinations in real time and to recommend tourist destinations customized based on the user's emotions.

[0896] "SNS platform" refers to an online service that allows a large number of users to post and interact with each other.

[0897] "Data" includes information collected from social media platforms, such as photos, text, posting dates and times, and location information.

[0898] "Image analysis" refers to the process of detecting specific features from collected photographic data and determining the condition based on those features.

[0899] "Plant progress" refers to the extent to which plants such as cherry blossoms and autumn leaves have flowered or changed color.

[0900] "Text analysis" refers to the process of analyzing the context and keywords of collected text data to extract specific information.

[0901] "Authenticity" refers to the degree of confidence that a photograph or text accurately reflects the actual situation.

[0902] "Presence or absence of manipulation" refers to the state of determining whether a photograph or text has been digitally altered.

[0903] "Reverting" refers to the process of restoring digitally altered photographs or text to their original state.

[0904] "Integration" refers to combining the results of image analysis and text analysis into a single comprehensive piece of information.

[0905] "Tourist destination" refers to an area or facility that users visit for sightseeing.

[0906] "Estimation" refers to predicting information such as the progress of plants based on collected data.

[0907] "User" refers to the end user who uses this system to obtain tourist information.

[0908] "Providing" refers to displaying or notifying the user of the estimated information.

[0909] "Sentiment analysis" refers to the process of analyzing a user's posts and past data to understand their emotional state at the time.

[0910] "Recommendation" refers to selecting and presenting appropriate tourist destinations based on the user's emotional state.

[0911] The present invention combines a system that collects data related to tourist destinations from a social media platform, analyzes the data, and estimates and provides the progress of plants in real time, with an emotion engine that recognizes the user's emotions. The following specific forms are conceivable for carrying out the present invention.

[0912] Data collection

[0913] The server periodically sends requests to the API of the social media platform to collect posts containing hashtags such as "cherry blossoms" or "autumn leaves." The collected data includes photos, text, posting date and time, and location information, which is temporarily stored in a database.

[0914] Image analysis

[0915] The server sends the photos stored in the database to an AI image analysis module, which uses deep learning technology to detect plant characteristics in the photos and determine their state, such as full bloom or color. The analysis results, along with metadata, are then sent back to the server.

[0916] Text analytics

[0917] The server sends the text of the post to a natural language processing (NLP) module, which analyzes the context and keywords. The NLP module extracts keywords related to the plant's progress (e.g., "full bloom" or "beginning to change color") from the post and analyzes the context.

[0918] Image authenticity assessment and manipulation determination

[0919] The server evaluates the authenticity of the photo based on the results sent from the image analysis module. The authenticity assessment module assigns an authenticity score to the photo, and then the image manipulation determination module determines whether the photo has been digitally manipulated. If manipulation is confirmed, an algorithm is applied to restore the image to its original state and the result is sent back to the server.

[0920] Estimating progress

[0921] The server integrates the results of image and text analysis to estimate the progress of plants at each tourist spot, and uses an AI model to evaluate the current state of the tourist spot in real time, quantifying the progress, and determining the best viewing conditions.

[0922] Provision to users

[0923] Users use a smartphone app to search for the current plant status of a tourist attraction. The device sends this request to the server, which retrieves the latest information from the database. The server generates analytical information and related images and provides them to the user. Users can use the app to check the local situation based on real-time information and plan their trip.

[0924] Sentiment analysis and its applications

[0925] Emotion Engine

[0926] The server sends the user's posted content and past posted data to the emotion engine. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server. This makes it possible to understand the user's current emotions in real time.

[0927] Emotion-based tourist destination recommendation

[0928] The server recommends customized tourist spots based on the user's emotional data. For example, if a user feels like "I want to relax," the server can recommend tourist spots with a calm atmosphere. Similarly, if a user feels like "I want to get excited," the server can recommend active tourist spots.

[0929] Specific examples

[0930] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, they open a smartphone app and search for "Ueno Park cherry blossoms." The device sends this request to the server, which retrieves the latest cherry blossom information for Ueno Park from its existing database. If the photo analysis results indicate that the cherry blossoms are in full bloom, the server provides the user with the latest information: "The cherry blossoms in Ueno Park are currently in full bloom." Furthermore, the server generates near-real-time images and displays them in the app, allowing the user to visually check the current situation. Furthermore, by analyzing the user's emotional data, if the user feels like "relaxing," the server can recommend quiet tourist spots and relaxing places. This allows users to create travel plans that suit their individual needs.

[0931] The specific hardware and software used

[0932] Hardware: Servers, devices (smartphones, tablets), etc.

[0933] Software: Social media platform API, AI image analysis module, natural language processing (NLP) module, emotion engine

[0934] Other specific technology examples include TensorFlow, Google Cloud Natural Language API, MySQL, and PostgreSQL.

[0935] Example prompts for generative AI models

[0936] "I would like to know the current state of the cherry blossoms in Ueno Park in Tokyo."

[0937] Please recommend some relaxing tourist spots.

[0938] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0939] Step 1: Data collection

[0940] The server periodically sends requests to the API of the social media platform. As input data, it collects posts containing specific hashtags (e.g., "cherry blossoms" or "autumn leaves"). The collected data includes photos, text, posting date and time, and location information. The server temporarily stores the collected data in a database. Specifically, it uses the API of the social media platform to search for related posts, obtains the data in JSON format, and stores it in the database.

[0941] Step 2: Image analysis

[0942] The server sends the photo data stored in the database to the AI ​​image analysis module. By providing the photo as input data, the image analysis module uses deep learning technology to detect the characteristics of the plants in the photo. As output, the state of full bloom and color is returned along with metadata. Specifically, an image analysis model using TensorFlow is used to determine the full bloom state of the cherry blossoms, and metadata such as "75% full bloom" is returned.

[0943] Step 3: Text analysis

[0944] The server sends the collected text data of posts to a natural language processing (NLP) module. By providing the text as input data, the NLP module analyzes the context and keywords and extracts keywords related to the plant's progress (e.g., "in full bloom" or "beginning to change color"). The extracted keywords and their analysis results are returned as output. Specifically, the Google Cloud Natural Language API is used to extract the keyword "in full bloom" from the text and understand its context.

[0945] Step 4: Assess the authenticity of the image and determine whether it has been edited

[0946] The server evaluates the authenticity of the photo based on the results of image analysis. The image analysis results are provided as input data, and the authenticity assessment module assigns an authenticity score to the photo. The image manipulation determination module then determines whether the photo has been digitally manipulated. The output provides an authenticity score and a manipulation determination result. If manipulation is confirmed, an algorithm is applied to return the photo to its original state. Specifically, the Adobe Photoshop API is used to evaluate the authenticity of the image at 87%, detect traces of manipulation, and restore it to its original state if necessary.

[0947] Step 5: Estimating progress

[0948] The server combines the results of image analysis and text analysis. It provides both analysis results as input data and uses an AI model to evaluate the progress of the plants at the tourist destination in real time. As an output, the progress is quantified and the best viewing state is determined. Specifically, the AI ​​model combines past and present data to predict the progress of the plants as "85% full bloom."

[0949] Step 6: Inform users

[0950] A user uses a smartphone app to search for the current plant progress at a tourist spot. The device sends this request to the server. The server retrieves the latest information from the database, generates analysis information and related images, and provides them to the user. The system receives the user's request as input data and provides the analysis results and images as output. Specifically, when a user searches for "Ueno Park cherry blossoms," the server responds with "Currently 85% in full bloom," and also displays an image of the actual location.

[0951] Step 7: Sentiment analysis and its applications

[0952] The server sends the user's posted content and past data to the emotion engine. Text data is provided as input, and the emotion engine uses NLP technology to analyze the user's emotions. The analysis results, or emotional state, are sent back to the server as output. Based on the results, customized tourist destination recommendations are made. For example, if the user feels like "relaxing," tranquil tourist destinations are recommended. Specifically, if the user's emotion is determined to be "relaxed," the server will recommend quiet tourist destinations.

[0953] (Application example 2)

[0954] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0955] Conventional tourist destination information systems have limitations in terms of real-timeness and accuracy, making it particularly difficult to grasp seasonal information such as the progress of cherry blossoms and autumn leaves. Furthermore, they lacked the ability to recommend tourist destinations based on user emotions and to link with real-world store information, preventing user satisfaction. There is a need for a system that can solve these problems, provide users with highly accurate tourist information in real time, and also recommend tourist destinations based on emotions and provide information on nearby real-world stores.

[0956] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting photos and posted content related to tourist destinations from SNS platforms, means for performing image analysis on the collected photos to determine the progress of cherry blossoms and autumn leaves, means for analyzing the text of the collected posted content to determine the progress of cherry blossoms and autumn leaves, means for evaluating the authenticity of the image and whether it has been edited and restoring the image to its original state, means for integrating the results of the image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at the tourist destination, means for providing the estimated progress to the user, means for analyzing the user's emotions and recommending tourist destinations based on the emotions, and means for providing information on brick-and-mortar stores near the tourist destination. This allows users to not only obtain highly accurate tourist information in real time, but also receive recommendations for tourist destinations and information on brick-and-mortar stores in the vicinity that match their emotions at any given time.

[0957] "SNS Platform" means an online platform that provides social networking services, where users can post content such as photos and text to share with other users.

[0958] A "tourist destination" is a specific geographical location or region that tourists visit, and includes natural scenery, cultural assets, leisure facilities, etc.

[0959] A "photograph" is a still image recorded using a photographic device such as a camera, which visually captures the state and characteristics of the tourist destination.

[0960] "Posted content" refers to all content, including text, images, and videos, posted by users on social media platforms.

[0961] "Image analysis" is the process of analyzing the content of an image using computer vision techniques to detect and classify specific objects or features.

[0962] "Progression of cherry blossoms and autumn leaves" refers to the state and stage of progress of plants that change with the seasons, such as cherry blossoms blooming and reaching full bloom, and the color of autumn leaves.

[0963] "Text analysis" is the process of analyzing text data using natural language processing technology to evaluate and extract its content, context, sentiment, etc.

[0964] "Credibility" is the concept of assessing whether collected data or information is accurate and truthful.

[0965] "Manifold" refers to whether an image or text has been digitally altered or modified.

[0966] "Progress estimation" is the process of numerically or qualitatively assessing the current state of cherry blossoms and autumn leaves at tourist spots based on analyzed image and text data.

[0967] "Providing to the user" refers to the act of sending the analyzed information to the user's device and displaying it in visual or text format.

[0968] "Analyzing user emotions" is the process of assessing users' emotions and moods from social media and other digital content using natural language processing.

[0969] "Tourist destination recommendation" is the act of selecting and suggesting appropriate tourist destinations and spots to visit based on the user's current emotions and preferences.

[0970] "Information about nearby physical stores" refers to information that includes details about commercial facilities and service providers (e.g., cafes, souvenir shops, etc.) located around tourist destinations.

[0971] This invention is a system that collects photos and posts related to tourist spots from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves.It also recognizes the user's emotions and provides recommendations on tourist spots and brick-and-mortar store information based on those emotions.

[0972] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts include photos, text, posting date and time, and location information, which are temporarily stored in a database.

[0973] The server then sends the photos stored in the database to an AI image analysis module, which detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color. The analysis results are then sent back to the server along with metadata.

[0974] The server then sends the text of the post to a natural language processing (NLP) module for context and keyword analysis. The NLP module extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the post and analyzes the context.

[0975] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally altered. If manipulation is confirmed, an algorithm is applied to return the photo to its original state and the results are sent back to the server.

[0976] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[0977] Users use a smartphone app to search for the current cherry blossom or autumn foliage conditions at tourist spots. The device sends this request to the server, which retrieves the latest information from the database. The server then generates analytical information and related images and provides them to the user. Users can use the app to check the local conditions based on real-time information and plan their trip.

[0978] The server sends the user's posted content and past posted data to the emotion engine. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server. This makes it possible to understand in real time what emotions the user is currently feeling.

[0979] The server recommends customized tourist spots based on the user's emotional data. For example, if a user feels like "I want to relax," it can recommend tourist spots with a calm atmosphere. Similarly, if a user feels like "I want to get excited," it can recommend active tourist spots.

[0980] This will provide users with highly accurate, real-time information, making it easier to plan trips, and will also enable recommendations of tourist spots based on the user's emotions.In addition, real-time information on nearby physical stores will be provided, improving the overall tourist experience.

[0981] For example, if a user searches for "Ueno Park cherry blossoms," the system will provide real-time information on the current cherry blossom conditions in Ueno Park. Along with information such as "They're in full bloom," it will also display information on nearby cafes and souvenir shops. If the user feels like "I want to relax," the system will recommend quiet tourist spots and cafes.

[0982] Examples of prompts include:

[0983] "Please tell me the latest information about the cherry blossoms in Ueno Park and places to relax in the area."

[0984] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0985] Step 1:

[0986] The server sends a request to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The input is the social media platform's API URL and a specific hashtag, and the output is post data including photos, text, posting date and time, and location information. This post data is temporarily stored in a database.

[0987] Step 2:

[0988] The server sends the photos stored in the database to the AI ​​image analysis module. The input is the photo data, and the output is metadata including the results of detecting the characteristics of cherry blossoms and autumn leaves and their status (e.g., full bloom, color change). The AI ​​image analysis module analyzes the photo and sends the results back to the server.

[0989] Step 3:

[0990] The server sends the text of the post to a natural language processing (NLP) module to analyze the context and keywords. The input is the text of the SNS post, and the output is keywords and context analysis results related to the progress of cherry blossoms and autumn leaves (e.g., "in full bloom" and "beginning to change color").

[0991] Step 4:

[0992] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. The input is the result of the AI ​​image analysis, and the output is an authenticity score. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally manipulated. The input is the photo data for manipulation determination, and the output is a manipulation score. If manipulation is confirmed, an algorithm is applied to return the photo to its original state, and the photo is sent back to the server.

[0993] Step 5:

[0994] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The input is the results of image analysis and text analysis, and the output is evaluation data that quantifies the progress of each tourist spot. The AI ​​model evaluates the current state of tourist spots in real time and determines the best viewing conditions.

[0995] Step 6:

[0996] A user uses a smartphone app to search for the current cherry blossom and autumn foliage conditions at a tourist spot. The input is the user's search query (e.g., "Ueno Park cherry blossoms"), and the output is the latest information about the tourist spot (e.g., "In full bloom" and related images). The device sends the request to the server, which retrieves the latest information from the database, generates analytical information and related images, and provides them to the user.

[0997] Step 7:

[0998] The server sends the user's posted content and past posted data to the emotion engine. The input is the user's posted data, and the output is the result of analyzing the user's emotions. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server.

[0999] Step 8:

[1000] The server recommends customized tourist spots based on the user's emotional data. The input is the user's emotional data, and the output is a list of recommended tourist spots. For example, if the user feels like "I want to relax," tourist spots with a calm atmosphere will be recommended. Similarly, if the user feels like "I want to get excited," tourist spots with an active atmosphere will be recommended.

[1001] Step 9:

[1002] The server provides information about brick-and-mortar stores around tourist spots. The input is the location information of the tourist spot and filtering conditions based on the user's interests, and the output is information about brick-and-mortar stores such as nearby cafes and souvenir shops. Users can obtain this information through the app and check it in real time.

[1003] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1004] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1005] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1006] [Fourth embodiment]

[1007] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1008] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1009] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1010] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1011] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1012] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1013] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1014] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1015] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1016] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1017] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1018] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1019] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1020] The present invention relates to a system that collects photos and posts related to tourist spots from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves. The following specific embodiments are conceivable for carrying out the present invention.

[1021] Data collection

[1022] The server periodically collects posts with hashtags such as "cherry blossoms" and "autumn leaves" via the SNS platform API. Photos, text, posting date and time, and location information are extracted from the posts and temporarily stored in a database.

[1023] Data analysis

[1024] Image analysis

[1025] The server then sends the photos stored in the database to an AI image analysis module, which analyzes the image features to determine the state of cherry blossom blooming and the progress of autumn leaves. The analysis results are then sent back to the server along with metadata.

[1026] Text analytics

[1027] The server sends the text of the post to a natural language processing (NLP) module, which analyzes the context and keywords. The NLP module extracts keywords (e.g., "full bloom" or "beginning to change color") that predict the state of the cherry blossoms and autumn leaves from the post and returns the results to the server.

[1028] Image authenticity assessment and manipulation determination

[1029] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally altered. If manipulation is confirmed, an algorithm is applied to return the photo to its original state and the results are sent back to the server.

[1030] Estimating progress

[1031] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[1032] Provision to users

[1033] Users use a smartphone app to search for the current cherry blossom and autumn foliage conditions at tourist spots. The device sends the user's request to a server, which retrieves the latest information from a database. The server then generates analytical information and related images and provides them to the user. Users can then use the app to plan their trip based on real-time information.

[1034] Specific examples

[1035] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, the user opens the smartphone app and searches for "Ueno Park cherry blossoms." The device sends this request to the server, which retrieves the latest cherry blossom information for Ueno Park from an existing database.

[1036] As a result of photo analysis, the information "full bloom" was obtained, so the server provides the user with the latest information, "The cherry blossoms in Ueno Park are currently in full bloom." Furthermore, the server generates near real-time images and displays them on the app, allowing the user to visually check the local situation.

[1037] As described above, the present invention makes it easier for users to plan their travel destinations based on highly accurate, real-time information.

[1038] The processing flow will be explained below.

[1039] Step 1:

[1040] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts contain photos, text, posting date and time, and location information.

[1041] Step 2:

[1042] The server extracts photo data from the collected posts and sends it to an image analysis module, which detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color.

[1043] Step 3:

[1044] The server sends the text data of the post to a natural language processing (NLP) module, which extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the text and analyzes the context.

[1045] Step 4:

[1046] The server evaluates the authenticity of the photo based on the results from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo and sends the result back to the server.

[1047] Step 5:

[1048] The server sends the photo to the image manipulation determination module to check whether the image has been manipulated. If manipulation is confirmed, the image manipulation determination module processes the photo to return it to its original state and sends the converted image to the server.

[1049] Step 6:

[1050] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[1051] Step 7:

[1052] A user uses a smartphone app to search for the current cherry blossom or autumn foliage status of a tourist spot, and the device sends this request to the server.

[1053] Step 8:

[1054] The server retrieves the latest information on tourist spots from the database in response to user requests. The retrieved information includes estimated cherry blossom and autumn foliage progress and related images.

[1055] Step 9:

[1056] The server responds to users with images related to the current cherry blossom and autumn foliage conditions at tourist destinations, allowing them to use a smartphone app to check the local conditions based on real-time information and plan their trip.

[1057] Example 1

[1058] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1059] In today's world, there is a need to effectively utilize the vast amount of information available on social media to understand the progress of cherry blossoms and autumn leaves at tourist spots in real time. However, there are issues such as the effort required for users to manually search social media to gather information, the difficulty of assessing the credibility of the information, and the risk of photo manipulation. The challenge is to solve these issues and provide more accurate and reliable information in real time.

[1060] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1061] In this invention, the server includes means for collecting photos and posts related to tourist destinations from SNS platforms, means for determining the progress of cherry blossoms and autumn leaves in the collected photos using an AI image analysis module, means for analyzing the text of the collected posts using a natural language processing module to determine the progress of cherry blossoms and autumn leaves, means for evaluating the authenticity of the images and whether they have been edited using an AI image analysis module and an image manipulation determination module and restoring the images to their original state if necessary, means for integrating the results of the image analysis and text analysis and using an AI model to estimate the progress of cherry blossoms and autumn leaves at tourist destinations, and means for providing the estimated progress to users. This allows users to easily obtain accurate and reliable progress information on cherry blossoms and autumn leaves in real time, enabling them to efficiently plan their sightseeing trips.

[1062] "SNS Platform" means an online service that enables users to post photos and text and share those posts with other users.

[1063] The "image analysis module" is a program that uses AI technology to analyze the characteristics of a photo and automatically determine its content.

[1064] A "natural language processing module" is a technology for understanding and analyzing text data, and is a program capable of extracting specific keywords and context.

[1065] "Authenticity assessment" is the process of determining whether a photograph or text accurately reflects a real-world situation.

[1066] The "image manipulation determination module" is a technology for determining whether an image has been digitally manipulated.

[1067] An "AI model" is an artificial intelligence algorithm and architecture that is trained to perform a specific task based on specified input data.

[1068] "Real-time" means providing data or information with a very short time delay, so that it is nearly simultaneous with real time.

[1069] A "tourist destination" is a place or area where visitors gather to see natural elements such as cherry blossoms or autumn leaves.

[1070] "Progress" refers to the current stage of development of cherry blossoms or autumn leaves and the degree of their peak viewing.

[1071] "Text analysis" is the process by which a computer analyzes the content of a sentence or document to extract specific information.

[1072] A "post" is content such as a photo or text that a user publishes on a social media platform.

[1073] This invention is a system that collects photos and posts related to tourist spots from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves.To implement this system, a combination of multiple hardware and software components is used.

[1074] The main components of the system are:

[1075] 1. Data Collection

[1076] The server periodically collects posts tagged with hashtags such as "cherry blossoms" and "autumn leaves" using the APIs of social media platforms. Specifically, it does this through the Twitter API and Instagram API. The server extracts photos, text, posting date and time, and location information from the collected posts and temporarily stores this data in a database. For example, it manages the data using MySQL or PostgreSQL.

[1077] 2. Data Analysis

[1078] Image analysis:

[1079] The server sends the photos stored in the database to an AI image analysis module, which uses deep learning libraries such as TensorFlow and PyTorch, to identify the state of cherry blossom blooming and the progress of autumn leaves and generate corresponding metadata.

[1080] Text analysis:

[1081] The server sends the text of the post to a natural language processing (NLP) module that analyzes the context and keywords. The NLP module uses Hugging Face's Transformers library to extract keywords such as "full bloom" and "beginning to change color" from the text and returns the results to the server.

[1082] 3. Image authenticity assessment and manipulation determination

[1083] The server evaluates the authenticity of the photo based on the analysis results sent from the AI ​​image analysis module. This process uses technology that compares it with previously posted data and evaluates the degree of pattern matching. The image manipulation detection module also determines whether the photo has been digitally altered. Specifically, it checks for unnatural color changes and pixel consistency, and if any abnormalities are found, it applies an algorithm to correct the image.

[1084] 4. Estimating progress

[1085] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at specific tourist spots. It uses AI models such as Scikit-learn and TensorFlow to quantify the condition of each tourist spot and determine the best time to see them.

[1086] 5. Provision to Users

[1087] A user uses a smartphone application to search for the current cherry blossom or autumn foliage conditions at a tourist spot. The device sends the user's request to a server, which retrieves the latest information from a database. The server generates analytical information and related images and provides them to the user. The user can then plan their trip based on the information displayed within the app.

[1088] Specific examples

[1089] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, they open the smartphone app and search for "Ueno Park cherry blossoms." The device then sends this search request to the server. The server retrieves the latest cherry blossom information for Ueno Park from an existing database, analyzes the information, and generates near-real-time images to provide to the user. Based on this, users can visually check the local conditions and efficiently plan their trip.

[1090] Prompt Sentence Examples

[1091] For example, a possible prompt for a generative AI model might be:

[1092] "Please tell me the steps to analyze the progress of cherry blossoms and autumn leaves at tourist spots from social media posts."

[1093] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1094] Step 1:

[1095] Collecting social media posts

[1096] The server periodically collects posts tagged with specific hashtags, such as "cherry blossoms" or "autumn leaves," using the API of the social media platform. Specifically, it uses the Twitter API or Instagram API. The server extracts photos, text, posting date and time, and location information from the posts. The input is the raw posting data obtained from the social media API, and the output is a database entry with the extracted photos, text, posting date and time, and location information. This data is temporarily stored in a database. For example, it includes operations to retrieve data in JSON format and save each field to an SQL database.

[1097] Step 2:

[1098] Image analysis

[1099] The server sends the photos stored in the database to an AI image analysis module. Specifically, it uses a deep learning model using TensorFlow and PyTorch. The input is photo data retrieved from the database, and the output generates metadata indicating the state of cherry blossom blooming and the progress of autumn leaves. For example, it converts images into feature vectors and classifies them as "blooming" or "full bloom" based on those vectors.

[1100] Step 3:

[1101] Text analytics

[1102] The server sends the text of the post to a natural language processing (NLP) module, which analyzes keywords and context. The NLP module uses Hugging Face's Transformers library. The input is the post text, and the output is extracted keywords that indicate the state of the cherry blossoms and autumn leaves (e.g., "in full bloom" and "beginning to change color"). For example, this includes tokenizing the text and identifying keywords by capturing the context using the BERT model.

[1103] Step 4:

[1104] Credibility assessment and manipulation judgment

[1105] The server evaluates the authenticity of the photo based on the analysis results sent from the AI ​​image analysis module. First, the authenticity evaluation evaluates the degree of match with previously posted data and assigns an authenticity score. Next, the image manipulation determination module determines whether or not the photo has been digitally altered. Specifically, the image's pixel data is analyzed using a deep learning model. The input is the analyzed photo data, and the output is an authenticity score and an alteration determination result. If alteration is confirmed, the process also applies an algorithm to return the photo to its original state.

[1106] Step 5:

[1107] Estimating progress

[1108] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at specific tourist spots. The input is the result data of image and text analysis, and the output is a quantified result of the progress of cherry blossoms and autumn leaves at each tourist spot. Specifically, this involves using AI models such as Scikit-learn and TensorFlow to aggregate data points and estimate the progress.

[1109] Step 6:

[1110] Provision to users

[1111] A user uses a smartphone app to search for the current progress of cherry blossoms or autumn leaves at a tourist spot. The device sends the user's request to a server. The server retrieves the latest information from a database, generates analysis results and related images, and provides them to the user. The input is the user's search query, and the output is analysis information and near-real-time images. For example, this includes the actions of an app developed with Flutter or React Native sending a request to a server, receiving a response from the server, and displaying it.

[1112] (Application example 1)

[1113] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1114] In recent years, the spread of social media platforms has increased the number of ways to obtain real-time seasonal landscape information for tourist destinations. However, this information is scattered, making it difficult to centrally collect and provide accurate and reliable information to users. Furthermore, when planning a trip, it is necessary to simultaneously obtain information on seasonal food and beverages and related services, but no such system exists. Given this background, this invention aims to provide a system that accurately obtains the progress of cherry blossoms and autumn leaves at tourist destinations in real time and provides information on seasonal food and beverages based on that information.

[1115] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1116] In this invention, the server includes means for collecting images and posts related to tourist destinations from SNS platforms, means for performing image analysis on the collected images to determine the progress of cherry blossoms and autumn leaves, means for analyzing the text of the collected posts to determine the progress of cherry blossoms and autumn leaves, means for evaluating the authenticity of the images and whether they have been edited and restoring the images to their original state, means for integrating the results of the image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at the tourist destination, means for providing information related to seasonal food and drink based on the estimated progress, and means for providing the estimated progress and information on related food and drink to the user. This allows users to grasp the progress of cherry blossoms and autumn leaves at tourist destinations in real time, make optimal sightseeing plans based on that information, and simultaneously obtain information on seasonal food delivery and food and drink.

[1117] "SNS Platform" means an online social networking service that enables users to share posts such as photos and text.

[1118] A "tourist destination" is a specific place or area that tourists visit.

[1119] "Image analysis" is the process of analyzing image data using computer vision techniques to identify image content and features.

[1120] "Text analytics" is the process of analyzing text data using natural language processing techniques to understand its content and meaning.

[1121] "Credibility" is an indicator of the accuracy and reliability of information or data.

[1122] "Whether or not an image or data has been altered" refers to determining whether or not the image or data has been digitally altered.

[1123] "Estimation" is the prediction of unobserved or future states based on data analysis.

[1124] "Seasonal food and beverages" are foods and beverages that are only available during a particular season.

[1125] "Information provision" means providing useful information to users in a timely manner.

[1126] "User" refers to the general user of this system.

[1127] This invention is a system that provides information on seasonal scenery and food and drink options at tourist destinations. This system collects related images and posts from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves at tourist destinations. It also aims to improve convenience for tourists by simultaneously providing information on seasonal food and drink options.

[1128] Overall system flow:

[1129] 1. Data Collection

[1130] The server periodically uses the API of the social media platform to collect posts with hashtags such as "cherry blossoms" and "autumn leaves."

[1131] Images, text, posting date and time, and location information are extracted from posts and temporarily stored in a database.

[1132] 2. Image Analysis

[1133] The server sends the stored images to an AI image analysis module, which uses deep learning frameworks such as TensorFlow and Keras to determine the progression of cherry blossoms and autumn leaves.

[1134] 3. Text Analysis

[1135] The server sends the text of the post to a natural language processing (NLP) module, which extracts context and keywords to infer the progress of the cherry blossoms and autumn leaves.

[1136] 4. Credibility assessment and manipulation judgment

[1137] The server evaluates the authenticity of the image based on the output of the AI ​​image analysis module, and then the image manipulation determination module determines whether the image has been digitally manipulated, and if so, returns it to its original state.

[1138] 5. Estimating progress

[1139] The server integrates the results of image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot.

[1140] The estimated results are provided to the user in real time, along with information related to seasonal food and drink options for specific tourist destinations.

[1141] 6. Provision to Users

[1142] Users use a smartphone app to search for information on the progress of cherry blossoms and autumn leaves at tourist spots and related food and drink options.

[1143] The terminal sends the user's request to the server, which retrieves the latest information from the database.

[1144] The server generates and provides the user with information about food and drink related to the progress.

[1145] Specific hardware and software

[1146] Hardware: Using cloud platforms (e.g., AWS, Google Cloud), servers perform large-scale data processing.

[1147] Software: TensorFlow and Keras are used for image analysis, and SpaCy and Transformers are used for NLP.

[1148] Specific examples

[1149] For example, if a user wants to know about spring tourist spots in a certain area, the user opens a smartphone app and voice-inputs "tourist spot name cherry blossom food delivery." This prompt is sent to the server, which analyzes the collected SNS data and estimates the latest status of the cherry blossoms at that tourist spot. The analysis results in the information that the cherry blossoms are in full bloom, and related seasonal foods and drinks such as "cherry blossom bento" are recommended. The user can receive a notification that "The cherry blossoms at tourist spot name are currently in full bloom. The recommended food and drink is the cherry blossom bento."

[1150] Prompt Sentence Examples

[1151] "Ueno Park Cherry Blossom Food Delivery"

[1152] "Kyoto Autumn Leaves Recommended Menu"

[1153] As described above, the present invention enables users to plan trips based on real-time information on tourist spots, and also allows users to obtain information on seasonal food and drink all at once.

[1154] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1155] Step 1:

[1156] The server periodically collects posts tagged with hashtags such as "cherry blossoms" and "autumn leaves" via the API of the social media platform. At this time, the server extracts images, text, posting date and time, and location information, and temporarily stores them in a database. The input is the posting information from the social media platform, and the output is the raw data stored in the database.

[1157] Step 2:

[1158] The server sends the images stored in the database to an AI image analysis module. The AI ​​image analysis module (using TensorFlow and Keras) analyzes the image features and determines the progress of cherry blossoms and autumn leaves. The input is images collected from social media, and the output is the progress (e.g., full bloom, beginning to change color) as a result of analysis.

[1159] Step 3:

[1160] The server sends the collected text data to a natural language processing (NLP) module, which analyzes the text's context and keywords. The NLP module uses SpaCy and Transformers to extract keywords that describe the state of the cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color"). The input is the text data collected from social media, and the output is the extracted keywords and their analysis results.

[1161] Step 4:

[1162] The server integrates the results obtained from the AI ​​image analysis module and the NLP module. Specifically, it associates the image analysis results with the text analysis results and estimates the progress of cherry blossoms and autumn leaves at tourist spots based on location information and posting date and time. The inputs are the image analysis results and text analysis results, and the output is the integrated progress data.

[1163] Step 5:

[1164] The server evaluates the authenticity of images and determines whether they have been altered. The authenticity evaluation module assigns an authenticity score to the image, and the image alteration determination module determines whether the image has been digitally altered. If alteration is confirmed, the server applies an image correction algorithm to return the image to its original state. The input is image data collected from SNS, and the output is the authenticity score and the altered image.

[1165] Step 6:

[1166] The server provides information related to seasonal food and drink based on the estimated cherry blossom and autumn foliage progress. This information is linked to the progress of each tourist spot. The input is the integrated progress data, and the output is seasonal food and drink information.

[1167] Step 7:

[1168] A user uses a smartphone app to search for information on the progress of cherry blossoms and autumn leaves at tourist spots and related food and drink options. The device sends the user's request to a server, which retrieves the latest information from a database and provides it. The input is the user's search query, and the output is a notification to the user containing the progress of the tourist spot and related food and drink options.

[1169] This allows users to plan their trips based on real-time information on tourist spots, and also allows them to obtain seasonal food delivery information at the same time.The operation of this system realizes real-time and highly accurate information provision, and provides services that meet user demand.

[1170] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1171] The present invention combines a system that collects photos and posts related to tourist spots from SNS platforms, analyzes them, and estimates and provides the progress of cherry blossoms and autumn leaves in real time, with an emotion engine that recognizes the user's emotions. The following specific forms are conceivable for carrying out the present invention.

[1172] Data collection

[1173] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts include photos, text, posting date and time, and location information, which are temporarily stored in a database.

[1174] Data analysis

[1175] Image analysis

[1176] The server sends the photos stored in the database to an AI image analysis module, which detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color. The analysis results are then sent back to the server along with metadata.

[1177] Text analytics

[1178] The server sends the text of the post to a natural language processing (NLP) module, which analyzes the context and keywords. The NLP module extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the post and analyzes the context.

[1179] Image authenticity assessment and manipulation determination

[1180] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally altered. If manipulation is confirmed, an algorithm is applied to return the photo to its original state and the results are sent back to the server.

[1181] Estimating progress

[1182] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[1183] Provision to users

[1184] Users use a smartphone app to search for the current cherry blossom or autumn foliage conditions at tourist spots. The device sends this request to the server, which retrieves the latest information from the database. The server then generates analytical information and related images and provides them to the user. Users can use the app to check the local conditions based on real-time information and plan their trip.

[1185] Sentiment analysis and its applications

[1186] Emotion Engine

[1187] The server sends the user's posted content and past posted data to the emotion engine. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server. This makes it possible to understand in real time what emotions the user is currently feeling.

[1188] Emotion-based tourist destination recommendation

[1189] The server recommends customized tourist spots based on the user's emotional data. For example, if a user feels like "I want to relax," it can recommend tourist spots with a calm atmosphere. Similarly, if a user feels like "I want to get excited," it can recommend active tourist spots.

[1190] Specific examples

[1191] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, the user opens the smartphone app and searches for "Ueno Park cherry blossoms." The device sends this request to the server, which retrieves the latest cherry blossom information for Ueno Park from an existing database.

[1192] As a result of photo analysis, the information "full bloom" was obtained, so the server provides the user with the latest information, "The cherry blossoms in Ueno Park are currently in full bloom." Furthermore, the server generates near real-time images and displays them on the app, allowing the user to visually check the local situation.

[1193] In addition, by analyzing the user's emotional data, if the user feels like "relaxing," the server can recommend quiet tourist spots and places where they can relax. This allows the user to plan a trip that suits their state of mind.

[1194] As described above, the present invention not only makes it easier for users to plan their travel destinations based on highly accurate, real-time information, but also allows users to choose tourist destinations that suit their own emotions, resulting in a more satisfying travel experience.

[1195] The processing flow will be explained below.

[1196] Step 1:

[1197] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts include photos, text, posting date and time, and location information.

[1198] Step 2:

[1199] The server extracts photo data from the collected posts and sends it to an AI image analysis module. The AI ​​image analysis module detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color. The analysis results are then sent back to the server along with metadata.

[1200] Step 3:

[1201] The server sends the text data of the post to a natural language processing (NLP) module. The NLP module extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the text and analyzes the context. The analysis results are sent back to the server.

[1202] Step 4:

[1203] The server evaluates the authenticity of the photo based on the results from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo and sends the result back to the server.

[1204] Step 5:

[1205] The server sends the photo to the image manipulation determination module to check whether the image has been manipulated. If it is found to have been manipulated, the image manipulation determination module processes the photo to return it to its original state and sends the converted image to the server.

[1206] Step 6:

[1207] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[1208] Step 7:

[1209] A user uses a smartphone app to search for the current cherry blossom or autumn foliage status of a tourist spot, and the device sends this request to the server.

[1210] Step 8:

[1211] The server retrieves the latest information on tourist spots from the database in response to user requests, and generates images related to the progress of cherry blossoms and autumn leaves based on the retrieved information.

[1212] Step 9:

[1213] The server responds to the user with the current cherry blossom and autumn foliage conditions at the tourist destination and the generated images. The user can then use the smartphone app to check the local conditions based on real-time information and plan their trip.

[1214] Step 10:

[1215] The server sends data to the emotion engine to analyze the user's past posts and current emotions. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server.

[1216] Step 11:

[1217] The server recommends customized tourist spots based on the user's emotional data. For example, if the user feels like "I want to relax," the server will recommend quiet tourist spots and places where they can relax. Information about tourist spots that match the user's emotions is sent to the server.

[1218] Step 12:

[1219] Users can check tourist destination information provided through the smartphone app and create travel plans that suit their own preferences, resulting in a more satisfying travel experience.

[1220] Example 2

[1221] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1222] Conventional tourist information systems have had difficulty accurately grasping the progress of local plants (e.g., cherry blossoms and autumn leaves) in tourist destinations in real time. Furthermore, since tourist destination recommendations are not tailored to the user's emotions, satisfaction with travel plans can decrease. The purpose of this invention is to solve these problems.

[1223] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data related to tourist destinations from SNS platforms, means for performing image analysis on the collected data to determine the progress of plants, means for analyzing text in the collected data to determine the progress of plants, means for evaluating the authenticity of the image and whether it has been edited and restoring the image to its original state, means for integrating the results of the image analysis and text analysis to estimate the progress of plants at the tourist destination, means for providing the estimated progress to the user, and means for analyzing the user's emotions and recommending tourist destinations based on the results. This makes it possible to accurately grasp the progress of tourist destinations in real time and to recommend tourist destinations customized based on the user's emotions.

[1224] "SNS platform" refers to an online service that allows a large number of users to post and interact with each other.

[1225] "Data" includes information collected from social media platforms, such as photos, text, posting dates and times, and location information.

[1226] "Image analysis" refers to the process of detecting specific features from collected photographic data and determining the condition based on those features.

[1227] "Plant progress" refers to the extent to which plants such as cherry blossoms and autumn leaves have flowered or changed color.

[1228] "Text analysis" refers to the process of analyzing the context and keywords of collected text data to extract specific information.

[1229] "Authenticity" refers to the degree of confidence that a photograph or text accurately reflects the actual situation.

[1230] "Presence or absence of manipulation" refers to the state of determining whether a photograph or text has been digitally altered.

[1231] "Reverting" refers to the process of restoring digitally altered photographs or text to their original state.

[1232] "Integration" refers to combining the results of image analysis and text analysis into a single comprehensive piece of information.

[1233] "Tourist destination" refers to an area or facility that users visit for sightseeing.

[1234] "Estimation" refers to predicting information such as the progress of plants based on collected data.

[1235] "User" refers to the end user who uses this system to obtain tourist information.

[1236] "Providing" refers to displaying or notifying the user of the estimated information.

[1237] "Sentiment analysis" refers to the process of analyzing a user's posts and past data to understand their emotional state at the time.

[1238] "Recommendation" refers to selecting and presenting appropriate tourist destinations based on the user's emotional state.

[1239] The present invention combines a system that collects data related to tourist destinations from SNS platforms, analyzes the data, and estimates and provides the progress of plants in real time, with an emotion engine that recognizes the user's emotions. The following specific forms are conceivable for carrying out the present invention.

[1240] Data collection

[1241] The server periodically sends requests to the API of the social media platform to collect posts containing hashtags such as "cherry blossoms" or "autumn leaves." The collected data includes photos, text, posting date and time, and location information, which is temporarily stored in a database.

[1242] Image analysis

[1243] The server sends the photos stored in the database to an AI image analysis module, which uses deep learning technology to detect plant characteristics in the photos and determine their state, such as full bloom or color. The analysis results, along with metadata, are then sent back to the server.

[1244] Text analytics

[1245] The server sends the text of the post to a natural language processing (NLP) module, which analyzes the context and keywords. The NLP module extracts keywords related to the plant's progress (e.g., "full bloom" or "beginning to change color") from the post and analyzes the context.

[1246] Image authenticity assessment and manipulation determination

[1247] The server evaluates the authenticity of the photo based on the results sent from the image analysis module. The authenticity assessment module assigns an authenticity score to the photo, and then the image manipulation determination module determines whether the photo has been digitally manipulated. If manipulation is confirmed, an algorithm is applied to restore the image to its original state and the result is sent back to the server.

[1248] Estimating progress

[1249] The server integrates the results of image and text analysis to estimate the progress of plants at each tourist spot, and uses an AI model to evaluate the current state of the tourist spot in real time, quantifying the progress, and determining the best viewing conditions.

[1250] Provision to users

[1251] Users use a smartphone app to search for the current plant status of a tourist attraction. The device sends this request to the server, which retrieves the latest information from the database. The server generates analytical information and related images and provides them to the user. Users can use the app to check the local situation based on real-time information and plan their trip.

[1252] Sentiment analysis and its applications

[1253] Emotion Engine

[1254] The server sends the user's posted content and past posted data to the emotion engine. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server. This makes it possible to understand the user's current emotions in real time.

[1255] Emotion-based tourist destination recommendation

[1256] The server recommends customized tourist spots based on the user's emotional data. For example, if a user feels like "I want to relax," the server can recommend tourist spots with a calm atmosphere. Similarly, if a user feels like "I want to get excited," the server can recommend active tourist spots.

[1257] Specific examples

[1258] For example, if a user wants to know the status of the cherry blossoms in Ueno Park, Tokyo, they open a smartphone app and search for "Ueno Park cherry blossoms." The device sends this request to the server, which retrieves the latest cherry blossom information for Ueno Park from its existing database. If the photo analysis results indicate that the cherry blossoms are in full bloom, the server provides the user with the latest information: "The cherry blossoms in Ueno Park are currently in full bloom." Furthermore, the server generates near-real-time images and displays them in the app, allowing the user to visually check the current situation. Furthermore, by analyzing the user's emotional data, if the user feels like "relaxing," the server can recommend quiet tourist spots and relaxing places. This allows users to create travel plans that suit their individual needs.

[1259] The specific hardware and software used

[1260] Hardware: Servers, devices (smartphones, tablets), etc.

[1261] Software: Social media platform API, AI image analysis module, natural language processing (NLP) module, emotion engine

[1262] Other specific technology examples include TensorFlow, Google Cloud Natural Language API, MySQL, and PostgreSQL.

[1263] Example prompts for generative AI models

[1264] "I would like to know the current state of the cherry blossoms in Ueno Park in Tokyo."

[1265] Please recommend some relaxing tourist spots.

[1266] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1267] Step 1: Data collection

[1268] The server periodically sends requests to the API of the social media platform. As input data, it collects posts containing specific hashtags (e.g., "cherry blossoms" or "autumn leaves"). The collected data includes photos, text, posting date and time, and location information. The server temporarily stores the collected data in a database. Specifically, it uses the API of the social media platform to search for related posts, obtains the data in JSON format, and stores it in the database.

[1269] Step 2: Image analysis

[1270] The server sends the photo data stored in the database to the AI ​​image analysis module. By providing the photo as input data, the image analysis module uses deep learning technology to detect the characteristics of the plants in the photo. As output, the state of full bloom and color is returned along with metadata. Specifically, an image analysis model using TensorFlow is used to determine the full bloom state of the cherry blossoms, and metadata such as "75% full bloom" is returned.

[1271] Step 3: Text analysis

[1272] The server sends the collected text data of posts to a natural language processing (NLP) module. By providing the text as input data, the NLP module analyzes the context and keywords and extracts keywords related to the plant's progress (e.g., "in full bloom" or "beginning to change color"). The extracted keywords and their analysis results are returned as output. Specifically, the Google Cloud Natural Language API is used to extract the keyword "in full bloom" from the text and understand its context.

[1273] Step 4: Assess the authenticity of the image and determine whether it has been edited

[1274] The server evaluates the authenticity of the photo based on the results of image analysis. The image analysis results are provided as input data, and the authenticity assessment module assigns an authenticity score to the photo. The image manipulation determination module then determines whether the photo has been digitally manipulated. The output provides an authenticity score and a manipulation determination result. If manipulation is confirmed, an algorithm is applied to return the photo to its original state. Specifically, the Adobe Photoshop API is used to evaluate the authenticity of the image at 87%, detect traces of manipulation, and restore it to its original state if necessary.

[1275] Step 5: Estimating progress

[1276] The server combines the results of image analysis and text analysis. It provides both analysis results as input data and uses an AI model to evaluate the progress of the plants at the tourist destination in real time. As an output, the progress is quantified and the best viewing state is determined. Specifically, the AI ​​model combines past and present data to predict the progress of the plants as "85% full bloom."

[1277] Step 6: Inform users

[1278] A user uses a smartphone app to search for the current plant progress at a tourist spot. The device sends this request to the server. The server retrieves the latest information from the database, generates analysis information and related images, and provides them to the user. The system receives the user's request as input data and provides the analysis results and images as output. Specifically, when a user searches for "Ueno Park cherry blossoms," the server responds with "Currently 85% in full bloom," and also displays an image of the actual location.

[1279] Step 7: Sentiment analysis and its applications

[1280] The server sends the user's posted content and past data to the emotion engine. Text data is provided as input, and the emotion engine uses NLP technology to analyze the user's emotions. The analysis results, or emotional state, are sent back to the server as output. Based on the results, customized tourist destination recommendations are made. For example, if the user feels like "relaxing," tranquil tourist destinations are recommended. Specifically, if the user's emotion is determined to be "relaxed," the server will recommend quiet tourist destinations.

[1281] (Application example 2)

[1282] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1283] Conventional tourist destination information systems have limitations in terms of real-timeness and accuracy, making it particularly difficult to grasp seasonal information such as the progress of cherry blossoms and autumn leaves. Furthermore, they lacked the ability to recommend tourist destinations based on user emotions and to link with real-world store information, preventing user satisfaction. There is a need for a system that can solve these problems, provide users with highly accurate tourist information in real time, and also recommend tourist destinations based on emotions and provide information on nearby real-world stores.

[1284] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting photos and posted content related to tourist destinations from SNS platforms, means for performing image analysis on the collected photos to determine the progress of cherry blossoms and autumn leaves, means for analyzing the text of the collected posted content to determine the progress of cherry blossoms and autumn leaves, means for evaluating the authenticity of the image and whether it has been edited and restoring the image to its original state, means for integrating the results of the image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at the tourist destination, means for providing the estimated progress to the user, means for analyzing the user's emotions and recommending tourist destinations based on the emotions, and means for providing information on brick-and-mortar stores near the tourist destination. This allows users to not only obtain highly accurate tourist information in real time, but also receive recommendations for tourist destinations and information on brick-and-mortar stores in the vicinity that match their emotions at any given time.

[1285] "SNS Platform" means an online platform that provides social networking services, where users can post content such as photos and text to share with other users.

[1286] A "tourist destination" is a specific geographical location or region that tourists visit, and includes natural scenery, cultural assets, leisure facilities, etc.

[1287] A "photograph" is a still image recorded using a photographic device such as a camera, which visually captures the state and characteristics of the tourist destination.

[1288] "Posted content" refers to all content, including text, images, and videos, posted by users on social media platforms.

[1289] "Image analysis" is the process of analyzing the content of an image using computer vision techniques to detect and classify specific objects or features.

[1290] "Progression of cherry blossoms and autumn leaves" refers to the state and stage of progress of plants that change with the seasons, such as cherry blossoms blooming and reaching full bloom, and the color of autumn leaves.

[1291] "Text analysis" is the process of analyzing text data using natural language processing technology to evaluate and extract its content, context, sentiment, etc.

[1292] "Credibility" is the concept of assessing whether collected data or information is accurate and truthful.

[1293] "Manifold" refers to whether an image or text has been digitally altered or modified.

[1294] "Progress estimation" is the process of numerically or qualitatively assessing the current state of cherry blossoms and autumn leaves at tourist spots based on analyzed image and text data.

[1295] "Providing to the user" refers to the act of sending the analyzed information to the user's device and displaying it in visual or text format.

[1296] "Analyzing user emotions" is the process of assessing users' emotions and moods from social media and other digital content using natural language processing.

[1297] "Tourist destination recommendation" is the act of selecting and suggesting appropriate tourist destinations and spots to visit based on the user's current emotions and preferences.

[1298] "Information about nearby physical stores" refers to information that includes details about commercial facilities and service providers (e.g., cafes, souvenir shops, etc.) located around tourist destinations.

[1299] This invention is a system that collects photos and posts related to tourist spots from social media platforms, analyzes them, and estimates and provides real-time information on the progress of cherry blossoms and autumn leaves.It also recognizes the user's emotions and provides recommendations on tourist spots and brick-and-mortar store information based on those emotions.

[1300] The server periodically sends requests to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The collected posts include photos, text, posting date and time, and location information, which are temporarily stored in a database.

[1301] The server then sends the photos stored in the database to an AI image analysis module, which detects the characteristics of the cherry blossoms and autumn leaves in the photos and determines their state, such as full bloom and color. The analysis results are then sent back to the server along with metadata.

[1302] The server then sends the text of the post to a natural language processing (NLP) module for context and keyword analysis. The NLP module extracts keywords related to the state of cherry blossoms and autumn leaves (e.g., "full bloom" and "beginning to change color") from the post and analyzes the context.

[1303] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally altered. If manipulation is confirmed, an algorithm is applied to return the photo to its original state and the results are sent back to the server.

[1304] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The AI ​​model evaluates the current state of the tourist spot in real time, quantifies the progress, and determines the best viewing conditions.

[1305] Users use a smartphone app to search for the current cherry blossom or autumn foliage conditions at tourist spots. The device sends this request to the server, which retrieves the latest information from the database. The server then generates analytical information and related images and provides them to the user. Users can use the app to check the local conditions based on real-time information and plan their trip.

[1306] The server sends the user's posted content and past posted data to the emotion engine. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server. This makes it possible to understand in real time what emotions the user is currently feeling.

[1307] The server recommends customized tourist spots based on the user's emotional data. For example, if a user feels like "I want to relax," it can recommend tourist spots with a calm atmosphere. Similarly, if a user feels like "I want to get excited," it can recommend active tourist spots.

[1308] This will provide users with highly accurate, real-time information, making it easier to plan trips, and will also enable recommendations of tourist spots based on the user's emotions. Furthermore, real-time information on nearby brick-and-mortar stores will also be provided, improving the overall tourist experience.

[1309] For example, if a user searches for "Ueno Park cherry blossoms," the system will provide real-time information on the current cherry blossom conditions in Ueno Park. Along with information such as "They're in full bloom," it will also display information on nearby cafes and souvenir shops. If the user feels like "I want to relax," the system will recommend quiet tourist spots and cafes.

[1310] Examples of prompts include:

[1311] "Please tell me the latest information about the cherry blossoms in Ueno Park and places to relax in the area."

[1312] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1313] Step 1:

[1314] The server sends a request to the social media platform API to collect posts containing hashtags such as "cherry blossoms" and "autumn leaves." The input is the social media platform's API URL and a specific hashtag, and the output is post data including photos, text, posting date and time, and location information. This post data is temporarily stored in a database.

[1315] Step 2:

[1316] The server sends the photos stored in the database to the AI ​​image analysis module. The input is the photo data, and the output is metadata including the results of detecting the characteristics of cherry blossoms and autumn leaves and their status (e.g., full bloom, color change). The AI ​​image analysis module analyzes the photo and sends the results back to the server.

[1317] Step 3:

[1318] The server sends the text of the post to a natural language processing (NLP) module to analyze the context and keywords. The input is the text of the SNS post, and the output is keywords and context analysis results related to the progress of cherry blossoms and autumn leaves (e.g., "in full bloom" and "beginning to change color").

[1319] Step 4:

[1320] The server evaluates the authenticity of the photo based on the results sent from the AI ​​image analysis module. The authenticity evaluation module assigns an authenticity score to the photo. The input is the result of the AI ​​image analysis, and the output is an authenticity score. Next, the image manipulation determination module analyzes the photo and determines whether it has been digitally manipulated. The input is the photo data for manipulation determination, and the output is a manipulation score. If manipulation is confirmed, an algorithm is applied to return the photo to its original state, and the photo is sent back to the server.

[1321] Step 5:

[1322] The server integrates the results of image and text analysis to estimate the progress of cherry blossoms and autumn leaves at each tourist spot. The input is the results of image analysis and text analysis, and the output is evaluation data that quantifies the progress of each tourist spot. The AI ​​model evaluates the current state of tourist spots in real time and determines the best viewing conditions.

[1323] Step 6:

[1324] A user uses a smartphone app to search for the current cherry blossom and autumn foliage conditions at a tourist spot. The input is the user's search query (e.g., "Ueno Park cherry blossoms"), and the output is the latest information about the tourist spot (e.g., "In full bloom" and related images). The device sends the request to the server, which retrieves the latest information from the database, generates analytical information and related images, and provides them to the user.

[1325] Step 7:

[1326] The server sends the user's posted content and past posted data to the emotion engine. The input is the user's posted data, and the output is the result of analyzing the user's emotions. The emotion engine uses NLP technology to analyze the user's emotions (e.g., joy, excitement, anticipation, etc.) and sends the results back to the server.

[1327] Step 8:

[1328] The server recommends customized tourist spots based on the user's emotional data. The input is the user's emotional data, and the output is a list of recommended tourist spots. For example, if the user feels like "I want to relax," tourist spots with a calm atmosphere will be recommended. Similarly, if the user feels like "I want to get excited," tourist spots with an active atmosphere will be recommended.

[1329] Step 9:

[1330] The server provides information about brick-and-mortar stores around tourist spots. The input is the location information of the tourist spot and filtering conditions based on the user's interests, and the output is information about brick-and-mortar stores such as nearby cafes and souvenir shops. Users can obtain this information through the app and check it in real time.

[1331] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1332] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1333] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1334] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1335] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1336] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1337] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1338] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1339] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1340] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1341] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1342] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1343] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1344] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1345] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1346] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1347] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1348] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1349] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1350] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1351] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1352] The following is further disclosed regarding the above embodiment.

[1353] (Claim 1)

[1354] A means for collecting photos and posts related to tourist destinations from social media platforms;

[1355] A method for analyzing collected photos to determine the progress of cherry blossoms and autumn leaves,

[1356] A method for analyzing the text of collected posts to determine the progress of cherry blossoms and autumn leaves,

[1357] a means for assessing the authenticity and alteration of the image and restoring the image to its original state;

[1358] A means for integrating the results of the image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at tourist spots;

[1359] and a means for providing the estimated progress to the user.

[1360] (Claim 2)

[1361] The system according to claim 1, further comprising means for identifying tourist spots and estimating progress status based on location information and posting date and time of photos and posted content collected from the SNS platform.

[1362] (Claim 3)

[1363] 2. The system according to claim 1, wherein the means for providing to the user includes means for generating and providing an image in near real time.

[1364] "Example 1"

[1365] (Claim 1)

[1366] A means for collecting photos and posts related to tourist destinations from social media platforms;

[1367] A method to use an AI image analysis module to determine the progress of cherry blossoms and autumn leaves from collected photos;

[1368] A method for analyzing the collected text of posts using a natural language processing module to determine the progress of cherry blossoms and autumn leaves.

[1369] A means for evaluating the authenticity and whether an image has been manipulated using an AI image analysis module and an image manipulation determination module, and for restoring the image to its original state if necessary;

[1370] A method for integrating the results of the image analysis and text analysis and using an AI model to estimate the progress of cherry blossoms and autumn leaves at tourist spots.

[1371] and means for providing the estimated progress to a user.

[1372] (Claim 2)

[1373] The system according to claim 1, further comprising means for identifying tourist spots based on location information and posting dates and times of photos and posted content collected from the SNS platform, and estimating the progress of the tourist spots.

[1374] (Claim 3)

[1375] 2. The system according to claim 1, wherein the means for providing to the user includes means for generating and providing an image in near real time.

[1376] "Application Example 1"

[1377] (Claim 1)

[1378] A means for collecting images and posts related to tourist destinations from social media platforms;

[1379] A method for anal...

Claims

1. A means for collecting photos and posts related to tourist destinations from social media platforms; A method for analyzing collected photos to determine the progress of cherry blossoms and autumn leaves, A method for analyzing the text of collected posts to determine the progress of cherry blossoms and autumn leaves, a means for assessing the authenticity and alteration of the image and restoring the image to its original state; A means for integrating the results of the image analysis and text analysis to estimate the progress of cherry blossoms and autumn leaves at tourist spots; and a means for providing the estimated progress to the user.

2. The system according to claim 1 , further comprising means for identifying tourist spots and estimating progress status based on location information and posting date and time of photos and posted content collected from the SNS platform.

3. 2. The system according to claim 1, wherein said means for providing to said user includes means for generating and providing an image in near real time.

Images