System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving a search condition input from a user terminal; means for acquiring current position information related to the search condition; means for extracting a related sightseeing spot from a database based on the search condition and the position information; means for ranking the extracted sightseeing spot based on a preference of a user; means for generating a short video for the ranked sightseeing spot; and means for transmitting the generated short video and ranking information to the user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional guidebooks and special feature websites require users to spend a lot of time searching through a large amount of information to find the tourist spots they want to visit, making it difficult to visually understand their appeal. Furthermore, there is a lack of methods for providing recommended information that reflects the user's current location information and hobbies and preferences in real time, resulting in a lack of realism. The present invention aims to solve these problems and provide users with the most suitable tourist spots in an efficient and attractive format. [Means for solving the problem]

[0005] The present invention provides a system including means for receiving search criteria entered from a user terminal, means for acquiring current location information related to the search criteria, means for extracting relevant tourist spots from a database based on the search criteria and location information, means for ranking the extracted tourist spots based on the user's hobbies and preferences, means for generating short videos of the ranked tourist spots, and means for transmitting the generated short videos and ranking information to the user terminal, thereby enabling users to easily find the tourist spots that best suit them and visually understand their appeal.

[0006] A "user terminal" is an electronic device operated by a user, such as a computer, smartphone, or tablet.

[0007] "Search conditions" are requirements and desires regarding tourist spots entered by the user, such as time restrictions, location, weather conditions, and other information.

[0008] "Location information" is data indicating the user's current location, and is obtained by GPS or the like.

[0009] A "database" is a system that stores information about tourist spots and is configured to allow efficient data search and extraction.

[0010] "Tourist spots" are places or facilities that users visit, including tourist spots and famous places.

[0011] "Ranking" is the process of prioritizing the extracted tourist spots based on the user's hobbies, preferences, and past behavioral history.

[0012] A "short video" is a video created to visually introduce the attractions of tourist spots in a short amount of time, and is designed to attract the user's interest.

[0013] The "generative model" is an AI model for creating short videos that automatically picks out the main attractions of each tourist spot and turns them into video format.

[0014] The term "system" refers to a combination of a series of hardware and software configured to execute the above means. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] To specifically implement the present invention, it is necessary to build a system that includes a user terminal, a server, and various communication means. The purpose of this system is to make it easier for users to search for tourist spots and visually understand their attractions. The program processing of this system is as follows.

[0037] User terminal operation

[0038] 1. Displaying the input form

[0039] The user terminal displays a form for the user to input search criteria, such as the time, location, and specific preferences (e.g., natural beauty, historical sites, etc.) that the user wants to visit.

[0040] 2. Obtaining location information

[0041] The user device obtains the user's current location information using the device's built-in GPS function.

[0042] 3. Data Transmission

[0043] The user terminal transmits the search conditions entered by the user and the acquired location information to the server.

[0044] Server Operation

[0045] 1. Data Reception

[0046] The server receives the search conditions and location information sent from the user terminal.

[0047] 2. Database Matching

[0048] The server compares the received data with a database of tourist spots, and selects tourist spots that match the user's current location and search criteria.

[0049] 3. Filtering tourist attractions

[0050] The server filters the extracted tourist spots based on the user's hobbies, preferences, and past behavioral history, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[0051] 4. Short video generation

[0052] The server uses a generative AI model to generate short videos for each selected tourist spot. Each video is designed to convey the characteristics and appeal of the tourist spot in a short time of around 30 seconds.

[0053] 5. Data Transmission

[0054] The server transmits the generated short video and ranking information to the user terminal.

[0055] User terminal operation (continued)

[0056] 1. Data Reception

[0057] The user terminal receives the short video and ranking information transmitted from the server.

[0058] 2. Video playback and information display

[0059] The user device plays the received short video and displays the tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[0060] 3. Selecting tourist spots

[0061] Users can select the tourist spots they want to visit based on the displayed information, which helps users to plan their sightseeing trips comfortably and efficiently.

[0062] Specific examples

[0063] User Behavior

[0064] For example, suppose a user is searching for "tourist spots that can be enjoyed within an hour around Shinjuku Station." The user enters "1 hour" into the input form on the device and confirms that the current location is Shinjuku Station.

[0065] Device behavior

[0066] The device uses its GPS function to obtain the user's location information and sends the conditions "Shinjuku Station" and "1 hour" to the server.

[0067] Server Operation

[0068] After receiving the data, the server extracts tourist spots that can be reached and returned from Shinjuku Station within an hour from the database. Then, taking into consideration the user's past search history and hobbies (e.g., a love of nature), it selects and ranks Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street in Harajuku. It then generates short videos showcasing the attractions of these tourist spots and sends them to the user's device.

[0069] User terminal operation (continued)

[0070] The device plays the received short video and displays ranking information to the user, allowing the user to select Meiji Jingu.

[0071] In this way, a system embodying the present invention allows a user to intuitively and efficiently find the best tourist spots.

[0072] The processing flow will be explained below.

[0073] User terminal processing

[0074] Step 1:

[0075] The user inputs search conditions (for example, desired time and location) into a search form displayed on the terminal.

[0076] Step 2:

[0077] The device uses the GPS function to obtain current location information, which is then stored in the internal memory.

[0078] Step 3:

[0079] The terminal transmits the search conditions entered by the user and the acquired location information to the server.

[0080] Server Processing

[0081] Step 4:

[0082] The server receives the search conditions and location information sent from the user terminal.

[0083] Step 5:

[0084] The server checks the tourist spot database based on the received search criteria and location information. For example, it extracts relevant tourist spots based on the criteria "within one hour from Shinjuku Station."

[0085] Step 6:

[0086] The server filters the extracted tourist spots based on the user's hobbies, preferences, and past behavioral history, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[0087] Step 7:

[0088] The server uses a generative AI model to generate short videos for each ranked tourist spot, each of which is about 30 seconds long and introduces the main attractions of the tourist spot.

[0089] Step 8:

[0090] The server transmits the generated short video and ranking information to the user terminal.

[0091] User terminal processing (continued)

[0092] Step 9:

[0093] The terminal receives the short video and ranking information transmitted from the server.

[0094] Step 10:

[0095] The device plays the received short video to the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[0096] Step 11:

[0097] The user selects tourist spots they would like to visit based on the provided information, and the selected information can be used to help the user plan their trip.

[0098] In this way, tourist spots that meet the user's current needs are efficiently provided.

[0099] Example 1

[0100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] In the past, there were few ways for users to efficiently search for tourist spots and intuitively understand their attractions. As a result, it took a lot of time and effort for users to find the appropriate tourist spot based on their hobbies, preferences, and current location information. Furthermore, since the information provided was text-based, it was difficult to understand visually.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0103] In this invention, the server includes means for receiving search criteria entered from a user terminal, means for acquiring current location information related to the search criteria, means for extracting relevant tourist spots from a database based on the search criteria and location information, means for ranking the extracted tourist spots based on the user's hobbies and preferences, means for generating short videos for the ranked tourist spots using a generative AI model, and means for transmitting the generated short videos and ranking information to the user terminal, thereby enabling users to intuitively and efficiently find the best tourist spots.

[0104] "Search conditions" are information including the place and time the user wants to visit, and specific desired conditions (e.g., a place rich in nature, a historical place, etc.).

[0105] "Location information" is data that indicates the user's current geographical location, and is mainly obtained using the GPS function.

[0106] "Tourist attraction" refers to a place or facility that a user wishes to visit, the place or facility being publicly known for tourism or recreational purposes.

[0107] A "database" is a system for efficiently storing, accessing, and managing large amounts of information, including information about tourist attractions.

[0108] "Hobbies and preferences" refers to a user's personal tastes and interests, and are determined based on past behavioral history and profile data.

[0109] A "generative AI model" is an artificial intelligence model that uses machine learning techniques to generate new data (e.g., short videos). Specifically, it includes techniques for generating videos from text and images.

[0110] A "short video" is a video that conveys the appeal and characteristics of a tourist spot within a short period of time (usually 30 seconds or less).

[0111] "Ranking information" is a list that ranks the attractions of tourist spots based on the user's hobbies, preferences, and search conditions.

[0112] "User terminal" refers to an electronic device that is directly operated by a user, including smartphones, tablets, and personal computers.

[0113] A "server" is a central computer system that receives and processes data sent from user terminals, generates the necessary information, and retransmits it.

[0114] The present invention is implemented by constructing a system including a user terminal, a server, and various communication means. The purpose of this system is to enable users to easily search for tourist spots and visually understand their attractions.

[0115] User terminal operation

[0116] The user device first displays a form for entering search criteria for tourist attractions. This allows the user to enter the time, location, and specific preferences for the desired visit (e.g., natural beauty, historical sites, etc.). The user device also uses its built-in GPS to obtain the user's current location. This information is sent to the server and used to improve the accuracy of search results.

[0117] Server Operation

[0118] The server receives the search criteria and location information sent from the user's device. Based on the received data, the server checks a database of tourist spots and extracts relevant tourist spots. The server then filters the extracted list of tourist spots based on the user's hobbies, preferences, and past behavioral history, and organizes them in a ranking format. Based on this ranking information, the server uses a generative AI model (e.g., OpenAI's GPT-4 (registered trademark)) to generate short videos. Each video is approximately 30 seconds long and conveys the characteristics and appeal of the tourist spot. Finally, the server sends the generated short videos and ranking information to the user's device.

[0119] User terminal operation (continued)

[0120] The user device receives the short videos and ranking information sent from the server. This information is used to visually convey the attractions of tourist spots to the user. By playing the received short videos and displaying the ranking information, the user can intuitively and efficiently select tourist spots. This information is useful when the user selects tourist spots they want to visit.

[0121] Example operation

[0122] For example, consider the case where a user is searching for "fun tourist spots around Shinjuku Station within an hour." The user enters "1 hour" into the input form on their device and checks their current location. The device then uses its GPS function to obtain location information and sends the conditions "Shinjuku Station" and "1 hour" to the server. After receiving these conditions, the server extracts matching tourist spots from its database. Taking into consideration past search history and the user's hobbies, it selects Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street, creates short videos showcasing the appeal of each tourist spot, and sends them to the user's device. The device then plays these videos and displays ranking information. The user can then view these videos and select the spot they want to visit (for example, Meiji Jingu Shrine).

[0123] Example prompts to input to the generative AI model

[0124] "Please introduce tourist spots that can be enjoyed within an hour around Shinjuku Station. Please create a short video of 30 seconds or less that introduces the appeal of each spot. Please also take into consideration your past search history and hobbies (nature lovers)."

[0125] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0126] Step 1:

[0127] The user device displays a form for entering search criteria for tourist spots. Here, the user enters the desired time, location, and specific desired conditions for visiting. At this point, the input is the time (e.g., 1 hour), location (e.g., around Shinjuku Station), and desired conditions (e.g., a place rich in nature).

[0128] Step 2:

[0129] The user device obtains the current location information using the built-in GPS function. This is the operation of obtaining the current location data provided by the user device. The obtained location information (e.g., latitude and longitude data) is used for subsequent processing.

[0130] Step 3:

[0131] The user terminal transmits the input search criteria and the acquired location information to the server. The transmitted data includes the time, location, desired criteria, and current location information.

[0132] Step 4:

[0133] The server receives the search criteria and location information sent from the user terminal. The received data is temporarily stored in the server's memory and used in the next database matching process.

[0134] Step 5:

[0135] The server checks the tourist attraction database based on the received search criteria and location information. This process uses an SQL query to extract matching tourist attraction records from the database. The output data is a list of tourist attractions that match the search criteria.

[0136] Step 6:

[0137] The server filters the extracted tourist spot list based on the user's hobbies, preferences, and past behavioral history. The user profile data is used as input data, and a ranking list of tourist spots optimized for the user is generated as output.

[0138] Step 7:

[0139] The server generates short videos for the selected tourist spots using a generative AI model (e.g., OpenAI's GPT-4). In this process, tourist spot information is converted into prompt sentences as input data, and the generative AI model generates the video. The output data is a short video of about 30 seconds that conveys the characteristics of the tourist spot.

[0140] Step 8:

[0141] The server transmits the generated short video and ranking information to the user terminal. The transmitted data includes the short video file and the ranking information.

[0142] Step 9:

[0143] The user terminal receives the short video and ranking information sent from the server, and the received data is stored in the memory of the user terminal.

[0144] Step 10:

[0145] The user device plays the received short video and displays the ranking information. This process uses a media player for video playback and a UI for displaying the rankings. By viewing this, the user can intuitively understand the appeal of the tourist spot.

[0146] Step 11:

[0147] The user selects the tourist spots they wish to visit based on the displayed information, and this selection information is saved in the user's terminal and used for subsequent activities.

[0148] (Application example 1)

[0149] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0150] Modern logistics centers and industrial facilities are required to maximize worker efficiency and improve work speed and accuracy. However, there is a lack of immediate and intuitive guidance systems to achieve this. Furthermore, few systems can present optimal work patterns using individual worker location information and past behavioral history. Tourism information systems also lack visual representations that allow users to intuitively select the best tourist spots.

[0151] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0152] In this invention, the server includes means for receiving search criteria and worker location information input from a user terminal, means for extracting related tourist spots and work patterns from a database based on the search criteria and location information, means for ranking the extracted tourist spots and work patterns based on the user's hobbies and preferences and the worker's efficiency, means for generating short videos of the ranked tourist spots and work patterns, and means for transmitting the generated short videos and ranking information to the user terminal. This allows users to intuitively and efficiently select tourist spots and enables workers to efficiently identify optimal work patterns.

[0153] A "user terminal" is an electronic device used by a user, such as a computer, smartphone, or tablet.

[0154] "Search conditions" are desired conditions and requirements for tourist spots and work patterns entered by the user.

[0155] "Location information" is data indicating the current location of a user or worker, and is obtained using technologies such as GPS.

[0156] A "database" is a collection of information that stores information on tourist spots and work patterns and can be searched and extracted.

[0157] "Extraction means" is a function that extracts necessary information from a database based on specific conditions.

[0158] The "ranking means" is a function that ranks the extracted tourist spots and work patterns based on the user's preferences and the efficiency of the workers.

[0159] "Means for generating" refers to the function of creating visual content such as short videos based on related information.

[0160] The "transmitting means" is a communication function for sending the generated data or information to the user terminal.

[0161] A "system" is a collection of components that includes user terminals, servers, communication means, etc., and that as a whole realizes a specific function.

[0162] "Available time" is the time frame in which a user or worker can perform a particular activity.

[0163] "Weather conditions" is information that indicates the weather conditions in a particular area.

[0164] "Past search history" is a record of searches previously performed by a user.

[0165] "Behavioral history" is a record of the actions and work a worker has performed in the past.

[0166] "Mood state" is information that indicates the user's current state of mind or emotions.

[0167] A "generative model" is a program that uses specific algorithms and machine learning techniques to automatically generate materials such as short videos.

[0168] "Tourist spots" are tourist destinations and famous places that users can visit.

[0169] A "work pattern" is a combination of procedures and movements that allow workers to perform work efficiently.

[0170] To specifically implement this invention, it is necessary to build a system including a user terminal, a server, and various communication means. The purpose of this system is to make it easier for users to search for tourist spots and workers' work patterns and to visually understand their attractiveness and efficiency.

[0171] User terminal operation

[0172] 1. Displaying the input form

[0173] The user terminal displays a form that allows the user to enter search criteria. For example, in the case of tourist spots, users can enter the desired time, location, and specific preferences (such as places rich in nature or historical sites) for visiting. In the case of a logistics center, workers can enter their work details and desired conditions (such as working hours and break times).

[0174] 2. Obtaining location information

[0175] The user terminal obtains the user's current location information. This is done using the terminal's built-in GPS function. The logistics center uses the built-in GPS function or other location tracking technology.

[0176] 3. Data Transmission

[0177] The user terminal transmits the search conditions entered by the user and the acquired location information to the server.

[0178] Server Operation

[0179] 1. Data Reception

[0180] The server receives the search conditions and location information sent from the user terminal.

[0181] 2. Database Matching

[0182] The server compares the received data with a database of tourist spots and work patterns, and selects tourist spots and work patterns that match the user's current location and search criteria.

[0183] 3. Filtering tourist attractions and work patterns

[0184] The server filters the extracted tourist spots and work patterns based on the user's hobbies and preferences and the worker's past behavioral history, thereby ranking the tourist spots and work patterns that are most suitable for the user and worker.

[0185] 4. Short video generation

[0186] The server uses the generative AI model to generate short videos for each selected tourist spot and work pattern. Each video is designed to convey the characteristics and appeal of the tourist spot and the efficiency of the work pattern in a short time of around 30 seconds.

[0187] 5. Data Transmission

[0188] The server transmits the generated short video and ranking information to the user terminal.

[0189] User terminal operation (continued)

[0190] 1. Data Reception

[0191] The user terminal receives the short video and ranking information transmitted from the server.

[0192] 2. Video playback and information display

[0193] The user device plays the received short video and displays ranking information for tourist attractions and work patterns, allowing users and workers to visually understand the appeal of each tourist attraction and the efficiency of their work patterns.

[0194] 3. Selection of tourist spots and work patterns

[0195] Based on the displayed information, users can select tourist spots they want to visit and work patterns they want to perform. This information helps users to plan their sightseeing trips comfortably and efficiently, and also contributes to workers' efficient work.

[0196] Specific examples

[0197] User Behavior

[0198] For example, if a user is looking for "tourist spots that can be enjoyed within an hour around Shinjuku Station," the user enters "1 hour" into the input form on the device and confirms that the current location is Shinjuku Station. At a logistics center, a worker enters "I would like to check the inventory in Area A."

[0199] Device behavior

[0200] The device uses its GPS function to obtain the user's location information and sends the conditions "Shinjuku Station" and "1 hour" to the server. The logistics center obtains the worker's current location and sends "Area A" and "Work content" to the server.

[0201] Server Operation

[0202] After receiving the data, the server extracts tourist spots that can be reached and returned from Shinjuku Station within an hour from the database. It also considers past search history and the user's hobbies (e.g., a love of nature) to select and rank the tourist spots. The logistics center extracts and ranks efficient work patterns.

[0203] User terminal operation (continued)

[0204] The device plays the received short video and displays ranking information to the user, allowing the user to select tourist spots they want to visit. Similarly, workers can visually understand and select efficient work patterns.

[0205] Example prompts for generative AI models

[0206] "For a worker who is currently in area C and wants to perform picking work in area A, please generate a 30-second guidance video that includes an efficient route and important points to note."

[0207] The system allows users to intuitively and efficiently find tourist spots and helps workers identify optimal work patterns.

[0208] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0209] Step 1:

[0210] Displaying the input form

[0211] The user terminal displays a form that allows the user to enter search criteria. The user enters the desired time, location, and specific preferences (such as a place rich in nature or a historical site) as search criteria. At the logistics center, workers enter their work details and desired conditions (such as working hours and break times).

[0212] Input: The time, place, and specific wishes you want to visit are entered by the user.

[0213] Output: User input information (search criteria)

[0214] Step 2:

[0215] Obtaining location information

[0216] The user device obtains the user's current location information using the device's built-in GPS function. In the logistics center, the location information of workers is obtained using the built-in GPS function or other location tracking technology.

[0217] Input: Data from the device's built-in GPS function or location tracking technology

[0218] Output: Current location information obtained

[0219] Step 3:

[0220] Data transmission

[0221] The user terminal transmits the search conditions entered by the user and the acquired location information to the server.

[0222] Input: User input information (search criteria), current location information obtained

[0223] Output: Data sent to the server

[0224] Step 4:

[0225] Data reception

[0226] The server receives the search conditions and location information sent from the user terminal.

[0227] Input: Search criteria and location information sent from the user device

[0228] Output: Received search conditions, location information

[0229] Step 5:

[0230] Database collation

[0231] The server compares the received data with a database of tourist spots and work patterns, and searches for tourist spots and work patterns using the search criteria entered by the user and the current location information.

[0232] Input: Received search conditions, location information

[0233] Output: Extracted tourist spots or work patterns

[0234] Step 6:

[0235] Filtering tourist attractions and work patterns

[0236] The server filters the extracted tourist spots and work patterns based on the user's hobbies and preferences and the worker's past behavioral history. "User hobbies and preferences" are determined based on past search history and behavioral history, and "worker efficiency" is determined based on behavioral history and time measurement data.

[0237] Input: Extracted tourist spots or work patterns, user's hobby and preference information, worker's past behavior history information

[0238] Output: Ranked tourist spots or activity patterns

[0239] Step 7:

[0240] Short video generation

[0241] The server uses a generative AI model to generate short videos for each selected tourist spot and work pattern. Each video is short, about 30 seconds long, and conveys the characteristics and appeal of the tourist spot and the efficiency of the work pattern. A prompt sentence is input into the generative AI model to generate the video.

[0242] Input: Prompt sentences such as "Description and characteristics of tourist spots" or "Work procedures and efficiency of workers"

[0243] Output: Generated short video

[0244] Step 8:

[0245] Data transmission

[0246] The server transmits the generated short video and ranking information to the user terminal.

[0247] Input: Generated short videos, ranking information

[0248] Output: Data sent to the user terminal

[0249] Step 9:

[0250] Data reception

[0251] The user terminal receives the short video and ranking information transmitted from the server.

[0252] Input: Short video sent from the server, ranking information

[0253] Output: Received short videos and ranking information

[0254] Step 10:

[0255] Video playback and information display

[0256] The user device plays the received short video and displays ranking information for tourist spots and work patterns, allowing users and workers to visually understand the appeal of each tourist spot and the efficiency of their work patterns.

[0257] Input: Received short video, ranking information

[0258] Output: Short video played, ranking information displayed

[0259] Step 11:

[0260] Selection of tourist spots and work patterns

[0261] Based on the displayed information, users can select tourist spots they want to visit and work patterns they want to perform. This information helps users to plan their sightseeing trips comfortably and efficiently, and also contributes to workers' efficient work.

[0262] Input: Displayed ranking information, played short video

[0263] Output: Selected tourist spots or work patterns

[0264] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0265] To specifically implement the present invention, it is necessary to build a system that includes a user terminal, a server, an emotion engine, and various communication means. The purpose of this system is to allow users to search for tourist spots and visually understand their attractions, while also taking into account the user's emotional state to make more appropriate recommendations. The program processing of this system is as follows.

[0266] User terminal operation

[0267] 1. Displaying the input form

[0268] The user terminal displays a form for the user to input search criteria, such as the time, location, and specific preferences (e.g., natural beauty, historical sites, etc.) that the user wants to visit.

[0269] 2. Obtaining location information

[0270] The user device acquires current location information using the GPS function and stores the acquired location information in its internal memory.

[0271] 3. Operation of the Emotion Engine

[0272] The user device uses a built-in emotion engine to recognize the user's emotional state from their voice and facial expressions. The emotion engine determines the user's current emotional state through analysis of the user's voice tone and facial expressions.

[0273] 4. Data Transmission

[0274] The user terminal transmits to the server the search conditions entered by the user, the acquired location information, and the emotional state recognized by the emotion engine.

[0275] Server Operation

[0276] 1. Data Reception

[0277] The server receives the search conditions, location information, and emotional state transmitted from the user terminal.

[0278] 2. Database Matching

[0279] The server compares the received search criteria, location information, and emotional state with a database of tourist spots. For example, it extracts relevant tourist spots based on the criteria "within one hour from Shinjuku Station" and "places where you can relax."

[0280] 3. Filtering tourist attractions

[0281] The server filters the extracted tourist spots based on the user's hobbies, preferences, past behavioral history, and emotional state, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[0282] 4. Short video generation

[0283] The server uses a generative AI model to generate short videos for each ranked tourist spot, each of which is about 30 seconds long and introduces the main attractions of the tourist spot.

[0284] 5. Data Transmission

[0285] The server transmits the generated short video and ranking information to the user terminal.

[0286] User terminal operation (continued)

[0287] 1. Data Reception

[0288] The user terminal receives the short video and ranking information transmitted from the server.

[0289] 2. Video playback and information display

[0290] The user device plays the received short video for the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[0291] 3. Selecting tourist spots

[0292] The user can select the tourist spots they want to visit based on the displayed information, and the selected information can be used to help the user plan their trip.

[0293] Specific examples

[0294] User Behavior

[0295] For example, suppose a user is looking for "a relaxing tourist spot that can be enjoyed within an hour around Shinjuku Station." The user enters "1 hour" and "relaxation" into the input form on the device and confirms that the current location is Shinjuku Station.

[0296] Device behavior

[0297] The device uses its GPS function to obtain the user's location information, and then uses an emotion engine to analyze the user's current emotional state from their voice and facial expressions, and sends the results to the server.

[0298] Server Operation

[0299] After receiving the data, the server extracts tourist spots that can be reached and returned from Shinjuku Station within an hour from the database. Furthermore, taking into account the emotional state of "places where you can relax," it selects and ranks Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street in Harajuku. It then generates short videos showcasing the attractions of these tourist spots and sends them to the user's device.

[0300] User terminal operation (continued)

[0301] The device plays the received short video and displays ranking information to the user, allowing the user to select Meiji Jingu.

[0302] In this way, the system embodying the present invention efficiently provides optimal tourist spots, taking into account the user's current emotional state.

[0303] The processing flow will be explained below.

[0304] User terminal processing

[0305] Step 1:

[0306] The user inputs search criteria (for example, the place they want to visit and the time they can be there) into the search form displayed on the terminal.

[0307] Step 2:

[0308] The device uses the built-in GPS function to obtain current location information, which is then stored in the internal memory.

[0309] Step 3:

[0310] The device uses a built-in emotion engine to recognize the user's emotional state from their voice and facial expressions. The emotion engine determines the user's current emotional state (e.g., excitement, relaxation, stress, etc.) through analysis of the user's voice tone and facial expressions.

[0311] Step 4:

[0312] The terminal transmits the search criteria entered by the user, the acquired location information, and the emotional state recognized by the emotion engine to the server.

[0313] Server Processing

[0314] Step 5:

[0315] The server receives the search conditions, location information, and emotional state transmitted from the user terminal.

[0316] Step 6:

[0317] The server compares the received search criteria, location information, and emotional state with a database of tourist spots. For example, it extracts relevant tourist spots based on the criteria "within one hour from Shinjuku Station" and "places where you can relax."

[0318] Step 7:

[0319] The server then filters the extracted tourist spots based on the user's preferences, past activities, and emotional state, prioritizing spots that meet criteria such as "places rich in nature" or "quiet places."

[0320] Step 8:

[0321] The server uses a generative AI model to generate short videos for each of the filtered tourist spots, each of which is about 30 seconds long and introduces the main attractions of the tourist spot.

[0322] Step 9:

[0323] The server transmits the generated short video and ranking information to the user terminal.

[0324] User terminal processing (continued)

[0325] Step 10:

[0326] The terminal receives the short video and ranking information transmitted from the server.

[0327] Step 11:

[0328] The device plays the received short video to the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[0329] Step 12:

[0330] The user selects tourist spots that they would like to visit based on the provided information, and the selected information can be used to help the user plan their trip.

[0331] Specific examples

[0332] For example, suppose a user is searching for "a relaxing tourist spot that can be enjoyed within an hour around Shinjuku Station." The user enters "1 hour" and "relaxing" into the input form on the device and confirms that the current location is Shinjuku Station.

[0333] The device uses its GPS to obtain location information for Shinjuku Station, and then uses an emotion engine to analyze the user's current emotional state based on their voice and facial expressions, obtaining a "relaxed state" as the result. This information is then sent to the server.

[0334] The server receives this data and compares it with a database of tourist spots based on the emotional information, such as "within an hour from Shinjuku Station" and "a relaxing place." As a result, Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street in Harajuku are selected and ranked. A 30-second short video introducing the attractions of the selected tourist spots is generated and sent to the user's device.

[0335] The user terminal plays the received short video and displays ranking information for Meiji Shrine, Yoyogi Park, and Takeshita Street in Harajuku to the user, who then decides to visit Meiji Shrine.

[0336] This series of steps allows users to efficiently find tourist spots that best suit their current emotional state.

[0337] Example 2

[0338] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0339] Conventional tourist spot recommendation systems suggest tourist spots based on the search criteria and location information entered by the user, but they are unable to make recommendations that take into account the user's current emotional state, making it difficult to fully meet the user's needs.In addition, there is a lack of a means to efficiently convey the appeal of the suggested tourist spots, which often leaves users unsure which one to choose.

[0340] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0341] In this invention, the server includes means for receiving search criteria entered from a user terminal, means for acquiring current location information related to the search criteria, means for acquiring and analyzing emotional data such as voice and facial expression, means for extracting related tourist spots from a database based on the search criteria, location information, and emotional data, means for ranking the extracted tourist spots based on the user's hobbies, preferences, and emotional state, means for using a generative AI model to generate short videos for the ranked tourist spots, and means for transmitting the generated short videos and ranking information to the user terminal. This makes it possible to efficiently provide the user with the most suitable tourist spots while taking into account the user's current emotional state, and to visually convey their appeal.

[0342] A "user terminal" is an electronic device through which a user inputs search criteria and receives search results, and includes a smartphone, tablet, or personal computer.

[0343] "Search conditions" are information that a user inputs as conditions for the tourist spot they would like to visit, and include the time and place they would like to visit, specific requests, etc.

[0344] "Location information" is geographical data that indicates where the user is currently located, obtained using the GPS function of the user terminal.

[0345] "Emotion data" is information about the user's emotional state obtained by analyzing the user's voice, facial expressions, and the like.

[0346] A "database" is a data storage that accumulates information about tourist spots, and allows for the collation and extraction of information.

[0347] "Ranking" is the process of prioritizing and organizing the extracted tourist spots based on the user's hobbies, preferences, and emotional state.

[0348] "Generative AI model" is an abbreviation for generative artificial intelligence model, and refers to the AI ​​technology used to generate short videos and the like in this invention.

[0349] "Short videos" are short video contents created to introduce the main attractions of tourist spots in a short amount of time.

[0350] A "prompt" is an instruction given to a generative AI model, which indicates the specific content that the model needs to generate an appropriate video.

[0351] MODE FOR CARRYING OUT THE INVENTION

[0352] To specifically implement the present invention, it is necessary to build a system including a user terminal, a server, an emotion engine, and various communication means. The purpose of this system is to enable users to search for tourist spots and visually understand their attractions, while also providing more appropriate recommendations that take into account the user's emotional state.

[0353] Hardware and software used

[0354] User devices: smartphones, tablets, personal computers, etc.

[0355] Server: A high-performance computer that receives, processes, and transmits data.

[0356] Emotion Engine: Software for voice tone and facial expression analysis

[0357] Generative AI models: Generative artificial intelligence models, such as OpenAI's GPT-4

[0358] Program processing explanation

[0359] User terminal

[0360] 1. Displaying the input form

[0361] The user terminal displays a form for the user to input search criteria, such as the time, location, and specific preferences (e.g., natural beauty, historical sites, etc.) that the user wants to visit.

[0362] 2. Obtaining location information

[0363] The user device acquires current location information using the built-in GPS function and stores the acquired location information in its internal memory.

[0364] 3. Operation of the Emotion Engine

[0365] The user device uses a built-in emotion engine to recognize the user's emotional state from their voice and facial expressions. The emotion engine determines the user's current emotional state through analysis of the user's voice tone and facial expressions.

[0366] 4. Data Transmission

[0367] The user terminal transmits to the server the search conditions entered by the user, the acquired location information, and the emotional state recognized by the emotion engine.

[0368] server

[0369] 1. Data Reception

[0370] The server receives the search conditions, location information, and emotional state transmitted from the user terminal.

[0371] 2. Database Matching

[0372] The server searches a database of tourist attractions based on the received search criteria, location information, and emotional state. For example, it extracts relevant tourist attractions based on the criteria "within an hour from the city center" and "places where you can relax."

[0373] 3. Filtering tourist attractions

[0374] The server filters the extracted tourist spots based on the user's hobbies, preferences, past behavioral history, and emotional state, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[0375] 4. Short video generation

[0376] The server uses a generative AI model to generate a short video for each ranked tourist spot. Each video introduces the main attractions of the tourist spot in a short time of about 30 seconds. The following sentence is used as a prompt for the generative AI model:

[0377] "Write a short 30-second video script about a relaxing tourist spot within an hour of the city center. The spot is a famous park."

[0378] 5. Data Transmission

[0379] The server transmits the generated short video and ranking information to the user terminal.

[0380] User terminal

[0381] 1. Data Reception

[0382] The user terminal receives the short video and ranking information transmitted from the server.

[0383] 2. Video playback and information display

[0384] The user device plays the received short video for the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[0385] 3. Selecting tourist spots

[0386] The user can select the tourist spots they want to visit based on the displayed information, and this selection information can be used to help the user plan their trip.

[0387] Specific examples

[0388] User Behavior

[0389] For example, suppose a user is looking for "a relaxing and enjoyable tourist spot within an hour of the city center." The user enters "1 hour" and "relaxing" into the input form on the device and confirms that the current location is the city center.

[0390] Device behavior

[0391] The device uses its GPS function to obtain the user's location information, and then uses an emotion engine to analyze the user's current emotional state from their voice and facial expressions, and sends the results to the server.

[0392] Server Operation

[0393] After receiving the data, the server extracts tourist spots that can be reached and returned from the city center within an hour from the database. It then selects and ranks several suitable locations, taking into account the emotional state of "a place to relax." It then generates short videos showcasing the attractions of these tourist spots and sends them to the user's device.

[0394] User terminal operation

[0395] The device plays the received short video and displays ranking information to the user, allowing the user to select the tourist spots they would like to visit.

[0396] In this way, the system embodying the present invention efficiently provides optimal tourist spots, taking into account the user's current emotional state.

[0397] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0398] Step 1:

[0399] The user terminal displays a search condition input form.

[0400] Input: User's tourist attraction preferences (time, location, specific preferences, etc.)

[0401] Output: The search criteria entered by the user

[0402] How it works: Using a drawing library or UI component, a search field is displayed on the form screen, which the user can follow to enter specific search criteria.

[0403] Step 2:

[0404] The user terminal acquires location information using the GPS function.

[0405] Input: Current GPS signal

[0406] Output: Current location of the user (latitude and longitude information)

[0407] How it works: Calls the device's location service to obtain the user's real-time location, which is then stored in the device's internal memory.

[0408] Step 3:

[0409] The user terminal uses an emotion engine to recognize the user's emotional state.

[0410] Input: User's voice data, facial expression images

[0411] Output: User's emotional state (e.g., relaxed, excited, tired, etc.)

[0412] How it works: Analyzes the tone of voice input and performs facial expression analysis using image data acquired from the camera, thereby determining the user's emotions and treating their emotional state as data.

[0413] Step 4:

[0414] The user terminal transmits search conditions, location information, and emotional state to the server.

[0415] Input: Search criteria, location, emotional state

[0416] Output: Data packet to send to the server

[0417] What it does: It formats the data to be sent into packets and sends them over the network to the server.

[0418] Step 5:

[0419] The server receives the data sent from the user terminal.

[0420] Input: Search criteria, location information, and emotional state sent from the user device

[0421] Output: Data used for database matching

[0422] Operation: Analyzes the received data, converts it into the required format and stores it in the internal memory.

[0423] Step 6:

[0424] The server checks the database and extracts relevant tourist spots.

[0425] Input: Search criteria, location, emotional state

[0426] Output: A list of matched attractions

[0427] Behavior: Generates a database query and performs a search based on the received data. Returns a list of matching attractions.

[0428] Step 7:

[0429] The server filters and ranks the extracted tourist attractions.

[0430] Input: A list of matched tourist spots, the user's interests and preferences, and their emotional state

[0431] Output: A list of ranked tourist attractions

[0432] How it works: Based on the user's recorded hobbies, preferences, past behavioral history, and current emotional state, the system calculates and ranks the relevance of each tourist spot.

[0433] Step 8:

[0434] The server generates the short video using the generative AI model.

[0435] Input: A list of ranked tourist attractions, a prompt

[0436] Output: Short videos about each tourist attraction

[0437] How it works: The model is fed with preconditions and prompts, and generates a 30-second video introducing the main attractions of each tourist spot.

[0438] Step 9:

[0439] The server transmits the short video and ranking information to the user terminal.

[0440] Input: Generated short videos, ranking information

[0441] Output: Data sent to the user's terminal

[0442] Operation: The generated video and ranking information are packaged into a data packet and sent to the user's terminal via the network.

[0443] Step 10:

[0444] The user terminal receives the short video and ranking information transmitted from the server.

[0445] Input: Short video and ranking information sent from the server

[0446] Output: Data to display

[0447] Operation: Receives data packets, parses them and stores them in internal memory.

[0448] Step 11:

[0449] The user terminal plays the received short video and displays the ranking information.

[0450] Input: Saved short videos, ranking information

[0451] Output: Video and information displayed to the user

[0452] How it works: Displays videos through a player and uses UI components to display ranking information on the screen.

[0453] Step 12:

[0454] The user selects the tourist spot they would like to visit based on the displayed information.

[0455] Input: Viewed video and ranking information

[0456] Output: Selected tourist spots

[0457] How it works: The user clicks or taps on a tourist attraction to select it, and that information is reflected in the user's itinerary.

[0458] (Application example 2)

[0459] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0460] Conventional tourist spot recommendation systems are unable to consider the user's current emotional state, making it difficult to suggest appropriate tourist spots. Furthermore, due to a lack of visually understandable information provision methods, users are unable to easily grasp the appeal of tourist spots. Furthermore, when applying similar technology to virtual stores, it has not been possible to achieve advanced personalization, such as product recommendations based on the user's emotional state.

[0461] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0462] In this invention, the server includes means for receiving search criteria entered from a user terminal, means for acquiring current location information, means for utilizing an emotion engine that recognizes the emotional state of the user from their voice and facial expression, means for extracting relevant tourist spots from a database based on the search criteria and location information, means for ranking the extracted tourist spots based on the user's hobbies, preferences, and emotional state, means for generating short videos of the ranked tourist spots, and means for transmitting the generated short videos and ranking information to the user terminal. This enables advanced personalization that takes into account the user's current emotional state and makes it possible to provide information that is visually easy to understand.

[0463] A "user terminal" is an electronic device that allows a user to input information or view received information, and includes smartphones, tablets, PCs, etc.

[0464] "Location information" is data indicating the user's current location, and is geographical location data obtained by a GPS function or the like.

[0465] An "emotion engine" is software or hardware that recognizes the user's emotional state from their voice and facial expressions.

[0466] A "database" is a system for storing and managing related information, which can be searched and extracted based on specific conditions.

[0467] "Ranking" refers to prioritizing and ordering items based on specific criteria.

[0468] A "short video" is a video that introduces specific content in a short period of time, usually about 30 seconds long.

[0469] A "generative model" is an algorithm or system for generating new content based on specific data or conditions.

[0470] A "server" is a computer system that provides and processes information over a network, and plays a central role in sending, receiving, and processing data.

[0471] To specifically implement the present invention, it is necessary to build the following system, which includes a user terminal, a server, an emotion engine, and various communication means.

[0472] User terminal configuration

[0473] 1. Displaying the input form

[0474] The user terminal uses HTML and JavaScript (registered trademark) to display a form in which the user can enter product categories and search conditions. Through this form, the user can enter the products they want to purchase and their desired conditions.

[0475] 2. Obtaining location information

[0476] The user device obtains the current location information using the JavaScript navigator.geolocation API, which is used for subsequent processing.

[0477] 3. Operation of the Emotion Engine

[0478] The user device uses Google Cloud's Vision API and Speech-to-Text API to recognize the user's emotional state from their voice and facial expressions, allowing it to analyze the user's current emotional state.

[0479] 4. Data Transmission

[0480] The user device sends the search criteria entered by the user, the acquired location information, and the emotional state recognized by the emotion engine to the server via a POST request. The HTTPS protocol is used for communication with the server.

[0481] Server Configuration

[0482] 1. Data Reception

[0483] The server uses Node.js and Express to receive data sent from the user's device, including search criteria, location information, and emotional state.

[0484] 2. Database Matching

[0485] The server then compares the received data with the MongoDB database and extracts relevant information, specifically, extracting appropriate products from the product database, taking into account the user's emotional state and search criteria.

[0486] 3. Ranking

[0487] The server ranks the extracted products based on the user's tastes and preferences and emotional state, allowing it to prioritize the most suitable products for the user.

[0488] 4. Short video generation

[0489] The server uses FFmpeg and a generative AI model (e.g., OpenAI's GPT-3®) to generate a short video for each ranked product. The video content is short, about 30 seconds long, showcasing the main appeal of each product.

[0490] 5. Data Transmission

[0491] The server transmits the generated short videos and ranking information to the user's terminal, allowing the user to visually understand the appeal of each product.

[0492] Specific examples

[0493] For example, when a user searches for "fashion items" using a virtual store app, the system works as follows: The user enters their search criteria and current location information into their device, and the emotion engine analyzes their emotional state. Based on this information, the server extracts and ranks products from the database that best fit the user's emotional state. Finally, a short video is generated using FFmpeg and a generative AI model and sent to the user's device.

[0494] Prompt Sentence Examples

[0495] If the user is relaxed, recommend some relaxed outfits that suit the user's taste from the latest fashion items. Example: Introduce five fashion items each in a short 30-second video.

[0496] In this way, highly accurate product recommendations that take into account the user's emotional state and easy-to-understand information can be provided.

[0497] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0498] Step 1:

[0499] The user inputs product categories and search conditions into an input form on the user's terminal. At this time, the user's input is sent as a form and organized as input data.

[0500] Step 2:

[0501] The user device obtains the current location information using the JavaScript navigator.geolocation API. The location information is obtained as latitude and longitude data and added to the input data.

[0502] Step 3:

[0503] The user device uses Google Cloud's Vision API and Speech-to-Text API to analyze voice and facial expressions to recognize the user's emotional state, and the recognition results are included in the input data.

[0504] Step 4:

[0505] The user device sends the entered search criteria, the acquired location information, and the recognized emotional state as data to the server in JSON format.

[0506] Step 5:

[0507] The server uses Node.js and Express to receive data sent from the user's device, including search criteria, location information, and emotional state.

[0508] Step 6:

[0509] The server then matches the received data with a MongoDB database to extract relevant product data, taking into account the emotional state of the user and providing filtered product data.

[0510] Step 7:

[0511] The server inputs the extracted product data into a generative AI model (OpenAI's GPT-3) to generate prompts that introduce the appeal of each product. These prompts then become the input data for generating short videos.

[0512] Step 8:

[0513] The server uses FFmpeg to generate a short video for each product based on the prompt text obtained from the generative AI model. The generated short video is about 30 seconds long and introduces the main appeal of the product.

[0514] Step 9:

[0515] The server transmits the generated short video and ranking information to the user terminal, and the transmitted data is formatted to be displayed on the user terminal.

[0516] Step 10:

[0517] The user terminal receives the short videos and ranking information sent from the server, and the received data is visually displayed to the user.

[0518] Step 11:

[0519] The user checks the displayed short video and ranking information and selects the product they wish to purchase. The selected product information is used for the next process.

[0520] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0521] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0522] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0523] [Second embodiment]

[0524] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0525] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0526] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0527] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0528] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0529] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0530] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0531] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0532] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0533] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0534] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0535] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0536] To specifically implement the present invention, it is necessary to build a system that includes a user terminal, a server, and various communication means. The purpose of this system is to make it easier for users to search for tourist spots and visually understand their attractions. The program processing of this system is as follows.

[0537] User terminal operation

[0538] 1. Displaying the input form

[0539] The user terminal displays a form for the user to input search criteria, such as the time, location, and specific preferences (e.g., natural beauty, historical sites, etc.) that the user wants to visit.

[0540] 2. Obtaining location information

[0541] The user device obtains the user's current location information using the device's built-in GPS function.

[0542] 3. Data Transmission

[0543] The user terminal transmits the search conditions entered by the user and the acquired location information to the server.

[0544] Server Operation

[0545] 1. Data Reception

[0546] The server receives the search conditions and location information sent from the user terminal.

[0547] 2. Database Matching

[0548] The server compares the received data with a database of tourist spots, and selects tourist spots that match the user's current location and search criteria.

[0549] 3. Filtering tourist attractions

[0550] The server filters the extracted tourist spots based on the user's hobbies, preferences, and past behavioral history, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[0551] 4. Short video generation

[0552] The server uses a generative AI model to generate short videos for each selected tourist spot. Each video is designed to convey the characteristics and appeal of the tourist spot in a short time of around 30 seconds.

[0553] 5. Data Transmission

[0554] The server transmits the generated short video and ranking information to the user terminal.

[0555] User terminal operation (continued)

[0556] 1. Data Reception

[0557] The user terminal receives the short video and ranking information transmitted from the server.

[0558] 2. Video playback and information display

[0559] The user device plays the received short video and displays the tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[0560] 3. Selecting tourist spots

[0561] Users can select the tourist spots they want to visit based on the displayed information, which helps users to plan their sightseeing trips comfortably and efficiently.

[0562] Specific examples

[0563] User Behavior

[0564] For example, suppose a user is searching for "tourist spots that can be enjoyed within an hour around Shinjuku Station." The user enters "1 hour" into the input form on the device and confirms that the current location is Shinjuku Station.

[0565] Device behavior

[0566] The device uses its GPS function to obtain the user's location information and sends the conditions "Shinjuku Station" and "1 hour" to the server.

[0567] Server Operation

[0568] After receiving the data, the server extracts tourist spots that can be reached and returned from Shinjuku Station within an hour from the database. Then, taking into consideration the user's past search history and hobbies (e.g., a love of nature), it selects and ranks Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street in Harajuku. It then generates short videos showcasing the attractions of these tourist spots and sends them to the user's device.

[0569] User terminal operation (continued)

[0570] The device plays the received short video and displays ranking information to the user, allowing the user to select Meiji Jingu.

[0571] In this way, a system embodying the present invention allows a user to intuitively and efficiently find the best tourist spots.

[0572] The processing flow will be explained below.

[0573] User terminal processing

[0574] Step 1:

[0575] The user inputs search conditions (for example, desired time and location) into a search form displayed on the terminal.

[0576] Step 2:

[0577] The device uses the GPS function to obtain current location information, which is then stored in the internal memory.

[0578] Step 3:

[0579] The terminal transmits the search conditions entered by the user and the acquired location information to the server.

[0580] Server Processing

[0581] Step 4:

[0582] The server receives the search conditions and location information sent from the user terminal.

[0583] Step 5:

[0584] The server checks the tourist spot database based on the received search criteria and location information. For example, it extracts relevant tourist spots based on the criteria "within one hour from Shinjuku Station."

[0585] Step 6:

[0586] The server filters the extracted tourist spots based on the user's hobbies, preferences, and past behavioral history, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[0587] Step 7:

[0588] The server uses a generative AI model to generate short videos for each ranked tourist spot, each of which is about 30 seconds long and introduces the main attractions of the tourist spot.

[0589] Step 8:

[0590] The server transmits the generated short video and ranking information to the user terminal.

[0591] User terminal processing (continued)

[0592] Step 9:

[0593] The terminal receives the short video and ranking information transmitted from the server.

[0594] Step 10:

[0595] The device plays the received short video to the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[0596] Step 11:

[0597] The user selects tourist spots they would like to visit based on the provided information, and the selected information can be used to help the user plan their trip.

[0598] In this way, tourist spots that meet the user's current needs are efficiently provided.

[0599] Example 1

[0600] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0601] In the past, there were few ways for users to efficiently search for tourist spots and intuitively understand their attractions. As a result, it took a lot of time and effort for users to find the appropriate tourist spot based on their hobbies, preferences, and current location information. Furthermore, since the information provided was text-based, it was difficult to understand visually.

[0602] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0603] In this invention, the server includes means for receiving search criteria entered from a user terminal, means for acquiring current location information related to the search criteria, means for extracting relevant tourist spots from a database based on the search criteria and location information, means for ranking the extracted tourist spots based on the user's hobbies and preferences, means for generating short videos for the ranked tourist spots using a generative AI model, and means for transmitting the generated short videos and ranking information to the user terminal, thereby enabling users to intuitively and efficiently find the best tourist spots.

[0604] "Search conditions" are information including the place and time the user wants to visit, and specific desired conditions (e.g., a place rich in nature, a historical place, etc.).

[0605] "Location information" is data that indicates the user's current geographical location, and is mainly obtained using the GPS function.

[0606] "Tourist attraction" refers to a place or facility that a user wishes to visit, the place or facility being publicly known for tourism or recreational purposes.

[0607] A "database" is a system for efficiently storing, accessing, and managing large amounts of information, including information about tourist attractions.

[0608] "Hobbies and preferences" refers to a user's personal tastes and interests, and are determined based on past behavioral history and profile data.

[0609] A "generative AI model" is an artificial intelligence model that uses machine learning techniques to generate new data (e.g., short videos). Specifically, it includes techniques for generating videos from text and images.

[0610] A "short video" is a video that conveys the appeal and characteristics of a tourist spot within a short period of time (usually 30 seconds or less).

[0611] "Ranking information" is a list that ranks the attractions of tourist spots based on the user's hobbies, preferences, and search conditions.

[0612] "User terminal" refers to an electronic device that is directly operated by a user, including smartphones, tablets, and personal computers.

[0613] A "server" is a central computer system that receives and processes data sent from user terminals, generates the necessary information, and retransmits it.

[0614] The present invention is implemented by constructing a system including a user terminal, a server, and various communication means. The purpose of this system is to enable users to easily search for tourist spots and visually understand their attractions.

[0615] User terminal operation

[0616] The user device first displays a form for entering search criteria for tourist attractions. This allows the user to enter the time, location, and specific preferences for the desired visit (e.g., natural beauty, historical sites, etc.). The user device also uses its built-in GPS to obtain the user's current location. This information is sent to the server and used to improve the accuracy of search results.

[0617] Server Operation

[0618] The server receives the search criteria and location information sent from the user's device. Based on the received data, the server checks a database of tourist spots and extracts relevant tourist spots. The extracted list of tourist spots is then filtered based on the user's hobbies, preferences, and past behavioral history, and organized in a ranking format. Based on this ranking information, the server uses a generative AI model (e.g., OpenAI's GPT-4) to generate short videos. Each video is approximately 30 seconds long and conveys the characteristics and appeal of the tourist spot. Finally, the server sends the generated short videos and ranking information to the user's device.

[0619] User terminal operation (continued)

[0620] The user device receives the short videos and ranking information sent from the server. This information is used to visually convey the attractions of tourist spots to the user. By playing the received short videos and displaying the ranking information, the user can intuitively and efficiently select tourist spots. This information is useful when the user selects tourist spots they want to visit.

[0621] Example operation

[0622] For example, consider the case where a user is searching for "fun tourist spots around Shinjuku Station within an hour." The user enters "1 hour" into the input form on their device and checks their current location. The device then uses its GPS function to obtain location information and sends the conditions "Shinjuku Station" and "1 hour" to the server. After receiving these conditions, the server extracts matching tourist spots from its database. Taking into consideration past search history and the user's hobbies, it selects Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street, creates short videos showcasing the appeal of each tourist spot, and sends them to the user's device. The device then plays these videos and displays ranking information. The user can then view these videos and select the spot they want to visit (for example, Meiji Jingu Shrine).

[0623] Example prompts to input to the generative AI model

[0624] "Please introduce tourist spots that can be enjoyed within an hour around Shinjuku Station. Please create a short video of 30 seconds or less that introduces the appeal of each spot. Please also take into consideration your past search history and hobbies (nature lovers)."

[0625] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0626] Step 1:

[0627] The user device displays a form for entering search criteria for tourist spots. Here, the user enters the desired time, location, and specific desired conditions for visiting. At this point, the input is the time (e.g., 1 hour), location (e.g., around Shinjuku Station), and desired conditions (e.g., a place rich in nature).

[0628] Step 2:

[0629] The user device obtains the current location information using the built-in GPS function. This is the operation of obtaining the current location data provided by the user device. The obtained location information (e.g., latitude and longitude data) is used for subsequent processing.

[0630] Step 3:

[0631] The user terminal transmits the input search criteria and the acquired location information to the server. The transmitted data includes the time, location, desired criteria, and current location information.

[0632] Step 4:

[0633] The server receives the search criteria and location information sent from the user terminal. The received data is temporarily stored in the server's memory and used in the next database matching process.

[0634] Step 5:

[0635] The server checks the tourist attraction database based on the received search criteria and location information. This process uses an SQL query to extract matching tourist attraction records from the database. The output data is a list of tourist attractions that match the search criteria.

[0636] Step 6:

[0637] The server filters the extracted tourist spot list based on the user's hobbies, preferences, and past behavioral history. The user profile data is used as input data, and a ranking list of tourist spots optimized for the user is generated as output.

[0638] Step 7:

[0639] The server generates short videos for the selected tourist spots using a generative AI model (e.g., OpenAI's GPT-4). In this process, tourist spot information is converted into prompt sentences as input data, and the generative AI model generates the video. The output data is a short video of about 30 seconds that conveys the characteristics of the tourist spot.

[0640] Step 8:

[0641] The server transmits the generated short video and ranking information to the user terminal. The transmitted data includes the short video file and the ranking information.

[0642] Step 9:

[0643] The user terminal receives the short video and ranking information sent from the server, and the received data is stored in the memory of the user terminal.

[0644] Step 10:

[0645] The user device plays the received short video and displays the ranking information. This process uses a media player for video playback and a UI for displaying the rankings. By viewing this, the user can intuitively understand the appeal of the tourist spot.

[0646] Step 11:

[0647] The user selects the tourist spots they wish to visit based on the displayed information, and this selection information is saved in the user's terminal and used for subsequent activities.

[0648] (Application example 1)

[0649] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0650] Modern logistics centers and industrial facilities are required to maximize worker efficiency and improve work speed and accuracy. However, there is a lack of immediate and intuitive guidance systems to achieve this. Furthermore, few systems can present optimal work patterns using individual worker location information and past behavioral history. Tourism information systems also lack visual representations that allow users to intuitively select the best tourist spots.

[0651] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0652] In this invention, the server includes means for receiving search criteria and worker location information input from a user terminal, means for extracting related tourist spots and work patterns from a database based on the search criteria and location information, means for ranking the extracted tourist spots and work patterns based on the user's hobbies and preferences and the worker's efficiency, means for generating short videos of the ranked tourist spots and work patterns, and means for transmitting the generated short videos and ranking information to the user terminal. This allows users to intuitively and efficiently select tourist spots and enables workers to efficiently identify optimal work patterns.

[0653] A "user terminal" is an electronic device used by a user, such as a computer, smartphone, or tablet.

[0654] "Search conditions" are desired conditions and requirements for tourist spots and work patterns entered by the user.

[0655] "Location information" is data indicating the current location of a user or worker, and is obtained using technologies such as GPS.

[0656] A "database" is a collection of information that stores information on tourist spots and work patterns and can be searched and extracted.

[0657] "Extraction means" is a function that extracts necessary information from a database based on specific conditions.

[0658] The "ranking means" is a function that ranks the extracted tourist spots and work patterns based on the user's preferences and the efficiency of the workers.

[0659] "Means for generating" refers to the function of creating visual content such as short videos based on related information.

[0660] The "transmitting means" is a communication function for sending the generated data or information to the user terminal.

[0661] A "system" is a collection of components that includes user terminals, servers, communication means, etc., and that as a whole realizes a specific function.

[0662] "Available time" is the time frame in which a user or worker can perform a particular activity.

[0663] "Weather conditions" is information that indicates the weather conditions in a particular area.

[0664] "Past search history" is a record of searches previously performed by a user.

[0665] "Behavioral history" is a record of the actions and work a worker has performed in the past.

[0666] "Mood state" is information that indicates the user's current state of mind or emotions.

[0667] A "generative model" is a program that uses specific algorithms and machine learning techniques to automatically generate materials such as short videos.

[0668] "Tourist spots" are tourist destinations and famous places that users can visit.

[0669] A "work pattern" is a combination of procedures and movements that allow workers to perform work efficiently.

[0670] To specifically implement this invention, it is necessary to build a system including a user terminal, a server, and various communication means. The purpose of this system is to make it easier for users to search for tourist spots and workers' work patterns and to visually understand their attractiveness and efficiency.

[0671] User terminal operation

[0672] 1. Displaying the input form

[0673] The user terminal displays a form that allows the user to enter search criteria. For example, in the case of tourist spots, users can enter the desired time, location, and specific preferences (such as places rich in nature or historical sites) for visiting. In the case of a logistics center, workers can enter their work details and desired conditions (such as working hours and break times).

[0674] 2. Obtaining location information

[0675] The user terminal obtains the user's current location information. This is done using the terminal's built-in GPS function. The logistics center uses the built-in GPS function or other location tracking technology.

[0676] 3. Data Transmission

[0677] The user terminal transmits the search conditions entered by the user and the acquired location information to the server.

[0678] Server Operation

[0679] 1. Data Reception

[0680] The server receives the search conditions and location information sent from the user terminal.

[0681] 2. Database Matching

[0682] The server compares the received data with a database of tourist spots and work patterns, and selects tourist spots and work patterns that match the user's current location and search criteria.

[0683] 3. Filtering tourist attractions and work patterns

[0684] The server filters the extracted tourist spots and work patterns based on the user's hobbies and preferences and the worker's past behavioral history, thereby ranking the tourist spots and work patterns that are most suitable for the user and worker.

[0685] 4. Short video generation

[0686] The server uses the generative AI model to generate short videos for each selected tourist spot and work pattern. Each video is designed to convey the characteristics and appeal of the tourist spot and the efficiency of the work pattern in a short time of around 30 seconds.

[0687] 5. Data Transmission

[0688] The server transmits the generated short video and ranking information to the user terminal.

[0689] User terminal operation (continued)

[0690] 1. Data Reception

[0691] The user terminal receives the short video and ranking information transmitted from the server.

[0692] 2. Video playback and information display

[0693] The user device plays the received short video and displays ranking information for tourist attractions and work patterns, allowing users and workers to visually understand the appeal of each tourist attraction and the efficiency of their work patterns.

[0694] 3. Selection of tourist spots and work patterns

[0695] Based on the displayed information, users can select tourist spots they want to visit and work patterns they want to perform. This information helps users to plan their sightseeing trips comfortably and efficiently, and also contributes to workers' efficient work.

[0696] Specific examples

[0697] User Behavior

[0698] For example, if a user is looking for "tourist spots that can be enjoyed within an hour around Shinjuku Station," the user enters "1 hour" into the input form on the device and confirms that the current location is Shinjuku Station. At a logistics center, a worker enters "I would like to check the inventory in Area A."

[0699] Device behavior

[0700] The device uses its GPS function to obtain the user's location information and sends the conditions "Shinjuku Station" and "1 hour" to the server. The logistics center obtains the worker's current location and sends "Area A" and "Work content" to the server.

[0701] Server Operation

[0702] After receiving the data, the server extracts tourist spots that can be reached and returned from Shinjuku Station within an hour from the database. It also considers past search history and the user's hobbies (e.g., a love of nature) to select and rank the tourist spots. The logistics center extracts and ranks efficient work patterns.

[0703] User terminal operation (continued)

[0704] The device plays the received short video and displays ranking information to the user, allowing the user to select tourist spots they want to visit. Similarly, workers can visually understand and select efficient work patterns.

[0705] Example prompts for generative AI models

[0706] "For a worker who is currently in area C and wants to perform picking work in area A, please generate a 30-second guidance video that includes an efficient route and important points to note."

[0707] The system allows users to intuitively and efficiently find tourist spots and helps workers identify optimal work patterns.

[0708] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0709] Step 1:

[0710] Displaying the input form

[0711] The user terminal displays a form that allows the user to enter search criteria. The user enters the desired time, location, and specific preferences (such as a place rich in nature or a historical site) as search criteria. At the logistics center, workers enter their work details and desired conditions (such as working hours and break times).

[0712] Input: The time, place, and specific wishes you want to visit are entered by the user.

[0713] Output: User input information (search criteria)

[0714] Step 2:

[0715] Obtaining location information

[0716] The user device obtains the user's current location information using the device's built-in GPS function. In the logistics center, the location information of workers is obtained using the built-in GPS function or other location tracking technology.

[0717] Input: Data from the device's built-in GPS function or location tracking technology

[0718] Output: Current location information obtained

[0719] Step 3:

[0720] Data transmission

[0721] The user terminal transmits the search conditions entered by the user and the acquired location information to the server.

[0722] Input: User input information (search criteria), current location information obtained

[0723] Output: Data sent to the server

[0724] Step 4:

[0725] Data reception

[0726] The server receives the search conditions and location information sent from the user terminal.

[0727] Input: Search criteria and location information sent from the user device

[0728] Output: Received search conditions, location information

[0729] Step 5:

[0730] Database collation

[0731] The server compares the received data with a database of tourist spots and work patterns, and searches for tourist spots and work patterns using the search criteria entered by the user and the current location information.

[0732] Input: Received search conditions, location information

[0733] Output: Extracted tourist spots or work patterns

[0734] Step 6:

[0735] Filtering tourist attractions and work patterns

[0736] The server filters the extracted tourist spots and work patterns based on the user's hobbies and preferences and the worker's past behavioral history. "User hobbies and preferences" are determined based on past search history and behavioral history, and "worker efficiency" is determined based on behavioral history and time measurement data.

[0737] Input: Extracted tourist spots or work patterns, user's hobby and preference information, worker's past behavior history information

[0738] Output: Ranked tourist spots or activity patterns

[0739] Step 7:

[0740] Short video generation

[0741] The server uses a generative AI model to generate short videos for each selected tourist spot and work pattern. Each video is short, about 30 seconds long, and conveys the characteristics and appeal of the tourist spot and the efficiency of the work pattern. A prompt sentence is input into the generative AI model to generate the video.

[0742] Input: Prompt sentences such as "Description and characteristics of tourist spots" or "Work procedures and efficiency of workers"

[0743] Output: Generated short video

[0744] Step 8:

[0745] Data transmission

[0746] The server transmits the generated short video and ranking information to the user terminal.

[0747] Input: Generated short videos, ranking information

[0748] Output: Data sent to the user terminal

[0749] Step 9:

[0750] Data reception

[0751] The user terminal receives the short video and ranking information transmitted from the server.

[0752] Input: Short video sent from the server, ranking information

[0753] Output: Received short videos and ranking information

[0754] Step 10:

[0755] Video playback and information display

[0756] The user device plays the received short video and displays ranking information for tourist spots and work patterns, allowing users and workers to visually understand the appeal of each tourist spot and the efficiency of their work patterns.

[0757] Input: Received short video, ranking information

[0758] Output: Short video played, ranking information displayed

[0759] Step 11:

[0760] Selection of tourist spots and work patterns

[0761] Based on the displayed information, users can select tourist spots they want to visit and work patterns they want to perform. This information helps users to plan their sightseeing trips comfortably and efficiently, and also contributes to workers' efficient work.

[0762] Input: Displayed ranking information, played short video

[0763] Output: Selected tourist spots or work patterns

[0764] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0765] To specifically implement the present invention, it is necessary to build a system that includes a user terminal, a server, an emotion engine, and various communication means. The purpose of this system is to allow users to search for tourist spots and visually understand their attractions, while also taking into account the user's emotional state to make more appropriate recommendations. The program processing of this system is as follows.

[0766] User terminal operation

[0767] 1. Displaying the input form

[0768] The user terminal displays a form for the user to input search criteria, such as the time, location, and specific preferences (e.g., natural beauty, historical sites, etc.) that the user wants to visit.

[0769] 2. Obtaining location information

[0770] The user device acquires current location information using the GPS function and stores the acquired location information in its internal memory.

[0771] 3. Operation of the Emotion Engine

[0772] The user device uses a built-in emotion engine to recognize the user's emotional state from their voice and facial expressions. The emotion engine determines the user's current emotional state through analysis of the user's voice tone and facial expressions.

[0773] 4. Data Transmission

[0774] The user terminal transmits to the server the search conditions entered by the user, the acquired location information, and the emotional state recognized by the emotion engine.

[0775] Server Operation

[0776] 1. Data Reception

[0777] The server receives the search conditions, location information, and emotional state transmitted from the user terminal.

[0778] 2. Database Matching

[0779] The server compares the received search criteria, location information, and emotional state with a database of tourist spots. For example, it extracts relevant tourist spots based on the criteria "within one hour from Shinjuku Station" and "places where you can relax."

[0780] 3. Filtering tourist attractions

[0781] The server filters the extracted tourist spots based on the user's hobbies, preferences, past behavioral history, and emotional state, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[0782] 4. Short video generation

[0783] The server uses a generative AI model to generate short videos for each ranked tourist spot, each of which is about 30 seconds long and introduces the main attractions of the tourist spot.

[0784] 5. Data Transmission

[0785] The server transmits the generated short video and ranking information to the user terminal.

[0786] User terminal operation (continued)

[0787] 1. Data Reception

[0788] The user terminal receives the short video and ranking information transmitted from the server.

[0789] 2. Video playback and information display

[0790] The user device plays the received short video for the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[0791] 3. Selecting tourist spots

[0792] The user can select the tourist spots they want to visit based on the displayed information, and the selected information can be used to help the user plan their trip.

[0793] Specific examples

[0794] User Behavior

[0795] For example, suppose a user is looking for "a relaxing tourist spot that can be enjoyed within an hour around Shinjuku Station." The user enters "1 hour" and "relaxation" into the input form on the device and confirms that the current location is Shinjuku Station.

[0796] Device behavior

[0797] The device uses its GPS function to obtain the user's location information, and then uses an emotion engine to analyze the user's current emotional state from their voice and facial expressions, and sends the results to the server.

[0798] Server Operation

[0799] After receiving the data, the server extracts tourist spots that can be reached and returned from Shinjuku Station within an hour from the database. Furthermore, taking into account the emotional state of "places where you can relax," it selects and ranks Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street in Harajuku. It then generates short videos showcasing the attractions of these tourist spots and sends them to the user's device.

[0800] User terminal operation (continued)

[0801] The device plays the received short video and displays ranking information to the user, allowing the user to select Meiji Jingu.

[0802] In this way, the system embodying the present invention efficiently provides optimal tourist spots, taking into account the user's current emotional state.

[0803] The processing flow will be explained below.

[0804] User terminal processing

[0805] Step 1:

[0806] The user inputs search criteria (for example, the place they want to visit and the time they can be there) into the search form displayed on the terminal.

[0807] Step 2:

[0808] The device uses the built-in GPS function to obtain current location information, which is then stored in the internal memory.

[0809] Step 3:

[0810] The device uses a built-in emotion engine to recognize the user's emotional state from their voice and facial expressions. The emotion engine determines the user's current emotional state (e.g., excitement, relaxation, stress, etc.) through analysis of the user's voice tone and facial expressions.

[0811] Step 4:

[0812] The terminal transmits the search criteria entered by the user, the acquired location information, and the emotional state recognized by the emotion engine to the server.

[0813] Server Processing

[0814] Step 5:

[0815] The server receives the search conditions, location information, and emotional state transmitted from the user terminal.

[0816] Step 6:

[0817] The server compares the received search criteria, location information, and emotional state with a database of tourist spots. For example, it extracts relevant tourist spots based on the criteria "within one hour from Shinjuku Station" and "places where you can relax."

[0818] Step 7:

[0819] The server then filters the extracted tourist spots based on the user's preferences, past activities, and emotional state, prioritizing spots that meet criteria such as "places rich in nature" or "quiet places."

[0820] Step 8:

[0821] The server uses a generative AI model to generate short videos for each of the filtered tourist spots, each of which is about 30 seconds long and introduces the main attractions of the tourist spot.

[0822] Step 9:

[0823] The server transmits the generated short video and ranking information to the user terminal.

[0824] User terminal processing (continued)

[0825] Step 10:

[0826] The terminal receives the short video and ranking information transmitted from the server.

[0827] Step 11:

[0828] The device plays the received short video to the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[0829] Step 12:

[0830] The user selects tourist spots that they would like to visit based on the provided information, and the selected information can be used to help the user plan their trip.

[0831] Specific examples

[0832] For example, suppose a user is searching for "a relaxing tourist spot that can be enjoyed within an hour around Shinjuku Station." The user enters "1 hour" and "relaxing" into the input form on the device and confirms that the current location is Shinjuku Station.

[0833] The device uses its GPS to obtain location information for Shinjuku Station, and then uses an emotion engine to analyze the user's current emotional state based on their voice and facial expressions, obtaining a "relaxed state" as the result. This information is then sent to the server.

[0834] The server receives this data and compares it with a database of tourist spots based on the emotional information, such as "within an hour from Shinjuku Station" and "a relaxing place." As a result, Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street in Harajuku are selected and ranked. A 30-second short video introducing the attractions of the selected tourist spots is generated and sent to the user's device.

[0835] The user terminal plays the received short video and displays ranking information for Meiji Shrine, Yoyogi Park, and Takeshita Street in Harajuku to the user, who then decides to visit Meiji Shrine.

[0836] This series of steps allows users to efficiently find tourist spots that best suit their current emotional state.

[0837] Example 2

[0838] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0839] Conventional tourist spot recommendation systems suggest tourist spots based on the search criteria and location information entered by the user, but they are unable to make recommendations that take into account the user's current emotional state, making it difficult to fully meet the user's needs.In addition, there is a lack of a means to efficiently convey the appeal of the suggested tourist spots, which often leaves users unsure which one to choose.

[0840] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0841] In this invention, the server includes means for receiving search criteria entered from a user terminal, means for acquiring current location information related to the search criteria, means for acquiring and analyzing emotional data such as voice and facial expression, means for extracting related tourist spots from a database based on the search criteria, location information, and emotional data, means for ranking the extracted tourist spots based on the user's hobbies, preferences, and emotional state, means for using a generative AI model to generate short videos for the ranked tourist spots, and means for transmitting the generated short videos and ranking information to the user terminal. This makes it possible to efficiently provide the user with the most suitable tourist spots while taking into account the user's current emotional state, and to visually convey their appeal.

[0842] A "user terminal" is an electronic device through which a user inputs search criteria and receives search results, and includes a smartphone, tablet, or personal computer.

[0843] "Search conditions" are information that a user inputs as conditions for the tourist spot they would like to visit, and include the time and place they would like to visit, specific requests, etc.

[0844] "Location information" is geographical data that indicates where the user is currently located, obtained using the GPS function of the user terminal.

[0845] "Emotion data" is information about the user's emotional state obtained by analyzing the user's voice, facial expressions, and the like.

[0846] A "database" is a data storage that accumulates information about tourist spots, and allows for the collation and extraction of information.

[0847] "Ranking" is the process of prioritizing and organizing the extracted tourist spots based on the user's hobbies, preferences, and emotional state.

[0848] "Generative AI model" is an abbreviation for generative artificial intelligence model, and refers to the AI ​​technology used to generate short videos and the like in this invention.

[0849] "Short videos" are short video contents created to introduce the main attractions of tourist spots in a short amount of time.

[0850] A "prompt" is an instruction given to a generative AI model, which indicates the specific content that the model needs to generate an appropriate video.

[0851] MODE FOR CARRYING OUT THE INVENTION

[0852] To specifically implement the present invention, it is necessary to build a system including a user terminal, a server, an emotion engine, and various communication means. The purpose of this system is to enable users to search for tourist spots and visually understand their attractions, while also providing more appropriate recommendations that take into account the user's emotional state.

[0853] Hardware and software used

[0854] User devices: smartphones, tablets, personal computers, etc.

[0855] Server: A high-performance computer that receives, processes, and transmits data.

[0856] Emotion Engine: Software for voice tone and facial expression analysis

[0857] Generative AI models: Generative artificial intelligence models, such as OpenAI's GPT-4

[0858] Program processing explanation

[0859] User terminal

[0860] 1. Displaying the input form

[0861] The user terminal displays a form for the user to input search criteria, such as the time, location, and specific preferences (e.g., natural beauty, historical sites, etc.) that the user wants to visit.

[0862] 2. Obtaining location information

[0863] The user device acquires current location information using the built-in GPS function and stores the acquired location information in its internal memory.

[0864] 3. Operation of the Emotion Engine

[0865] The user device uses a built-in emotion engine to recognize the user's emotional state from their voice and facial expressions. The emotion engine determines the user's current emotional state through analysis of the user's voice tone and facial expressions.

[0866] 4. Data Transmission

[0867] The user terminal transmits to the server the search conditions entered by the user, the acquired location information, and the emotional state recognized by the emotion engine.

[0868] server

[0869] 1. Data Reception

[0870] The server receives the search conditions, location information, and emotional state transmitted from the user terminal.

[0871] 2. Database Matching

[0872] The server searches a database of tourist attractions based on the received search criteria, location information, and emotional state. For example, it extracts relevant tourist attractions based on the criteria "within an hour from the city center" and "places where you can relax."

[0873] 3. Filtering tourist attractions

[0874] The server filters the extracted tourist spots based on the user's hobbies, preferences, past behavioral history, and emotional state, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[0875] 4. Short video generation

[0876] The server uses a generative AI model to generate a short video for each ranked tourist spot. Each video introduces the main attractions of the tourist spot in a short time of about 30 seconds. The following sentence is used as a prompt for the generative AI model:

[0877] "Write a short 30-second video script about a relaxing tourist spot within an hour of the city center. The spot is a famous park."

[0878] 5. Data Transmission

[0879] The server transmits the generated short video and ranking information to the user terminal.

[0880] User terminal

[0881] 1. Data Reception

[0882] The user terminal receives the short video and ranking information transmitted from the server.

[0883] 2. Video playback and information display

[0884] The user device plays the received short video for the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[0885] 3. Selecting tourist spots

[0886] The user can select the tourist spots they want to visit based on the displayed information, and this selection information can be used to help the user plan their trip.

[0887] Specific examples

[0888] User Behavior

[0889] For example, suppose a user is looking for "a relaxing and enjoyable tourist spot within an hour of the city center." The user enters "1 hour" and "relaxing" into the input form on the device and confirms that the current location is the city center.

[0890] Device behavior

[0891] The device uses its GPS function to obtain the user's location information, and then uses an emotion engine to analyze the user's current emotional state from their voice and facial expressions, and sends the results to the server.

[0892] Server Operation

[0893] After receiving the data, the server extracts tourist spots that can be reached and returned from the city center within an hour from the database. It then selects and ranks several suitable locations, taking into account the emotional state of "a place to relax." It then generates short videos showcasing the attractions of these tourist spots and sends them to the user's device.

[0894] User terminal operation

[0895] The device plays the received short video and displays ranking information to the user, allowing the user to select the tourist spots they would like to visit.

[0896] In this way, the system embodying the present invention efficiently provides optimal tourist spots, taking into account the user's current emotional state.

[0897] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0898] Step 1:

[0899] The user terminal displays a search condition input form.

[0900] Input: User's tourist attraction preferences (time, location, specific preferences, etc.)

[0901] Output: The search criteria entered by the user

[0902] How it works: Using a drawing library or UI component, a search field is displayed on the form screen, which the user can follow to enter specific search criteria.

[0903] Step 2:

[0904] The user terminal acquires location information using the GPS function.

[0905] Input: Current GPS signal

[0906] Output: Current location of the user (latitude and longitude information)

[0907] How it works: Calls the device's location service to obtain the user's real-time location, which is then stored in the device's internal memory.

[0908] Step 3:

[0909] The user terminal uses an emotion engine to recognize the user's emotional state.

[0910] Input: User's voice data, facial expression images

[0911] Output: User's emotional state (e.g., relaxed, excited, tired, etc.)

[0912] How it works: Analyzes the tone of voice input and performs facial expression analysis using image data acquired from the camera, thereby determining the user's emotions and treating their emotional state as data.

[0913] Step 4:

[0914] The user terminal transmits search conditions, location information, and emotional state to the server.

[0915] Input: Search criteria, location, emotional state

[0916] Output: Data packet to send to the server

[0917] What it does: It formats the data to be sent into packets and sends them over the network to the server.

[0918] Step 5:

[0919] The server receives the data sent from the user terminal.

[0920] Input: Search criteria, location information, and emotional state sent from the user device

[0921] Output: Data used for database matching

[0922] Operation: Analyzes the received data, converts it into the required format and stores it in the internal memory.

[0923] Step 6:

[0924] The server checks the database and extracts relevant tourist spots.

[0925] Input: Search criteria, location, emotional state

[0926] Output: A list of matched attractions

[0927] Behavior: Generates a database query and performs a search based on the received data. Returns a list of matching attractions.

[0928] Step 7:

[0929] The server filters and ranks the extracted tourist attractions.

[0930] Input: A list of matched tourist spots, the user's interests and preferences, and their emotional state

[0931] Output: A list of ranked tourist attractions

[0932] How it works: Based on the user's recorded hobbies, preferences, past behavioral history, and current emotional state, the system calculates and ranks the relevance of each tourist spot.

[0933] Step 8:

[0934] The server generates the short video using the generative AI model.

[0935] Input: A list of ranked tourist attractions, a prompt

[0936] Output: Short videos about each tourist attraction

[0937] How it works: The model is fed with preconditions and prompts, and generates a 30-second video introducing the main attractions of each tourist spot.

[0938] Step 9:

[0939] The server transmits the short video and ranking information to the user terminal.

[0940] Input: Generated short videos, ranking information

[0941] Output: Data sent to the user's terminal

[0942] Operation: The generated video and ranking information are packaged into a data packet and sent to the user's terminal via the network.

[0943] Step 10:

[0944] The user terminal receives the short video and ranking information transmitted from the server.

[0945] Input: Short video and ranking information sent from the server

[0946] Output: Data to display

[0947] Operation: Receives data packets, parses them and stores them in internal memory.

[0948] Step 11:

[0949] The user terminal plays the received short video and displays the ranking information.

[0950] Input: Saved short videos, ranking information

[0951] Output: Video and information displayed to the user

[0952] How it works: Displays videos through a player and uses UI components to display ranking information on the screen.

[0953] Step 12:

[0954] The user selects the tourist spot they would like to visit based on the displayed information.

[0955] Input: Viewed video and ranking information

[0956] Output: Selected tourist spots

[0957] How it works: The user clicks or taps on a tourist attraction to select it, and that information is reflected in the user's itinerary.

[0958] (Application example 2)

[0959] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0960] Conventional tourist spot recommendation systems are unable to consider the user's current emotional state, making it difficult to suggest appropriate tourist spots. Furthermore, due to a lack of visually understandable information provision methods, users are unable to easily grasp the appeal of tourist spots. Furthermore, when applying similar technology to virtual stores, it has not been possible to achieve advanced personalization, such as product recommendations based on the user's emotional state.

[0961] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0962] In this invention, the server includes means for receiving search criteria entered from a user terminal, means for acquiring current location information, means for utilizing an emotion engine that recognizes the emotional state of the user from their voice and facial expression, means for extracting relevant tourist spots from a database based on the search criteria and location information, means for ranking the extracted tourist spots based on the user's hobbies, preferences, and emotional state, means for generating short videos of the ranked tourist spots, and means for transmitting the generated short videos and ranking information to the user terminal. This enables advanced personalization that takes into account the user's current emotional state and makes it possible to provide information that is visually easy to understand.

[0963] A "user terminal" is an electronic device that allows a user to input information or view received information, and includes smartphones, tablets, PCs, etc.

[0964] "Location information" is data indicating the user's current location, and is geographical location data obtained by a GPS function or the like.

[0965] An "emotion engine" is software or hardware that recognizes the user's emotional state from their voice and facial expressions.

[0966] A "database" is a system for storing and managing related information, which can be searched and extracted based on specific conditions.

[0967] "Ranking" refers to prioritizing and ordering items based on specific criteria.

[0968] A "short video" is a video that introduces specific content in a short period of time, usually about 30 seconds long.

[0969] A "generative model" is an algorithm or system for generating new content based on specific data or conditions.

[0970] A "server" is a computer system that provides and processes information over a network, and plays a central role in sending, receiving, and processing data.

[0971] To specifically implement the present invention, it is necessary to build the following system, which includes a user terminal, a server, an emotion engine, and various communication means.

[0972] User terminal configuration

[0973] 1. Displaying the input form

[0974] The user terminal uses HTML and JavaScript to display a form in which the user can enter product categories and search conditions. Through this form, the user can enter the products they want to purchase and their desired conditions.

[0975] 2. Obtaining location information

[0976] The user device obtains the current location information using the JavaScript navigator.geolocation API, which is used for subsequent processing.

[0977] 3. Operation of the Emotion Engine

[0978] The user device uses Google Cloud's Vision API and Speech-to-Text API to recognize the user's emotional state from their voice and facial expressions, allowing it to analyze the user's current emotional state.

[0979] 4. Data Transmission

[0980] The user device sends the search criteria entered by the user, the acquired location information, and the emotional state recognized by the emotion engine to the server via a POST request. The HTTPS protocol is used for communication with the server.

[0981] Server Configuration

[0982] 1. Data Reception

[0983] The server uses Node.js and Express to receive data sent from the user's device, including search criteria, location information, and emotional state.

[0984] 2. Database Matching

[0985] The server then compares the received data with the MongoDB database and extracts relevant information, specifically, extracting appropriate products from the product database, taking into account the user's emotional state and search criteria.

[0986] 3. Ranking

[0987] The server ranks the extracted products based on the user's tastes and preferences and emotional state, allowing it to prioritize the most suitable products for the user.

[0988] 4. Short video generation

[0989] The server uses FFmpeg and a generative AI model (e.g., OpenAI's GPT-3) to generate a short video for each ranked product, approximately 30 seconds long, showcasing the main appeal of each product.

[0990] 5. Data Transmission

[0991] The server transmits the generated short videos and ranking information to the user's terminal, allowing the user to visually understand the appeal of each product.

[0992] Specific examples

[0993] For example, when a user searches for "fashion items" using a virtual store app, the system works as follows: The user enters their search criteria and current location information into their device, and the emotion engine analyzes their emotional state. Based on this information, the server extracts and ranks products from the database that best fit the user's emotional state. Finally, a short video is generated using FFmpeg and a generative AI model and sent to the user's device.

[0994] Prompt Sentence Examples

[0995] If the user is relaxed, recommend some relaxed outfits that suit the user's taste from the latest fashion items. Example: Introduce five fashion items each in a short 30-second video.

[0996] In this way, highly accurate product recommendations that take into account the user's emotional state and easy-to-understand information can be provided.

[0997] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0998] Step 1:

[0999] The user inputs product categories and search conditions into an input form on the user's terminal. At this time, the user's input is sent as a form and organized as input data.

[1000] Step 2:

[1001] The user device obtains the current location information using the JavaScript navigator.geolocation API. The location information is obtained as latitude and longitude data and added to the input data.

[1002] Step 3:

[1003] The user device uses Google Cloud's Vision API and Speech-to-Text API to analyze voice and facial expressions to recognize the user's emotional state, and the recognition results are included in the input data.

[1004] Step 4:

[1005] The user device sends the entered search criteria, the acquired location information, and the recognized emotional state as data to the server in JSON format.

[1006] Step 5:

[1007] The server uses Node.js and Express to receive data sent from the user's device, including search criteria, location information, and emotional state.

[1008] Step 6:

[1009] The server then matches the received data with a MongoDB database to extract relevant product data, taking into account the emotional state of the user and providing filtered product data.

[1010] Step 7:

[1011] The server inputs the extracted product data into a generative AI model (OpenAI's GPT-3) to generate prompts that introduce the appeal of each product. These prompts then become the input data for generating short videos.

[1012] Step 8:

[1013] The server uses FFmpeg to generate a short video for each product based on the prompt text obtained from the generative AI model. The generated short video is about 30 seconds long and introduces the main appeal of the product.

[1014] Step 9:

[1015] The server transmits the generated short video and ranking information to the user terminal, and the transmitted data is formatted to be displayed on the user terminal.

[1016] Step 10:

[1017] The user terminal receives the short videos and ranking information sent from the server, and the received data is visually displayed to the user.

[1018] Step 11:

[1019] The user checks the displayed short video and ranking information and selects the product they wish to purchase. The selected product information is used for the next process.

[1020] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1021] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1022] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1023] [Third embodiment]

[1024] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1025] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1027] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1028] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1029] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1031] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1032] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1034] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1035] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1036] To specifically implement the present invention, it is necessary to build a system that includes a user terminal, a server, and various communication means. The purpose of this system is to make it easier for users to search for tourist spots and visually understand their attractions. The program processing of this system is as follows.

[1037] User terminal operation

[1038] 1. Displaying the input form

[1039] The user terminal displays a form for the user to input search criteria, such as the time, location, and specific preferences (e.g., natural beauty, historical sites, etc.) that the user wants to visit.

[1040] 2. Obtaining location information

[1041] The user device obtains the user's current location information using the device's built-in GPS function.

[1042] 3. Data Transmission

[1043] The user terminal transmits the search conditions entered by the user and the acquired location information to the server.

[1044] Server Operation

[1045] 1. Data Reception

[1046] The server receives the search conditions and location information sent from the user terminal.

[1047] 2. Database Matching

[1048] The server compares the received data with a database of tourist spots, and selects tourist spots that match the user's current location and search criteria.

[1049] 3. Filtering tourist attractions

[1050] The server filters the extracted tourist spots based on the user's hobbies, preferences, and past behavioral history, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[1051] 4. Short video generation

[1052] The server uses a generative AI model to generate short videos for each selected tourist spot. Each video is designed to convey the characteristics and appeal of the tourist spot in a short time of around 30 seconds.

[1053] 5. Data Transmission

[1054] The server transmits the generated short video and ranking information to the user terminal.

[1055] User terminal operation (continued)

[1056] 1. Data Reception

[1057] The user terminal receives the short video and ranking information transmitted from the server.

[1058] 2. Video playback and information display

[1059] The user device plays the received short video and displays the tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[1060] 3. Selecting tourist spots

[1061] Users can select the tourist spots they want to visit based on the displayed information, which helps users to plan their sightseeing trips comfortably and efficiently.

[1062] Specific examples

[1063] User Behavior

[1064] For example, suppose a user is searching for "tourist spots that can be enjoyed within an hour around Shinjuku Station." The user enters "1 hour" into the input form on the device and confirms that the current location is Shinjuku Station.

[1065] Device behavior

[1066] The device uses its GPS function to obtain the user's location information and sends the conditions "Shinjuku Station" and "1 hour" to the server.

[1067] Server Operation

[1068] After receiving the data, the server extracts tourist spots that can be reached and returned from Shinjuku Station within an hour from the database. Then, taking into consideration the user's past search history and hobbies (e.g., a love of nature), it selects and ranks Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street in Harajuku. It then generates short videos showcasing the attractions of these tourist spots and sends them to the user's device.

[1069] User terminal operation (continued)

[1070] The device plays the received short video and displays ranking information to the user, allowing the user to select Meiji Jingu.

[1071] In this way, a system embodying the present invention allows a user to intuitively and efficiently find the best tourist spots.

[1072] The processing flow will be explained below.

[1073] User terminal processing

[1074] Step 1:

[1075] The user inputs search conditions (for example, desired time and location) into a search form displayed on the terminal.

[1076] Step 2:

[1077] The device uses the GPS function to obtain current location information, which is then stored in the internal memory.

[1078] Step 3:

[1079] The terminal transmits the search conditions entered by the user and the acquired location information to the server.

[1080] Server Processing

[1081] Step 4:

[1082] The server receives the search conditions and location information sent from the user terminal.

[1083] Step 5:

[1084] The server checks the tourist spot database based on the received search criteria and location information. For example, it extracts relevant tourist spots based on the criteria "within one hour from Shinjuku Station."

[1085] Step 6:

[1086] The server filters the extracted tourist spots based on the user's hobbies, preferences, and past behavioral history, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[1087] Step 7:

[1088] The server uses a generative AI model to generate short videos for each ranked tourist spot, each of which is about 30 seconds long and introduces the main attractions of the tourist spot.

[1089] Step 8:

[1090] The server transmits the generated short video and ranking information to the user terminal.

[1091] User terminal processing (continued)

[1092] Step 9:

[1093] The terminal receives the short video and ranking information transmitted from the server.

[1094] Step 10:

[1095] The device plays the received short video to the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[1096] Step 11:

[1097] The user selects tourist spots they would like to visit based on the provided information, and the selected information can be used to help the user plan their trip.

[1098] In this way, tourist spots that meet the user's current needs are efficiently provided.

[1099] Example 1

[1100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1101] In the past, there were few ways for users to efficiently search for tourist spots and intuitively understand their attractions. As a result, it took a lot of time and effort for users to find the appropriate tourist spot based on their hobbies, preferences, and current location information. Furthermore, since the information provided was text-based, it was difficult to understand visually.

[1102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1103] In this invention, the server includes means for receiving search criteria entered from a user terminal, means for acquiring current location information related to the search criteria, means for extracting relevant tourist spots from a database based on the search criteria and location information, means for ranking the extracted tourist spots based on the user's hobbies and preferences, means for generating short videos for the ranked tourist spots using a generative AI model, and means for transmitting the generated short videos and ranking information to the user terminal, thereby enabling users to intuitively and efficiently find the best tourist spots.

[1104] "Search conditions" are information including the place and time the user wants to visit, and specific desired conditions (e.g., a place rich in nature, a historical place, etc.).

[1105] "Location information" is data that indicates the user's current geographical location, and is mainly obtained using the GPS function.

[1106] "Tourist attraction" refers to a place or facility that a user wishes to visit, the place or facility being publicly known for tourism or recreational purposes.

[1107] A "database" is a system for efficiently storing, accessing, and managing large amounts of information, including information about tourist attractions.

[1108] "Hobbies and preferences" refers to a user's personal tastes and interests, and are determined based on past behavioral history and profile data.

[1109] A "generative AI model" is an artificial intelligence model that uses machine learning techniques to generate new data (e.g., short videos). Specifically, it includes techniques for generating videos from text and images.

[1110] A "short video" is a video that conveys the appeal and characteristics of a tourist spot within a short period of time (usually 30 seconds or less).

[1111] "Ranking information" is a list that ranks the attractions of tourist spots based on the user's hobbies, preferences, and search conditions.

[1112] "User terminal" refers to an electronic device that is directly operated by a user, including smartphones, tablets, and personal computers.

[1113] A "server" is a central computer system that receives and processes data sent from user terminals, generates the necessary information, and retransmits it.

[1114] The present invention is implemented by constructing a system including a user terminal, a server, and various communication means. The purpose of this system is to enable users to easily search for tourist spots and visually understand their attractions.

[1115] User terminal operation

[1116] The user device first displays a form for entering search criteria for tourist attractions. This allows the user to enter the time, location, and specific preferences for the desired visit (e.g., natural beauty, historical sites, etc.). The user device also uses its built-in GPS to obtain the user's current location. This information is sent to the server and used to improve the accuracy of search results.

[1117] Server Operation

[1118] The server receives the search criteria and location information sent from the user's device. Based on the received data, the server checks a database of tourist spots and extracts relevant tourist spots. The extracted list of tourist spots is then filtered based on the user's hobbies, preferences, and past behavioral history, and organized in a ranking format. Based on this ranking information, the server uses a generative AI model (e.g., OpenAI's GPT-4) to generate short videos. Each video is approximately 30 seconds long and conveys the characteristics and appeal of the tourist spot. Finally, the server sends the generated short videos and ranking information to the user's device.

[1119] User terminal operation (continued)

[1120] The user device receives the short videos and ranking information sent from the server. This information is used to visually convey the attractions of tourist spots to the user. By playing the received short videos and displaying the ranking information, the user can intuitively and efficiently select tourist spots. This information is useful when the user selects tourist spots they want to visit.

[1121] Example operation

[1122] For example, consider the case where a user is searching for "fun tourist spots around Shinjuku Station within an hour." The user enters "1 hour" into the input form on their device and checks their current location. The device then uses its GPS function to obtain location information and sends the conditions "Shinjuku Station" and "1 hour" to the server. After receiving these conditions, the server extracts matching tourist spots from its database. Taking into consideration past search history and the user's hobbies, it selects Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street, creates short videos showcasing the appeal of each tourist spot, and sends them to the user's device. The device then plays these videos and displays ranking information. The user can then view these videos and select the spot they want to visit (for example, Meiji Jingu Shrine).

[1123] Example prompts to input to the generative AI model

[1124] "Please introduce tourist spots that can be enjoyed within an hour around Shinjuku Station. Please create a short video of 30 seconds or less that introduces the appeal of each spot. Please also take into consideration your past search history and hobbies (nature lovers)."

[1125] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1126] Step 1:

[1127] The user device displays a form for entering search criteria for tourist spots. Here, the user enters the desired time, location, and specific desired conditions for visiting. At this point, the input is the time (e.g., 1 hour), location (e.g., around Shinjuku Station), and desired conditions (e.g., a place rich in nature).

[1128] Step 2:

[1129] The user device obtains the current location information using the built-in GPS function. This is the operation of obtaining the current location data provided by the user device. The obtained location information (e.g., latitude and longitude data) is used for subsequent processing.

[1130] Step 3:

[1131] The user terminal transmits the input search criteria and the acquired location information to the server. The transmitted data includes the time, location, desired criteria, and current location information.

[1132] Step 4:

[1133] The server receives the search criteria and location information sent from the user terminal. The received data is temporarily stored in the server's memory and used in the next database matching process.

[1134] Step 5:

[1135] The server checks the tourist attraction database based on the received search criteria and location information. This process uses an SQL query to extract matching tourist attraction records from the database. The output data is a list of tourist attractions that match the search criteria.

[1136] Step 6:

[1137] The server filters the extracted tourist spot list based on the user's hobbies, preferences, and past behavioral history. The user profile data is used as input data, and a ranking list of tourist spots optimized for the user is generated as output.

[1138] Step 7:

[1139] The server generates short videos for the selected tourist spots using a generative AI model (e.g., OpenAI's GPT-4). In this process, tourist spot information is converted into prompt sentences as input data, and the generative AI model generates the video. The output data is a short video of about 30 seconds that conveys the characteristics of the tourist spot.

[1140] Step 8:

[1141] The server transmits the generated short video and ranking information to the user terminal. The transmitted data includes the short video file and the ranking information.

[1142] Step 9:

[1143] The user terminal receives the short video and ranking information sent from the server, and the received data is stored in the memory of the user terminal.

[1144] Step 10:

[1145] The user device plays the received short video and displays the ranking information. This process uses a media player for video playback and a UI for displaying the rankings. By viewing this, the user can intuitively understand the appeal of the tourist spot.

[1146] Step 11:

[1147] The user selects the tourist spots they wish to visit based on the displayed information, and this selection information is saved in the user's terminal and used for subsequent activities.

[1148] (Application example 1)

[1149] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1150] Modern logistics centers and industrial facilities are required to maximize worker efficiency and improve work speed and accuracy. However, there is a lack of immediate and intuitive guidance systems to achieve this. Furthermore, few systems can present optimal work patterns using individual worker location information and past behavioral history. Tourism information systems also lack visual representations that allow users to intuitively select the best tourist spots.

[1151] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1152] In this invention, the server includes means for receiving search criteria and worker location information input from a user terminal, means for extracting related tourist spots and work patterns from a database based on the search criteria and location information, means for ranking the extracted tourist spots and work patterns based on the user's hobbies and preferences and the worker's efficiency, means for generating short videos of the ranked tourist spots and work patterns, and means for transmitting the generated short videos and ranking information to the user terminal. This allows users to intuitively and efficiently select tourist spots and enables workers to efficiently identify optimal work patterns.

[1153] A "user terminal" is an electronic device used by a user, such as a computer, smartphone, or tablet.

[1154] "Search conditions" are desired conditions and requirements for tourist spots and work patterns entered by the user.

[1155] "Location information" is data indicating the current location of a user or worker, and is obtained using technologies such as GPS.

[1156] A "database" is a collection of information that stores information on tourist spots and work patterns and can be searched and extracted.

[1157] "Extraction means" is a function that extracts necessary information from a database based on specific conditions.

[1158] The "ranking means" is a function that ranks the extracted tourist spots and work patterns based on the user's preferences and the efficiency of the workers.

[1159] "Means for generating" refers to the function of creating visual content such as short videos based on related information.

[1160] The "transmitting means" is a communication function for sending the generated data or information to the user terminal.

[1161] A "system" is a collection of components that includes user terminals, servers, communication means, etc., and that as a whole realizes a specific function.

[1162] "Available time" is the time frame in which a user or worker can perform a particular activity.

[1163] "Weather conditions" is information that indicates the weather conditions in a particular area.

[1164] "Past search history" is a record of searches previously performed by a user.

[1165] "Behavioral history" is a record of the actions and work a worker has performed in the past.

[1166] "Mood state" is information that indicates the user's current state of mind or emotions.

[1167] A "generative model" is a program that uses specific algorithms and machine learning techniques to automatically generate materials such as short videos.

[1168] "Tourist spots" are tourist destinations and famous places that users can visit.

[1169] A "work pattern" is a combination of procedures and movements that allow workers to perform work efficiently.

[1170] To specifically implement this invention, it is necessary to build a system including a user terminal, a server, and various communication means. The purpose of this system is to make it easier for users to search for tourist spots and workers' work patterns and to visually understand their attractiveness and efficiency.

[1171] User terminal operation

[1172] 1. Displaying the input form

[1173] The user terminal displays a form that allows the user to enter search criteria. For example, in the case of tourist spots, users can enter the desired time, location, and specific preferences (such as places rich in nature or historical sites) for visiting. In the case of a logistics center, workers can enter their work details and desired conditions (such as working hours and break times).

[1174] 2. Obtaining location information

[1175] The user terminal obtains the user's current location information. This is done using the terminal's built-in GPS function. The logistics center uses the built-in GPS function or other location tracking technology.

[1176] 3. Data Transmission

[1177] The user terminal transmits the search conditions entered by the user and the acquired location information to the server.

[1178] Server Operation

[1179] 1. Data Reception

[1180] The server receives the search conditions and location information sent from the user terminal.

[1181] 2. Database Matching

[1182] The server compares the received data with a database of tourist spots and work patterns, and selects tourist spots and work patterns that match the user's current location and search criteria.

[1183] 3. Filtering tourist attractions and work patterns

[1184] The server filters the extracted tourist spots and work patterns based on the user's hobbies and preferences and the worker's past behavioral history, thereby ranking the tourist spots and work patterns that are most suitable for the user and worker.

[1185] 4. Short video generation

[1186] The server uses the generative AI model to generate short videos for each selected tourist spot and work pattern. Each video is designed to convey the characteristics and appeal of the tourist spot and the efficiency of the work pattern in a short time of around 30 seconds.

[1187] 5. Data Transmission

[1188] The server transmits the generated short video and ranking information to the user terminal.

[1189] User terminal operation (continued)

[1190] 1. Data Reception

[1191] The user terminal receives the short video and ranking information transmitted from the server.

[1192] 2. Video playback and information display

[1193] The user device plays the received short video and displays ranking information for tourist attractions and work patterns, allowing users and workers to visually understand the appeal of each tourist attraction and the efficiency of their work patterns.

[1194] 3. Selection of tourist spots and work patterns

[1195] Based on the displayed information, users can select tourist spots they want to visit and work patterns they want to perform. This information helps users to plan their sightseeing trips comfortably and efficiently, and also contributes to workers' efficient work.

[1196] Specific examples

[1197] User Behavior

[1198] For example, if a user is looking for "tourist spots that can be enjoyed within an hour around Shinjuku Station," the user enters "1 hour" into the input form on the device and confirms that the current location is Shinjuku Station. At a logistics center, a worker enters "I would like to check the inventory in Area A."

[1199] Device behavior

[1200] The device uses its GPS function to obtain the user's location information and sends the conditions "Shinjuku Station" and "1 hour" to the server. The logistics center obtains the worker's current location and sends "Area A" and "Work content" to the server.

[1201] Server Operation

[1202] After receiving the data, the server extracts tourist spots that can be reached and returned from Shinjuku Station within an hour from the database. It also considers past search history and the user's hobbies (e.g., a love of nature) to select and rank the tourist spots. The logistics center extracts and ranks efficient work patterns.

[1203] User terminal operation (continued)

[1204] The device plays the received short video and displays ranking information to the user, allowing the user to select tourist spots they want to visit. Similarly, workers can visually understand and select efficient work patterns.

[1205] Example prompts for generative AI models

[1206] "For a worker who is currently in area C and wants to perform picking work in area A, please generate a 30-second guidance video that includes an efficient route and important points to note."

[1207] The system allows users to intuitively and efficiently find tourist spots and helps workers identify optimal work patterns.

[1208] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1209] Step 1:

[1210] Displaying the input form

[1211] The user terminal displays a form that allows the user to enter search criteria. The user enters the desired time, location, and specific preferences (such as a place rich in nature or a historical site) as search criteria. At the logistics center, workers enter their work details and desired conditions (such as working hours and break times).

[1212] Input: The time, place, and specific wishes you want to visit are entered by the user.

[1213] Output: User input information (search criteria)

[1214] Step 2:

[1215] Obtaining location information

[1216] The user device obtains the user's current location information using the device's built-in GPS function. In the logistics center, the location information of workers is obtained using the built-in GPS function or other location tracking technology.

[1217] Input: Data from the device's built-in GPS function or location tracking technology

[1218] Output: Current location information obtained

[1219] Step 3:

[1220] Data transmission

[1221] The user terminal transmits the search conditions entered by the user and the acquired location information to the server.

[1222] Input: User input information (search criteria), current location information obtained

[1223] Output: Data sent to the server

[1224] Step 4:

[1225] Data reception

[1226] The server receives the search conditions and location information sent from the user terminal.

[1227] Input: Search criteria and location information sent from the user device

[1228] Output: Received search conditions, location information

[1229] Step 5:

[1230] Database collation

[1231] The server compares the received data with a database of tourist spots and work patterns, and searches for tourist spots and work patterns using the search criteria entered by the user and the current location information.

[1232] Input: Received search conditions, location information

[1233] Output: Extracted tourist spots or work patterns

[1234] Step 6:

[1235] Filtering tourist attractions and work patterns

[1236] The server filters the extracted tourist spots and work patterns based on the user's hobbies and preferences and the worker's past behavioral history. "User hobbies and preferences" are determined based on past search history and behavioral history, and "worker efficiency" is determined based on behavioral history and time measurement data.

[1237] Input: Extracted tourist spots or work patterns, user's hobby and preference information, worker's past behavior history information

[1238] Output: Ranked tourist spots or activity patterns

[1239] Step 7:

[1240] Short video generation

[1241] The server uses a generative AI model to generate short videos for each selected tourist spot and work pattern. Each video is short, about 30 seconds long, and conveys the characteristics and appeal of the tourist spot and the efficiency of the work pattern. A prompt sentence is input into the generative AI model to generate the video.

[1242] Input: Prompt sentences such as "Description and characteristics of tourist spots" or "Work procedures and efficiency of workers"

[1243] Output: Generated short video

[1244] Step 8:

[1245] Data transmission

[1246] The server transmits the generated short video and ranking information to the user terminal.

[1247] Input: Generated short videos, ranking information

[1248] Output: Data sent to the user terminal

[1249] Step 9:

[1250] Data reception

[1251] The user terminal receives the short video and ranking information transmitted from the server.

[1252] Input: Short video sent from the server, ranking information

[1253] Output: Received short videos and ranking information

[1254] Step 10:

[1255] Video playback and information display

[1256] The user device plays the received short video and displays ranking information for tourist spots and work patterns, allowing users and workers to visually understand the appeal of each tourist spot and the efficiency of their work patterns.

[1257] Input: Received short video, ranking information

[1258] Output: Short video played, ranking information displayed

[1259] Step 11:

[1260] Selection of tourist spots and work patterns

[1261] Based on the displayed information, users can select tourist spots they want to visit and work patterns they want to perform. This information helps users to plan their sightseeing trips comfortably and efficiently, and also contributes to workers' efficient work.

[1262] Input: Displayed ranking information, played short video

[1263] Output: Selected tourist spots or work patterns

[1264] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1265] To specifically implement the present invention, it is necessary to build a system that includes a user terminal, a server, an emotion engine, and various communication means. The purpose of this system is to allow users to search for tourist spots and visually understand their attractions, while also taking into account the user's emotional state to make more appropriate recommendations. The program processing of this system is as follows.

[1266] User terminal operation

[1267] 1. Displaying the input form

[1268] The user terminal displays a form for the user to input search criteria, such as the time, location, and specific preferences (e.g., natural beauty, historical sites, etc.) that the user wants to visit.

[1269] 2. Obtaining location information

[1270] The user device acquires current location information using the GPS function and stores the acquired location information in its internal memory.

[1271] 3. Operation of the Emotion Engine

[1272] The user device uses a built-in emotion engine to recognize the user's emotional state from their voice and facial expressions. The emotion engine determines the user's current emotional state through analysis of the user's voice tone and facial expressions.

[1273] 4. Data Transmission

[1274] The user terminal transmits to the server the search conditions entered by the user, the acquired location information, and the emotional state recognized by the emotion engine.

[1275] Server Operation

[1276] 1. Data Reception

[1277] The server receives the search conditions, location information, and emotional state transmitted from the user terminal.

[1278] 2. Database Matching

[1279] The server compares the received search criteria, location information, and emotional state with a database of tourist spots. For example, it extracts relevant tourist spots based on the criteria "within one hour from Shinjuku Station" and "places where you can relax."

[1280] 3. Filtering tourist attractions

[1281] The server filters the extracted tourist spots based on the user's hobbies, preferences, past behavioral history, and emotional state, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[1282] 4. Short video generation

[1283] The server uses a generative AI model to generate short videos for each ranked tourist spot, each of which is about 30 seconds long and introduces the main attractions of the tourist spot.

[1284] 5. Data Transmission

[1285] The server transmits the generated short video and ranking information to the user terminal.

[1286] User terminal operation (continued)

[1287] 1. Data Reception

[1288] The user terminal receives the short video and ranking information transmitted from the server.

[1289] 2. Video playback and information display

[1290] The user device plays the received short video for the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[1291] 3. Selecting tourist spots

[1292] The user can select the tourist spots they want to visit based on the displayed information, and the selected information can be used to help the user plan their trip.

[1293] Specific examples

[1294] User Behavior

[1295] For example, suppose a user is looking for "a relaxing tourist spot that can be enjoyed within an hour around Shinjuku Station." The user enters "1 hour" and "relaxation" into the input form on the device and confirms that the current location is Shinjuku Station.

[1296] Device behavior

[1297] The device uses its GPS function to obtain the user's location information, and then uses an emotion engine to analyze the user's current emotional state from their voice and facial expressions, and sends the results to the server.

[1298] Server Operation

[1299] After receiving the data, the server extracts tourist spots that can be reached and returned from Shinjuku Station within an hour from the database. Furthermore, taking into account the emotional state of "places where you can relax," it selects and ranks Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street in Harajuku. It then generates short videos showcasing the attractions of these tourist spots and sends them to the user's device.

[1300] User terminal operation (continued)

[1301] The device plays the received short video and displays ranking information to the user, allowing the user to select Meiji Jingu.

[1302] In this way, the system embodying the present invention efficiently provides optimal tourist spots, taking into account the user's current emotional state.

[1303] The processing flow will be explained below.

[1304] User terminal processing

[1305] Step 1:

[1306] The user inputs search criteria (for example, the place they want to visit and the time they can be there) into the search form displayed on the terminal.

[1307] Step 2:

[1308] The device uses the built-in GPS function to obtain current location information, which is then stored in the internal memory.

[1309] Step 3:

[1310] The device uses a built-in emotion engine to recognize the user's emotional state from their voice and facial expressions. The emotion engine determines the user's current emotional state (e.g., excitement, relaxation, stress, etc.) through analysis of the user's voice tone and facial expressions.

[1311] Step 4:

[1312] The terminal transmits the search criteria entered by the user, the acquired location information, and the emotional state recognized by the emotion engine to the server.

[1313] Server Processing

[1314] Step 5:

[1315] The server receives the search conditions, location information, and emotional state transmitted from the user terminal.

[1316] Step 6:

[1317] The server compares the received search criteria, location information, and emotional state with a database of tourist spots. For example, it extracts relevant tourist spots based on the criteria "within one hour from Shinjuku Station" and "places where you can relax."

[1318] Step 7:

[1319] The server then filters the extracted tourist spots based on the user's preferences, past activities, and emotional state, prioritizing spots that meet criteria such as "places rich in nature" or "quiet places."

[1320] Step 8:

[1321] The server uses a generative AI model to generate short videos for each of the filtered tourist spots, each of which is about 30 seconds long and introduces the main attractions of the tourist spot.

[1322] Step 9:

[1323] The server transmits the generated short video and ranking information to the user terminal.

[1324] User terminal processing (continued)

[1325] Step 10:

[1326] The terminal receives the short video and ranking information transmitted from the server.

[1327] Step 11:

[1328] The device plays the received short video to the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[1329] Step 12:

[1330] The user selects tourist spots that they would like to visit based on the provided information, and the selected information can be used to help the user plan their trip.

[1331] Specific examples

[1332] For example, suppose a user is searching for "a relaxing tourist spot that can be enjoyed within an hour around Shinjuku Station." The user enters "1 hour" and "relaxing" into the input form on the device and confirms that the current location is Shinjuku Station.

[1333] The device uses its GPS to obtain location information for Shinjuku Station, and then uses an emotion engine to analyze the user's current emotional state based on their voice and facial expressions, obtaining a "relaxed state" as the result. This information is then sent to the server.

[1334] The server receives this data and compares it with a database of tourist spots based on the emotional information, such as "within an hour from Shinjuku Station" and "a relaxing place." As a result, Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street in Harajuku are selected and ranked. A 30-second short video introducing the attractions of the selected tourist spots is generated and sent to the user's device.

[1335] The user terminal plays the received short video and displays ranking information for Meiji Shrine, Yoyogi Park, and Takeshita Street in Harajuku to the user, who then decides to visit Meiji Shrine.

[1336] This series of steps allows users to efficiently find tourist spots that best suit their current emotional state.

[1337] Example 2

[1338] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1339] Conventional tourist spot recommendation systems suggest tourist spots based on the search criteria and location information entered by the user, but they are unable to make recommendations that take into account the user's current emotional state, making it difficult to fully meet the user's needs.In addition, there is a lack of a means to efficiently convey the appeal of the suggested tourist spots, which often leaves users unsure which one to choose.

[1340] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1341] In this invention, the server includes means for receiving search criteria entered from a user terminal, means for acquiring current location information related to the search criteria, means for acquiring and analyzing emotional data such as voice and facial expression, means for extracting related tourist spots from a database based on the search criteria, location information, and emotional data, means for ranking the extracted tourist spots based on the user's hobbies, preferences, and emotional state, means for using a generative AI model to generate short videos for the ranked tourist spots, and means for transmitting the generated short videos and ranking information to the user terminal. This makes it possible to efficiently provide the user with the most suitable tourist spots while taking into account the user's current emotional state, and to visually convey their appeal.

[1342] A "user terminal" is an electronic device through which a user inputs search criteria and receives search results, and includes a smartphone, tablet, or personal computer.

[1343] "Search conditions" are information that a user inputs as conditions for the tourist spot they would like to visit, and include the time and place they would like to visit, specific requests, etc.

[1344] "Location information" is geographical data that indicates where the user is currently located, obtained using the GPS function of the user terminal.

[1345] "Emotion data" is information about the user's emotional state obtained by analyzing the user's voice, facial expressions, and the like.

[1346] A "database" is a data storage that accumulates information about tourist spots, and allows for the collation and extraction of information.

[1347] "Ranking" is the process of prioritizing and organizing the extracted tourist spots based on the user's hobbies, preferences, and emotional state.

[1348] "Generative AI model" is an abbreviation for generative artificial intelligence model, and refers to the AI ​​technology used to generate short videos and the like in this invention.

[1349] "Short videos" are short video contents created to introduce the main attractions of tourist spots in a short amount of time.

[1350] A "prompt" is an instruction given to a generative AI model, which indicates the specific content that the model needs to generate an appropriate video.

[1351] MODE FOR CARRYING OUT THE INVENTION

[1352] To specifically implement the present invention, it is necessary to build a system including a user terminal, a server, an emotion engine, and various communication means. The purpose of this system is to enable users to search for tourist spots and visually understand their attractions, while also providing more appropriate recommendations that take into account the user's emotional state.

[1353] Hardware and software used

[1354] User devices: smartphones, tablets, personal computers, etc.

[1355] Server: A high-performance computer that receives, processes, and transmits data.

[1356] Emotion Engine: Software for voice tone and facial expression analysis

[1357] Generative AI models: Generative artificial intelligence models, such as OpenAI's GPT-4

[1358] Program processing explanation

[1359] User terminal

[1360] 1. Displaying the input form

[1361] The user terminal displays a form for the user to input search criteria, such as the time, location, and specific preferences (e.g., natural beauty, historical sites, etc.) that the user wants to visit.

[1362] 2. Obtaining location information

[1363] The user device acquires current location information using the built-in GPS function and stores the acquired location information in its internal memory.

[1364] 3. Operation of the Emotion Engine

[1365] The user device uses a built-in emotion engine to recognize the user's emotional state from their voice and facial expressions. The emotion engine determines the user's current emotional state through analysis of the user's voice tone and facial expressions.

[1366] 4. Data Transmission

[1367] The user terminal transmits to the server the search conditions entered by the user, the acquired location information, and the emotional state recognized by the emotion engine.

[1368] server

[1369] 1. Data Reception

[1370] The server receives the search conditions, location information, and emotional state transmitted from the user terminal.

[1371] 2. Database Matching

[1372] The server searches a database of tourist attractions based on the received search criteria, location information, and emotional state. For example, it extracts relevant tourist attractions based on the criteria "within an hour from the city center" and "places where you can relax."

[1373] 3. Filtering tourist attractions

[1374] The server filters the extracted tourist spots based on the user's hobbies, preferences, past behavioral history, and emotional state, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[1375] 4. Short video generation

[1376] The server uses a generative AI model to generate a short video for each ranked tourist spot. Each video introduces the main attractions of the tourist spot in a short time of about 30 seconds. The following sentence is used as a prompt for the generative AI model:

[1377] "Write a short 30-second video script about a relaxing tourist spot within an hour of the city center. The spot is a famous park."

[1378] 5. Data Transmission

[1379] The server transmits the generated short video and ranking information to the user terminal.

[1380] User terminal

[1381] 1. Data Reception

[1382] The user terminal receives the short video and ranking information transmitted from the server.

[1383] 2. Video playback and information display

[1384] The user device plays the received short video for the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[1385] 3. Selecting tourist spots

[1386] The user can select the tourist spots they want to visit based on the displayed information, and this selection information can be used to help the user plan their trip.

[1387] Specific examples

[1388] User Behavior

[1389] For example, suppose a user is looking for "a relaxing and enjoyable tourist spot within an hour of the city center." The user enters "1 hour" and "relaxing" into the input form on the device and confirms that the current location is the city center.

[1390] Device behavior

[1391] The device uses its GPS function to obtain the user's location information, and then uses an emotion engine to analyze the user's current emotional state from their voice and facial expressions, and sends the results to the server.

[1392] Server Operation

[1393] After receiving the data, the server extracts tourist spots that can be reached and returned from the city center within an hour from the database. It then selects and ranks several suitable locations, taking into account the emotional state of "a place to relax." It then generates short videos showcasing the attractions of these tourist spots and sends them to the user's device.

[1394] User terminal operation

[1395] The device plays the received short video and displays ranking information to the user, allowing the user to select the tourist spots they would like to visit.

[1396] In this way, the system embodying the present invention efficiently provides optimal tourist spots, taking into account the user's current emotional state.

[1397] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1398] Step 1:

[1399] The user terminal displays a search condition input form.

[1400] Input: User's tourist attraction preferences (time, location, specific preferences, etc.)

[1401] Output: The search criteria entered by the user

[1402] How it works: Using a drawing library or UI component, a search field is displayed on the form screen, which the user can follow to enter specific search criteria.

[1403] Step 2:

[1404] The user terminal acquires location information using the GPS function.

[1405] Input: Current GPS signal

[1406] Output: Current location of the user (latitude and longitude information)

[1407] How it works: Calls the device's location service to obtain the user's real-time location, which is then stored in the device's internal memory.

[1408] Step 3:

[1409] The user terminal uses an emotion engine to recognize the user's emotional state.

[1410] Input: User's voice data, facial expression images

[1411] Output: User's emotional state (e.g., relaxed, excited, tired, etc.)

[1412] How it works: Analyzes the tone of voice input and performs facial expression analysis using image data acquired from the camera, thereby determining the user's emotions and treating their emotional state as data.

[1413] Step 4:

[1414] The user terminal transmits search conditions, location information, and emotional state to the server.

[1415] Input: Search criteria, location, emotional state

[1416] Output: Data packet to send to the server

[1417] What it does: It formats the data to be sent into packets and sends them over the network to the server.

[1418] Step 5:

[1419] The server receives the data sent from the user terminal.

[1420] Input: Search criteria, location information, and emotional state sent from the user device

[1421] Output: Data used for database matching

[1422] Operation: Analyzes the received data, converts it into the required format and stores it in the internal memory.

[1423] Step 6:

[1424] The server checks the database and extracts relevant tourist spots.

[1425] Input: Search criteria, location, emotional state

[1426] Output: A list of matched attractions

[1427] Behavior: Generates a database query and performs a search based on the received data. Returns a list of matching attractions.

[1428] Step 7:

[1429] The server filters and ranks the extracted tourist attractions.

[1430] Input: A list of matched tourist spots, the user's interests and preferences, and their emotional state

[1431] Output: A list of ranked tourist attractions

[1432] How it works: Based on the user's recorded hobbies, preferences, past behavioral history, and current emotional state, the system calculates and ranks the relevance of each tourist spot.

[1433] Step 8:

[1434] The server generates the short video using the generative AI model.

[1435] Input: A list of ranked tourist attractions, a prompt

[1436] Output: Short videos about each tourist attraction

[1437] How it works: The model is fed with preconditions and prompts, and generates a 30-second video introducing the main attractions of each tourist spot.

[1438] Step 9:

[1439] The server transmits the short video and ranking information to the user terminal.

[1440] Input: Generated short videos, ranking information

[1441] Output: Data sent to the user's terminal

[1442] Operation: The generated video and ranking information are packaged into a data packet and sent to the user's terminal via the network.

[1443] Step 10:

[1444] The user terminal receives the short video and ranking information transmitted from the server.

[1445] Input: Short video and ranking information sent from the server

[1446] Output: Data to display

[1447] Operation: Receives data packets, parses them and stores them in internal memory.

[1448] Step 11:

[1449] The user terminal plays the received short video and displays the ranking information.

[1450] Input: Saved short videos, ranking information

[1451] Output: Video and information displayed to the user

[1452] How it works: Displays videos through a player and uses UI components to display ranking information on the screen.

[1453] Step 12:

[1454] The user selects the tourist spot they would like to visit based on the displayed information.

[1455] Input: Viewed video and ranking information

[1456] Output: Selected tourist spots

[1457] How it works: The user clicks or taps on a tourist attraction to select it, and that information is reflected in the user's itinerary.

[1458] (Application example 2)

[1459] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1460] Conventional tourist spot recommendation systems are unable to consider the user's current emotional state, making it difficult to suggest appropriate tourist spots. Furthermore, due to a lack of visually understandable information provision methods, users are unable to easily grasp the appeal of tourist spots. Furthermore, when applying similar technology to virtual stores, it has not been possible to achieve advanced personalization, such as product recommendations based on the user's emotional state.

[1461] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1462] In this invention, the server includes means for receiving search criteria entered from a user terminal, means for acquiring current location information, means for utilizing an emotion engine that recognizes the emotional state of the user from their voice and facial expression, means for extracting relevant tourist spots from a database based on the search criteria and location information, means for ranking the extracted tourist spots based on the user's hobbies, preferences, and emotional state, means for generating short videos of the ranked tourist spots, and means for transmitting the generated short videos and ranking information to the user terminal. This enables advanced personalization that takes into account the user's current emotional state and makes it possible to provide information that is visually easy to understand.

[1463] A "user terminal" is an electronic device that allows a user to input information or view received information, and includes smartphones, tablets, PCs, etc.

[1464] "Location information" is data indicating the user's current location, and is geographical location data obtained by a GPS function or the like.

[1465] An "emotion engine" is software or hardware that recognizes the user's emotional state from their voice and facial expressions.

[1466] A "database" is a system for storing and managing related information, which can be searched and extracted based on specific conditions.

[1467] "Ranking" refers to prioritizing and ordering items based on specific criteria.

[1468] A "short video" is a video that introduces specific content in a short period of time, usually about 30 seconds long.

[1469] A "generative model" is an algorithm or system for generating new content based on specific data or conditions.

[1470] A "server" is a computer system that provides and processes information over a network, and plays a central role in sending, receiving, and processing data.

[1471] To specifically implement the present invention, it is necessary to build the following system, which includes a user terminal, a server, an emotion engine, and various communication means.

[1472] User terminal configuration

[1473] 1. Displaying the input form

[1474] The user terminal uses HTML and JavaScript to display a form in which the user can enter product categories and search conditions. Through this form, the user can enter the products they want to purchase and their desired conditions.

[1475] 2. Obtaining location information

[1476] The user device obtains the current location information using the JavaScript navigator.geolocation API, which is used for subsequent processing.

[1477] 3. Operation of the Emotion Engine

[1478] The user device uses Google Cloud's Vision API and Speech-to-Text API to recognize the user's emotional state from their voice and facial expressions, allowing it to analyze the user's current emotional state.

[1479] 4. Data Transmission

[1480] The user device sends the search criteria entered by the user, the acquired location information, and the emotional state recognized by the emotion engine to the server via a POST request. The HTTPS protocol is used for communication with the server.

[1481] Server Configuration

[1482] 1. Data Reception

[1483] The server uses Node.js and Express to receive data sent from the user's device, including search criteria, location information, and emotional state.

[1484] 2. Database Matching

[1485] The server then compares the received data with the MongoDB database and extracts relevant information, specifically, extracting appropriate products from the product database, taking into account the user's emotional state and search criteria.

[1486] 3. Ranking

[1487] The server ranks the extracted products based on the user's tastes and preferences and emotional state, allowing it to prioritize the most suitable products for the user.

[1488] 4. Short video generation

[1489] The server uses FFmpeg and a generative AI model (e.g., OpenAI's GPT-3) to generate a short video for each ranked product, approximately 30 seconds long, showcasing the main appeal of each product.

[1490] 5. Data Transmission

[1491] The server transmits the generated short videos and ranking information to the user's terminal, allowing the user to visually understand the appeal of each product.

[1492] Specific examples

[1493] For example, when a user searches for "fashion items" using a virtual store app, the system works as follows: The user enters their search criteria and current location information into their device, and the emotion engine analyzes their emotional state. Based on this information, the server extracts and ranks products from the database that best fit the user's emotional state. Finally, a short video is generated using FFmpeg and a generative AI model and sent to the user's device.

[1494] Prompt Sentence Examples

[1495] If the user is relaxed, recommend some relaxed outfits that suit the user's taste from the latest fashion items. Example: Introduce five fashion items each in a short 30-second video.

[1496] In this way, highly accurate product recommendations that take into account the user's emotional state and easy-to-understand information can be provided.

[1497] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1498] Step 1:

[1499] The user inputs product categories and search conditions into an input form on the user's terminal. At this time, the user's input is sent as a form and organized as input data.

[1500] Step 2:

[1501] The user device obtains the current location information using the JavaScript navigator.geolocation API. The location information is obtained as latitude and longitude data and added to the input data.

[1502] Step 3:

[1503] The user device uses Google Cloud's Vision API and Speech-to-Text API to analyze voice and facial expressions to recognize the user's emotional state, and the recognition results are included in the input data.

[1504] Step 4:

[1505] The user device sends the entered search criteria, the acquired location information, and the recognized emotional state as data to the server in JSON format.

[1506] Step 5:

[1507] The server uses Node.js and Express to receive data sent from the user's device, including search criteria, location information, and emotional state.

[1508] Step 6:

[1509] The server then matches the received data with a MongoDB database to extract relevant product data, taking into account the emotional state of the user and providing filtered product data.

[1510] Step 7:

[1511] The server inputs the extracted product data into a generative AI model (OpenAI's GPT-3) to generate prompts that introduce the appeal of each product. These prompts then become the input data for generating short videos.

[1512] Step 8:

[1513] The server uses FFmpeg to generate a short video for each product based on the prompt text obtained from the generative AI model. The generated short video is about 30 seconds long and introduces the main appeal of the product.

[1514] Step 9:

[1515] The server transmits the generated short video and ranking information to the user terminal, and the transmitted data is formatted to be displayed on the user terminal.

[1516] Step 10:

[1517] The user terminal receives the short videos and ranking information sent from the server, and the received data is visually displayed to the user.

[1518] Step 11:

[1519] The user checks the displayed short video and ranking information and selects the product they wish to purchase. The selected product information is used for the next process.

[1520] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1521] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1522] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1523] [Fourth embodiment]

[1524] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1525] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1526] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1527] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1528] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1529] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1530] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1531] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1532] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1533] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1534] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1535] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1536] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1537] To specifically implement the present invention, it is necessary to build a system that includes a user terminal, a server, and various communication means. The purpose of this system is to make it easier for users to search for tourist spots and visually understand their attractions. The program processing of this system is as follows.

[1538] User terminal operation

[1539] 1. Displaying the input form

[1540] The user terminal displays a form for the user to input search criteria, such as the time, location, and specific preferences (e.g., natural beauty, historical sites, etc.) that the user wants to visit.

[1541] 2. Obtaining location information

[1542] The user device obtains the user's current location information using the device's built-in GPS function.

[1543] 3. Data Transmission

[1544] The user terminal transmits the search conditions entered by the user and the acquired location information to the server.

[1545] Server Operation

[1546] 1. Data Reception

[1547] The server receives the search conditions and location information sent from the user terminal.

[1548] 2. Database Matching

[1549] The server compares the received data with a database of tourist spots, and selects tourist spots that match the user's current location and search criteria.

[1550] 3. Filtering tourist attractions

[1551] The server filters the extracted tourist spots based on the user's hobbies, preferences, and past behavioral history, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[1552] 4. Short video generation

[1553] The server uses a generative AI model to generate short videos for each selected tourist spot. Each video is designed to convey the characteristics and appeal of the tourist spot in a short time of around 30 seconds.

[1554] 5. Data Transmission

[1555] The server transmits the generated short video and ranking information to the user terminal.

[1556] User terminal operation (continued)

[1557] 1. Data Reception

[1558] The user terminal receives the short video and ranking information transmitted from the server.

[1559] 2. Video playback and information display

[1560] The user device plays the received short video and displays the tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[1561] 3. Selecting tourist spots

[1562] Users can select the tourist spots they want to visit based on the displayed information, which helps users to plan their sightseeing trips comfortably and efficiently.

[1563] Specific examples

[1564] User Behavior

[1565] For example, suppose a user is searching for "tourist spots that can be enjoyed within an hour around Shinjuku Station." The user enters "1 hour" into the input form on the device and confirms that the current location is Shinjuku Station.

[1566] Device behavior

[1567] The device uses its GPS function to obtain the user's location information and sends the conditions "Shinjuku Station" and "1 hour" to the server.

[1568] Server Operation

[1569] After receiving the data, the server extracts tourist spots that can be reached and returned from Shinjuku Station within an hour from the database. Then, taking into consideration the user's past search history and hobbies (e.g., a love of nature), it selects and ranks Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street in Harajuku. It then generates short videos showcasing the attractions of these tourist spots and sends them to the user's device.

[1570] User terminal operation (continued)

[1571] The device plays the received short video and displays ranking information to the user, allowing the user to select Meiji Jingu.

[1572] In this way, a system embodying the present invention allows a user to intuitively and efficiently find the best tourist spots.

[1573] The processing flow will be explained below.

[1574] User terminal processing

[1575] Step 1:

[1576] The user inputs search conditions (for example, desired time and location) into a search form displayed on the terminal.

[1577] Step 2:

[1578] The device uses the GPS function to obtain current location information, which is then stored in the internal memory.

[1579] Step 3:

[1580] The terminal transmits the search conditions entered by the user and the acquired location information to the server.

[1581] Server Processing

[1582] Step 4:

[1583] The server receives the search conditions and location information sent from the user terminal.

[1584] Step 5:

[1585] The server checks the tourist spot database based on the received search criteria and location information. For example, it extracts relevant tourist spots based on the criteria "within one hour from Shinjuku Station."

[1586] Step 6:

[1587] The server filters the extracted tourist spots based on the user's hobbies, preferences, and past behavioral history, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[1588] Step 7:

[1589] The server uses a generative AI model to generate short videos for each ranked tourist spot, each of which is about 30 seconds long and introduces the main attractions of the tourist spot.

[1590] Step 8:

[1591] The server transmits the generated short video and ranking information to the user terminal.

[1592] User terminal processing (continued)

[1593] Step 9:

[1594] The terminal receives the short video and ranking information transmitted from the server.

[1595] Step 10:

[1596] The device plays the received short video to the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[1597] Step 11:

[1598] The user selects tourist spots they would like to visit based on the provided information, and the selected information can be used to help the user plan their trip.

[1599] In this way, tourist spots that meet the user's current needs are efficiently provided.

[1600] Example 1

[1601] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1602] In the past, there were few ways for users to efficiently search for tourist spots and intuitively understand their attractions. As a result, it took a lot of time and effort for users to find the appropriate tourist spot based on their hobbies, preferences, and current location information. Furthermore, since the information provided was text-based, it was difficult to understand visually.

[1603] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1604] In this invention, the server includes means for receiving search criteria entered from a user terminal, means for acquiring current location information related to the search criteria, means for extracting relevant tourist spots from a database based on the search criteria and location information, means for ranking the extracted tourist spots based on the user's hobbies and preferences, means for generating short videos for the ranked tourist spots using a generative AI model, and means for transmitting the generated short videos and ranking information to the user terminal, thereby enabling users to intuitively and efficiently find the best tourist spots.

[1605] "Search conditions" are information including the place and time the user wants to visit, and specific desired conditions (e.g., a place rich in nature, a historical place, etc.).

[1606] "Location information" is data that indicates the user's current geographical location, and is mainly obtained using the GPS function.

[1607] "Tourist attraction" refers to a place or facility that a user wishes to visit, the place or facility being publicly known for tourism or recreational purposes.

[1608] A "database" is a system for efficiently storing, accessing, and managing large amounts of information, including information about tourist attractions.

[1609] "Hobbies and preferences" refers to a user's personal tastes and interests, and are determined based on past behavioral history and profile data.

[1610] A "generative AI model" is an artificial intelligence model that uses machine learning techniques to generate new data (e.g., short videos). Specifically, it includes techniques for generating videos from text and images.

[1611] A "short video" is a video that conveys the appeal and characteristics of a tourist spot within a short period of time (usually 30 seconds or less).

[1612] "Ranking information" is a list that ranks the attractions of tourist spots based on the user's hobbies, preferences, and search conditions.

[1613] "User terminal" refers to an electronic device that is directly operated by a user, including smartphones, tablets, and personal computers.

[1614] A "server" is a central computer system that receives and processes data sent from user terminals, generates the necessary information, and retransmits it.

[1615] The present invention is implemented by constructing a system including a user terminal, a server, and various communication means. The purpose of this system is to enable users to easily search for tourist spots and visually understand their attractions.

[1616] User terminal operation

[1617] The user device first displays a form for entering search criteria for tourist attractions. This allows the user to enter the time, location, and specific preferences for the desired visit (e.g., natural beauty, historical sites, etc.). The user device also uses its built-in GPS to obtain the user's current location. This information is sent to the server and used to improve the accuracy of search results.

[1618] Server Operation

[1619] The server receives the search criteria and location information sent from the user's device. Based on the received data, the server checks a database of tourist spots and extracts relevant tourist spots. The extracted list of tourist spots is then filtered based on the user's hobbies, preferences, and past behavioral history, and organized in a ranking format. Based on this ranking information, the server uses a generative AI model (e.g., OpenAI's GPT-4) to generate short videos. Each video is approximately 30 seconds long and conveys the characteristics and appeal of the tourist spot. Finally, the server sends the generated short videos and ranking information to the user's device.

[1620] User terminal operation (continued)

[1621] The user device receives the short videos and ranking information sent from the server. This information is used to visually convey the attractions of tourist spots to the user. By playing the received short videos and displaying the ranking information, the user can intuitively and efficiently select tourist spots. This information is useful when the user selects tourist spots they want to visit.

[1622] Example operation

[1623] For example, consider the case where a user is searching for "fun tourist spots around Shinjuku Station within an hour." The user enters "1 hour" into the input form on their device and checks their current location. The device then uses its GPS function to obtain location information and sends the conditions "Shinjuku Station" and "1 hour" to the server. After receiving these conditions, the server extracts matching tourist spots from its database. Taking into consideration past search history and the user's hobbies, it selects Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street, creates short videos showcasing the appeal of each tourist spot, and sends them to the user's device. The device then plays these videos and displays ranking information. The user can then view these videos and select the spot they want to visit (for example, Meiji Jingu Shrine).

[1624] Example prompts to input to the generative AI model

[1625] "Please introduce tourist spots that can be enjoyed within an hour around Shinjuku Station. Please create a short video of 30 seconds or less that introduces the appeal of each spot. Please also take into consideration your past search history and hobbies (nature lovers)."

[1626] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1627] Step 1:

[1628] The user device displays a form for entering search criteria for tourist spots. Here, the user enters the desired time, location, and specific desired conditions for visiting. At this point, the input is the time (e.g., 1 hour), location (e.g., around Shinjuku Station), and desired conditions (e.g., a place rich in nature).

[1629] Step 2:

[1630] The user device obtains the current location information using the built-in GPS function. This is the operation of obtaining the current location data provided by the user device. The obtained location information (e.g., latitude and longitude data) is used for subsequent processing.

[1631] Step 3:

[1632] The user terminal transmits the input search criteria and the acquired location information to the server. The transmitted data includes the time, location, desired criteria, and current location information.

[1633] Step 4:

[1634] The server receives the search criteria and location information sent from the user terminal. The received data is temporarily stored in the server's memory and used in the next database matching process.

[1635] Step 5:

[1636] The server checks the tourist attraction database based on the received search criteria and location information. This process uses an SQL query to extract matching tourist attraction records from the database. The output data is a list of tourist attractions that match the search criteria.

[1637] Step 6:

[1638] The server filters the extracted tourist spot list based on the user's hobbies, preferences, and past behavioral history. The user profile data is used as input data, and a ranking list of tourist spots optimized for the user is generated as output.

[1639] Step 7:

[1640] The server generates short videos for the selected tourist spots using a generative AI model (e.g., OpenAI's GPT-4). In this process, tourist spot information is converted into prompt sentences as input data, and the generative AI model generates the video. The output data is a short video of about 30 seconds that conveys the characteristics of the tourist spot.

[1641] Step 8:

[1642] The server transmits the generated short video and ranking information to the user terminal. The transmitted data includes the short video file and the ranking information.

[1643] Step 9:

[1644] The user terminal receives the short video and ranking information sent from the server, and the received data is stored in the memory of the user terminal.

[1645] Step 10:

[1646] The user device plays the received short video and displays the ranking information. This process uses a media player for video playback and a UI for displaying the rankings. By viewing this, the user can intuitively understand the appeal of the tourist spot.

[1647] Step 11:

[1648] The user selects the tourist spots they wish to visit based on the displayed information, and this selection information is saved in the user's terminal and used for subsequent activities.

[1649] (Application example 1)

[1650] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1651] Modern logistics centers and industrial facilities are required to maximize worker efficiency and improve work speed and accuracy. However, there is a lack of immediate and intuitive guidance systems to achieve this. Furthermore, few systems can present optimal work patterns using individual worker location information and past behavioral history. Tourism information systems also lack visual representations that allow users to intuitively select the best tourist spots.

[1652] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1653] In this invention, the server includes means for receiving search criteria and worker location information input from a user terminal, means for extracting related tourist spots and work patterns from a database based on the search criteria and location information, means for ranking the extracted tourist spots and work patterns based on the user's hobbies and preferences and the worker's efficiency, means for generating short videos of the ranked tourist spots and work patterns, and means for transmitting the generated short videos and ranking information to the user terminal. This allows users to intuitively and efficiently select tourist spots and enables workers to efficiently identify optimal work patterns.

[1654] A "user terminal" is an electronic device used by a user, such as a computer, smartphone, or tablet.

[1655] "Search conditions" are desired conditions and requirements for tourist spots and work patterns entered by the user.

[1656] "Location information" is data indicating the current location of a user or worker, and is obtained using technologies such as GPS.

[1657] A "database" is a collection of information that stores information on tourist spots and work patterns and can be searched and extracted.

[1658] "Extraction means" is a function that extracts necessary information from a database based on specific conditions.

[1659] The "ranking means" is a function that ranks the extracted tourist spots and work patterns based on the user's preferences and the efficiency of the workers.

[1660] "Means for generating" refers to the function of creating visual content such as short videos based on related information.

[1661] The "transmitting means" is a communication function for sending the generated data or information to the user terminal.

[1662] A "system" is a collection of components that includes user terminals, servers, communication means, etc., and that as a whole realizes a specific function.

[1663] "Available time" is the time frame in which a user or worker can perform a particular activity.

[1664] "Weather conditions" is information that indicates the weather conditions in a particular area.

[1665] "Past search history" is a record of searches previously performed by a user.

[1666] "Behavioral history" is a record of the actions and work a worker has performed in the past.

[1667] "Mood state" is information that indicates the user's current state of mind or emotions.

[1668] A "generative model" is a program that uses specific algorithms and machine learning techniques to automatically generate materials such as short videos.

[1669] "Tourist spots" are tourist destinations and famous places that users can visit.

[1670] A "work pattern" is a combination of procedures and movements that allow workers to perform work efficiently.

[1671] To specifically implement this invention, it is necessary to build a system including a user terminal, a server, and various communication means. The purpose of this system is to make it easier for users to search for tourist spots and workers' work patterns and to visually understand their attractiveness and efficiency.

[1672] User terminal operation

[1673] 1. Displaying the input form

[1674] The user terminal displays a form that allows the user to enter search criteria. For example, in the case of tourist spots, users can enter the desired time, location, and specific preferences (such as places rich in nature or historical sites) for visiting. In the case of a logistics center, workers can enter their work details and desired conditions (such as working hours and break times).

[1675] 2. Obtaining location information

[1676] The user terminal obtains the user's current location information. This is done using the terminal's built-in GPS function. The logistics center uses the built-in GPS function or other location tracking technology.

[1677] 3. Data Transmission

[1678] The user terminal transmits the search conditions entered by the user and the acquired location information to the server.

[1679] Server Operation

[1680] 1. Data Reception

[1681] The server receives the search conditions and location information sent from the user terminal.

[1682] 2. Database Matching

[1683] The server compares the received data with a database of tourist spots and work patterns, and selects tourist spots and work patterns that match the user's current location and search criteria.

[1684] 3. Filtering tourist attractions and work patterns

[1685] The server filters the extracted tourist spots and work patterns based on the user's hobbies and preferences and the worker's past behavioral history, thereby ranking the tourist spots and work patterns that are most suitable for the user and worker.

[1686] 4. Short video generation

[1687] The server uses the generative AI model to generate short videos for each selected tourist spot and work pattern. Each video is designed to convey the characteristics and appeal of the tourist spot and the efficiency of the work pattern in a short time of around 30 seconds.

[1688] 5. Data Transmission

[1689] The server transmits the generated short video and ranking information to the user terminal.

[1690] User terminal operation (continued)

[1691] 1. Data Reception

[1692] The user terminal receives the short video and ranking information transmitted from the server.

[1693] 2. Video playback and information display

[1694] The user device plays the received short video and displays ranking information for tourist attractions and work patterns, allowing users and workers to visually understand the appeal of each tourist attraction and the efficiency of their work patterns.

[1695] 3. Selection of tourist spots and work patterns

[1696] Based on the displayed information, users can select tourist spots they want to visit and work patterns they want to perform. This information helps users to plan their sightseeing trips comfortably and efficiently, and also contributes to workers' efficient work.

[1697] Specific examples

[1698] User Behavior

[1699] For example, if a user is looking for "tourist spots that can be enjoyed within an hour around Shinjuku Station," the user enters "1 hour" into the input form on the device and confirms that the current location is Shinjuku Station. At a logistics center, a worker enters "I would like to check the inventory in Area A."

[1700] Device behavior

[1701] The device uses its GPS function to obtain the user's location information and sends the conditions "Shinjuku Station" and "1 hour" to the server. The logistics center obtains the worker's current location and sends "Area A" and "Work content" to the server.

[1702] Server Operation

[1703] After receiving the data, the server extracts tourist spots that can be reached and returned from Shinjuku Station within an hour from the database. It also considers past search history and the user's hobbies (e.g., a love of nature) to select and rank the tourist spots. The logistics center extracts and ranks efficient work patterns.

[1704] User terminal operation (continued)

[1705] The device plays the received short video and displays ranking information to the user, allowing the user to select tourist spots they want to visit. Similarly, workers can visually understand and select efficient work patterns.

[1706] Example prompts for generative AI models

[1707] "For a worker who is currently in area C and wants to perform picking work in area A, please generate a 30-second guidance video that includes an efficient route and important points to note."

[1708] The system allows users to intuitively and efficiently find tourist spots and helps workers identify optimal work patterns.

[1709] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1710] Step 1:

[1711] Displaying the input form

[1712] The user terminal displays a form that allows the user to enter search criteria. The user enters the desired time, location, and specific preferences (such as a place rich in nature or a historical site) as search criteria. At the logistics center, workers enter their work details and desired conditions (such as working hours and break times).

[1713] Input: The time, place, and specific wishes you want to visit are entered by the user.

[1714] Output: User input information (search criteria)

[1715] Step 2:

[1716] Obtaining location information

[1717] The user device obtains the user's current location information using the device's built-in GPS function. In the logistics center, the location information of workers is obtained using the built-in GPS function or other location tracking technology.

[1718] Input: Data from the device's built-in GPS function or location tracking technology

[1719] Output: Current location information obtained

[1720] Step 3:

[1721] Data transmission

[1722] The user terminal transmits the search conditions entered by the user and the acquired location information to the server.

[1723] Input: User input information (search criteria), current location information obtained

[1724] Output: Data sent to the server

[1725] Step 4:

[1726] Data reception

[1727] The server receives the search conditions and location information sent from the user terminal.

[1728] Input: Search criteria and location information sent from the user device

[1729] Output: Received search conditions, location information

[1730] Step 5:

[1731] Database collation

[1732] The server compares the received data with a database of tourist spots and work patterns, and searches for tourist spots and work patterns using the search criteria entered by the user and the current location information.

[1733] Input: Received search conditions, location information

[1734] Output: Extracted tourist spots or work patterns

[1735] Step 6:

[1736] Filtering tourist attractions and work patterns

[1737] The server filters the extracted tourist spots and work patterns based on the user's hobbies and preferences and the worker's past behavioral history. "User hobbies and preferences" are determined based on past search history and behavioral history, and "worker efficiency" is determined based on behavioral history and time measurement data.

[1738] Input: Extracted tourist spots or work patterns, user's hobby and preference information, worker's past behavior history information

[1739] Output: Ranked tourist spots or activity patterns

[1740] Step 7:

[1741] Short video generation

[1742] The server uses a generative AI model to generate short videos for each selected tourist spot and work pattern. Each video is short, about 30 seconds long, and conveys the characteristics and appeal of the tourist spot and the efficiency of the work pattern. A prompt sentence is input into the generative AI model to generate the video.

[1743] Input: Prompt sentences such as "Description and characteristics of tourist spots" or "Work procedures and efficiency of workers"

[1744] Output: Generated short video

[1745] Step 8:

[1746] Data transmission

[1747] The server transmits the generated short video and ranking information to the user terminal.

[1748] Input: Generated short videos, ranking information

[1749] Output: Data sent to the user terminal

[1750] Step 9:

[1751] Data reception

[1752] The user terminal receives the short video and ranking information transmitted from the server.

[1753] Input: Short video sent from the server, ranking information

[1754] Output: Received short videos and ranking information

[1755] Step 10:

[1756] Video playback and information display

[1757] The user device plays the received short video and displays ranking information for tourist spots and work patterns, allowing users and workers to visually understand the appeal of each tourist spot and the efficiency of their work patterns.

[1758] Input: Received short video, ranking information

[1759] Output: Short video played, ranking information displayed

[1760] Step 11:

[1761] Selection of tourist spots and work patterns

[1762] Based on the displayed information, users can select tourist spots they want to visit and work patterns they want to perform. This information helps users to plan their sightseeing trips comfortably and efficiently, and also contributes to workers' efficient work.

[1763] Input: Displayed ranking information, played short video

[1764] Output: Selected tourist spots or work patterns

[1765] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1766] To specifically implement the present invention, it is necessary to build a system that includes a user terminal, a server, an emotion engine, and various communication means. The purpose of this system is to allow users to search for tourist spots and visually understand their attractions, while also taking into account the user's emotional state to make more appropriate recommendations. The program processing of this system is as follows.

[1767] User terminal operation

[1768] 1. Displaying the input form

[1769] The user terminal displays a form for the user to input search criteria, such as the time, location, and specific preferences (e.g., natural beauty, historical sites, etc.) that the user wants to visit.

[1770] 2. Obtaining location information

[1771] The user device acquires current location information using the GPS function and stores the acquired location information in its internal memory.

[1772] 3. Operation of the Emotion Engine

[1773] The user device uses a built-in emotion engine to recognize the user's emotional state from their voice and facial expressions. The emotion engine determines the user's current emotional state through analysis of the user's voice tone and facial expressions.

[1774] 4. Data Transmission

[1775] The user terminal transmits to the server the search conditions entered by the user, the acquired location information, and the emotional state recognized by the emotion engine.

[1776] Server Operation

[1777] 1. Data Reception

[1778] The server receives the search conditions, location information, and emotional state transmitted from the user terminal.

[1779] 2. Database Matching

[1780] The server compares the received search criteria, location information, and emotional state with a database of tourist spots. For example, it extracts relevant tourist spots based on the criteria "within one hour from Shinjuku Station" and "places where you can relax."

[1781] 3. Filtering tourist attractions

[1782] The server filters the extracted tourist spots based on the user's hobbies, preferences, past behavioral history, and emotional state, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[1783] 4. Short video generation

[1784] The server uses a generative AI model to generate short videos for each ranked tourist spot, each of which is about 30 seconds long and introduces the main attractions of the tourist spot.

[1785] 5. Data Transmission

[1786] The server transmits the generated short video and ranking information to the user terminal.

[1787] User terminal operation (continued)

[1788] 1. Data Reception

[1789] The user terminal receives the short video and ranking information transmitted from the server.

[1790] 2. Video playback and information display

[1791] The user device plays the received short video for the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[1792] 3. Selecting tourist spots

[1793] The user can select the tourist spots they want to visit based on the displayed information, and the selected information can be used to help the user plan their trip.

[1794] Specific examples

[1795] User Behavior

[1796] For example, suppose a user is looking for "a relaxing tourist spot that can be enjoyed within an hour around Shinjuku Station." The user enters "1 hour" and "relaxation" into the input form on the device and confirms that the current location is Shinjuku Station.

[1797] Device behavior

[1798] The device uses its GPS function to obtain the user's location information, and then uses an emotion engine to analyze the user's current emotional state from their voice and facial expressions, and sends the results to the server.

[1799] Server Operation

[1800] After receiving the data, the server extracts tourist spots that can be reached and returned from Shinjuku Station within an hour from the database. Furthermore, taking into account the emotional state of "places where you can relax," it selects and ranks Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street in Harajuku. It then generates short videos showcasing the attractions of these tourist spots and sends them to the user's device.

[1801] User terminal operation (continued)

[1802] The device plays the received short video and displays ranking information to the user, allowing the user to select Meiji Jingu.

[1803] In this way, the system embodying the present invention efficiently provides optimal tourist spots, taking into account the user's current emotional state.

[1804] The processing flow will be explained below.

[1805] User terminal processing

[1806] Step 1:

[1807] The user inputs search criteria (for example, the place they want to visit and the time they can be there) into the search form displayed on the terminal.

[1808] Step 2:

[1809] The device uses the built-in GPS function to obtain current location information, which is then stored in the internal memory.

[1810] Step 3:

[1811] The device uses a built-in emotion engine to recognize the user's emotional state from their voice and facial expressions. The emotion engine determines the user's current emotional state (e.g., excitement, relaxation, stress, etc.) through analysis of the user's voice tone and facial expressions.

[1812] Step 4:

[1813] The terminal transmits the search criteria entered by the user, the acquired location information, and the emotional state recognized by the emotion engine to the server.

[1814] Server Processing

[1815] Step 5:

[1816] The server receives the search conditions, location information, and emotional state transmitted from the user terminal.

[1817] Step 6:

[1818] The server compares the received search criteria, location information, and emotional state with a database of tourist spots. For example, it extracts relevant tourist spots based on the criteria "within one hour from Shinjuku Station" and "places where you can relax."

[1819] Step 7:

[1820] The server then filters the extracted tourist spots based on the user's preferences, past activities, and emotional state, prioritizing spots that meet criteria such as "places rich in nature" or "quiet places."

[1821] Step 8:

[1822] The server uses a generative AI model to generate short videos for each of the filtered tourist spots, each of which is about 30 seconds long and introduces the main attractions of the tourist spot.

[1823] Step 9:

[1824] The server transmits the generated short video and ranking information to the user terminal.

[1825] User terminal processing (continued)

[1826] Step 10:

[1827] The terminal receives the short video and ranking information transmitted from the server.

[1828] Step 11:

[1829] The device plays the received short video to the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[1830] Step 12:

[1831] The user selects tourist spots that they would like to visit based on the provided information, and the selected information can be used to help the user plan their trip.

[1832] Specific examples

[1833] For example, suppose a user is searching for "a relaxing tourist spot that can be enjoyed within an hour around Shinjuku Station." The user enters "1 hour" and "relaxing" into the input form on the device and confirms that the current location is Shinjuku Station.

[1834] The device uses its GPS to obtain location information for Shinjuku Station, and then uses an emotion engine to analyze the user's current emotional state based on their voice and facial expressions, obtaining a "relaxed state" as the result. This information is then sent to the server.

[1835] The server receives this data and compares it with a database of tourist spots based on the emotional information, such as "within an hour from Shinjuku Station" and "a relaxing place." As a result, Meiji Jingu Shrine, Yoyogi Park, and Takeshita Street in Harajuku are selected and ranked. A 30-second short video introducing the attractions of the selected tourist spots is generated and sent to the user's device.

[1836] The user terminal plays the received short video and displays ranking information for Meiji Shrine, Yoyogi Park, and Takeshita Street in Harajuku to the user, who then decides to visit Meiji Shrine.

[1837] This series of steps allows users to efficiently find tourist spots that best suit their current emotional state.

[1838] Example 2

[1839] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1840] Conventional tourist spot recommendation systems suggest tourist spots based on the search criteria and location information entered by the user, but they are unable to make recommendations that take into account the user's current emotional state, making it difficult to fully meet the user's needs.In addition, there is a lack of a means to efficiently convey the appeal of the suggested tourist spots, which often leaves users unsure which one to choose.

[1841] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1842] In this invention, the server includes means for receiving search criteria entered from a user terminal, means for acquiring current location information related to the search criteria, means for acquiring and analyzing emotional data such as voice and facial expression, means for extracting related tourist spots from a database based on the search criteria, location information, and emotional data, means for ranking the extracted tourist spots based on the user's hobbies, preferences, and emotional state, means for using a generative AI model to generate short videos for the ranked tourist spots, and means for transmitting the generated short videos and ranking information to the user terminal. This makes it possible to efficiently provide the user with the most suitable tourist spots while taking into account the user's current emotional state, and to visually convey their appeal.

[1843] A "user terminal" is an electronic device through which a user inputs search criteria and receives search results, and includes a smartphone, tablet, or personal computer.

[1844] "Search conditions" are information that a user inputs as conditions for the tourist spot they would like to visit, and include the time and place they would like to visit, specific requests, etc.

[1845] "Location information" is geographical data that indicates where the user is currently located, obtained using the GPS function of the user terminal.

[1846] "Emotion data" is information about the user's emotional state obtained by analyzing the user's voice, facial expressions, and the like.

[1847] A "database" is a data storage that accumulates information about tourist spots, and allows for the collation and extraction of information.

[1848] "Ranking" is the process of prioritizing and organizing the extracted tourist spots based on the user's hobbies, preferences, and emotional state.

[1849] "Generative AI model" is an abbreviation for generative artificial intelligence model, and refers to the AI ​​technology used to generate short videos and the like in this invention.

[1850] "Short videos" are short video contents created to introduce the main attractions of tourist spots in a short amount of time.

[1851] A "prompt" is an instruction given to a generative AI model, which indicates the specific content that the model needs to generate an appropriate video.

[1852] MODE FOR CARRYING OUT THE INVENTION

[1853] To specifically implement the present invention, it is necessary to build a system including a user terminal, a server, an emotion engine, and various communication means. The purpose of this system is to enable users to search for tourist spots and visually understand their attractions, while also providing more appropriate recommendations that take into account the user's emotional state.

[1854] Hardware and software used

[1855] User devices: smartphones, tablets, personal computers, etc.

[1856] Server: A high-performance computer that receives, processes, and transmits data.

[1857] Emotion Engine: Software for voice tone and facial expression analysis

[1858] Generative AI models: Generative artificial intelligence models, such as OpenAI's GPT-4

[1859] Program processing explanation

[1860] User terminal

[1861] 1. Displaying the input form

[1862] The user terminal displays a form for the user to input search criteria, such as the time, location, and specific preferences (e.g., natural beauty, historical sites, etc.) that the user wants to visit.

[1863] 2. Obtaining location information

[1864] The user device acquires current location information using the built-in GPS function and stores the acquired location information in its internal memory.

[1865] 3. Operation of the Emotion Engine

[1866] The user device uses a built-in emotion engine to recognize the user's emotional state from their voice and facial expressions. The emotion engine determines the user's current emotional state through analysis of the user's voice tone and facial expressions.

[1867] 4. Data Transmission

[1868] The user terminal transmits to the server the search conditions entered by the user, the acquired location information, and the emotional state recognized by the emotion engine.

[1869] server

[1870] 1. Data Reception

[1871] The server receives the search conditions, location information, and emotional state transmitted from the user terminal.

[1872] 2. Database Matching

[1873] The server searches a database of tourist attractions based on the received search criteria, location information, and emotional state. For example, it extracts relevant tourist attractions based on the criteria "within an hour from the city center" and "places where you can relax."

[1874] 3. Filtering tourist attractions

[1875] The server filters the extracted tourist spots based on the user's hobbies, preferences, past behavioral history, and emotional state, and then organizes the tourist spots that are most suitable for the user in a ranking format.

[1876] 4. Short video generation

[1877] The server uses a generative AI model to generate a short video for each ranked tourist spot. Each video introduces the main attractions of the tourist spot in a short time of about 30 seconds. The following sentence is used as a prompt for the generative AI model:

[1878] "Write a short 30-second video script about a relaxing tourist spot within an hour of the city center. The spot is a famous park."

[1879] 5. Data Transmission

[1880] The server transmits the generated short video and ranking information to the user terminal.

[1881] User terminal

[1882] 1. Data Reception

[1883] The user terminal receives the short video and ranking information transmitted from the server.

[1884] 2. Video playback and information display

[1885] The user device plays the received short video for the user and displays tourist spot ranking information, allowing the user to visually understand the appeal of each tourist spot.

[1886] 3. Selecting tourist spots

[1887] The user can select the tourist spots they want to visit based on the displayed information, and this selection information can be used to help the user plan their trip.

[1888] Specific examples

[1889] User Behavior

[1890] For example, suppose a user is looking for "a relaxing and enjoyable tourist spot within an hour of the city center." The user enters "1 hour" and "relaxing" into the input form on the device and confirms that the current location is the city center.

[1891] Device behavior

[1892] The device uses its GPS function to obtain the user's location information, and then uses an emotion engine to analyze the user's current emotional state from their voice and facial expressions, and sends the results to the server.

[1893] Server Operation

[1894] After receiving the data, the server extracts tourist spots that can be reached and returned from the city center within an hour from the database. It then selects and ranks several suitable locations, taking into account the emotional state of "a place to relax." It then generates short videos showcasing the attractions of these tourist spots and sends them to the user's device.

[1895] User terminal operation

[1896] The device plays the received short video and displays ranking information to the user, allowing the user to select the tourist spots they would like to visit.

[1897] In this way, the system embodying the present invention efficiently provides optimal tourist spots, taking into account the user's current emotional state.

[1898] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1899] Step 1:

[1900] The user terminal displays a search condition input form.

[1901] Input: User's tourist attraction preferences (time, location, specific preferences, etc.)

[1902] Output: The search criteria entered by the user

[1903] How it works: Using a drawing library or UI component, a search field is displayed on the form screen, which the user can follow to enter specific search criteria.

[1904] Step 2:

[1905] The user terminal acquires location information using the GPS function.

[1906] Input: Current GPS signal

[1907] Output: Current location of the user (latitude and longitude information)

[1908] How it works: Calls the device's location service to obtain the user's real-time location, which is then stored in the device's internal memory.

[1909] Step 3:

[1910] The user terminal uses an emotion engine to recognize the user's emotional state.

[1911] Input: User's voice data, facial expression images

[1912] Output: User's emotional state (e.g., relaxed, excited, tired, etc.)

[1913] How it works: Analyzes the tone of voice input and performs facial expression analysis using image data acquired from the camera, thereby determining the user's emotions and treating their emotional state as data.

[1914] Step 4:

[1915] The user terminal transmits search conditions, location information, and emotional state to the server.

[1916] Input: Search criteria, location, emotional state

[1917] Output: Data packet to send to the server

[1918] What it does: It formats the data to be sent into packets and sends them over the network to the server.

[1919] Step 5:

[1920] The server receives the data sent from the user terminal.

[1921] Input: Search criteria, location information, and emotional state sent from the user device

[1922] Output: Data used for database matching

[1923] Operation: Analyzes the received data, converts it into the required format and stores it in the internal memory.

[1924] Step 6:

[1925] The server checks the database and extracts relevant tourist spots.

[1926] Input: Search criteria, location, emotional state

[1927] Output: A list of matched attractions

[1928] Behavior: Generates a database query and performs a search based on the received data. Returns a list of matching attractions.

[1929] Step 7:

[1930] The server filters and ranks the extracted tourist attractions.

[1931] Input: A list of matched tourist spots, the user's interests and preferences, and their emotional state

[1932] Output: A list of ranked tourist attractions

[1933] How it works: Based on the user's recorded hobbies, preferences, past behavioral history, and current emotional state, the system calculates and ranks the relevance of each tourist spot.

[1934] Step 8:

[1935] The server generates the short video using the generative AI model.

[1936] Input: A list of ranked tourist attractions, a prompt

[1937] Output: Short videos about each tourist attraction

[1938] How it works: The model is fed with preconditions and prompts, and generates a 30-second video introducing the main attractions of each tourist spot.

[1939] Step 9:

[1940] The server transmits the short video and ranking information to the user terminal.

[1941] Input: Generated short videos, ranking information

[1942] Output: Data sent to the user's terminal

[1943] Operation: The generated video and ranking information are packaged into a data packet and sent to the user's terminal via the network.

[1944] Step 10:

[1945] The user terminal receives the short video and ranking information transmitted from the server.

[1946] Input: Short video and ranking information sent from the server

[1947] Output: Data to display

[1948] Operation: Receives data packets, parses them and stores them in internal memory.

[1949] Step 11:

[1950] The user terminal plays the received short video and displays the ranking information.

[1951] Input: Saved short videos, ranking information

[1952] Output: Video and information displayed to the user

[1953] How it works: Displays videos through a player and uses UI components to display ranking information on the screen.

[1954] Step 12:

[1955] The user selects the tourist spot they would like to visit based on the displayed information.

[1956] Input: Viewed video and ranking information

[1957] Output: Selected tourist spots

[1958] How it works: The user clicks or taps on a tourist attraction to select it, and that information is reflected in the user's itinerary.

[1959] (Application example 2)

[1960] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1961] Conventional tourist spot recommendation systems are unable to consider the user's current emotional state, making it difficult to suggest appropriate tourist spots. Furthermore, due to a lack of visually understandable information provision methods, users are unable to easily grasp the appeal of tourist spots. Furthermore, when applying similar technology to virtual stores, it has not been possible to achieve advanced personalization, such as product recommendations based on the user's emotional state.

[1962] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1963] In this invention, the server includes means for receiving search criteria entered from a user terminal, means for acquiring current location information, means for utilizing an emotion engine that recognizes the emotional state of the user from their voice and facial expression, means for extracting relevant tourist spots from a database based on the search criteria and location information, means for ranking the extracted tourist spots based on the user's hobbies, preferences, and emotional state, means for generating short videos of the ranked tourist spots, and means for transmitting the generated short videos and ranking information to the user terminal. This enables advanced personalization that takes into account the user's current emotional state and makes it possible to provide information that is visually easy to understand.

[1964] A "user terminal" is an electronic device that allows a user to input information or view received information, and includes smartphones, tablets, PCs, etc.

[1965] "Location information" is data indicating the user's current location, and is geographical location data obtained by a GPS function or the like.

[1966] An "emotion engine" is software or hardware that recognizes the user's emotional state from their voice and facial expressions.

[1967] A "database" is a system for storing and managing related information, which can be searched and extracted based on specific conditions.

[1968] "Ranking" refers to prioritizing and ordering items based on specific criteria.

[1969] A "short video" is a video that introduces specific content in a short period of time, usually about 30 seconds long.

[1970] A "generative model" is an algorithm or system for generating new content based on specific data or conditions.

[1971] A "server" is a computer system that provides and processes information over a network, and plays a central role in sending, receiving, and processing data.

[1972] To specifically implement the present invention, it is necessary to build the following system, which includes a user terminal, a server, an emotion engine, and various communication means.

[1973] User terminal configuration

[1974] 1. Displaying the input form

[1975] The user terminal uses HTML and JavaScript to display a form in which the user can enter product categories and search conditions. Through this form, the user can enter the products they want to purchase and their desired conditions.

[1976] 2. Obtaining location information

[1977] The user device obtains the current location information using the JavaScript navigator.geolocation API, which is used for subsequent processing.

[1978] 3. Operation of the Emotion Engine

[1979] The user device uses Google Cloud's Vision API and Speech-to-Text API to recognize the user's emotional state from their voice and facial expressions, allowing it to analyze the user's current emotional state.

[1980] 4. Data Transmission

[1981] The user device sends the search criteria entered by the user, the acquired location information, and the emotional state recognized by the emotion engine to the server via a POST request. The HTTPS protocol is used for communication with the server.

[1982] Server Configuration

[1983] 1. Data Reception

[1984] The server uses Node.js and Express to receive data sent from the user's device, including search criteria, location information, and emotional state.

[1985] 2. Database Matching

[1986] The server then compares the received data with the MongoDB database and extracts relevant information, specifically, extracting appropriate products from the product database, taking into account the user's emotional state and search criteria.

[1987] 3. Ranking

[1988] The server ranks the extracted products based on the user's tastes and preferences and emotional state, allowing it to prioritize the most suitable products for the user.

[1989] 4. Short video generation

[1990] The server uses FFmpeg and a generative AI model (e.g., OpenAI's GPT-3) to generate a short video for each ranked product, approximately 30 seconds long, showcasing the main appeal of each product.

[1991] 5. Data Transmission

[1992] The server transmits the generated short videos and ranking information to the user's terminal, allowing the user to visually understand the appeal of each product.

[1993] Specific examples

[1994] For example, when a user searches for "fashion items" using a virtual store app, the system works as follows: The user enters their search criteria and current location information into their device, and the emotion engine analyzes their emotional state. Based on this information, the server extracts and ranks products from the database that best fit the user's emotional state. Finally, a short video is generated using FFmpeg and a generative AI model and sent to the user's device.

[1995] Prompt Sentence Examples

[1996] If the user is relaxed, recommend some relaxed outfits that suit the user's taste from the latest fashion items. Example: Introduce five fashion items each in a short 30-second video.

[1997] In this way, highly accurate product recommendations that take into account the user's emotional state and easy-to-understand information can be provided.

[1998] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1999] Step 1:

[2000] The user inputs product categories and search conditions into an input form on the user's terminal. At this time, the user's input is sent as a form and organized as input data.

[2001] Step 2:

[2002] The user device obtains the current location information using the JavaScript navigator.geolocation API. The location information is obtained as latitude and longitude data and added to the input data.

[2003] Step 3:

[2004] The user device uses Google Cloud's Vision API and Speech-to-Text API to analyze voice and facial expressions to recognize the user's emotional state, and the recognition results are included in the input data.

[2005] Step 4:

[2006] The user device sends the entered search criteria, the acquired location information, and the recognized emotional state as data to the server in JSON format.

[2007] Step 5:

[2008] The server uses Node.js and Express to receive data sent from the user's device, including search criteria, location information, and emotional state.

[2009] Step 6:

[2010] The server then matches the received data with a MongoDB database to extract relevant product data, taking into account the emotional state of the user and providing filtered product data.

[2011] Step 7:

[2012] The server inputs the extracted product data into a generative AI model (OpenAI's GPT-3) to generate prompts that introduce the appeal of each product. These prompts then become the input data for generating short videos.

[2013] Step 8:

[2014] The server uses FFmpeg to generate a short video for each product based on the prompt text obtained from the generative AI model. The generated short video is about 30 seconds long and introduces the main appeal of the product.

[2015] Step 9:

[2016] The server transmits the generated short video and ranking information to the user terminal, and the transmitted data is formatted to be displayed on the user terminal.

[2017] Step 10:

[2018] The user terminal receives the short videos and ranking information sent from the server, and the received data is visually displayed to the user.

[2019] Step 11:

[2020] The user checks the displayed short video and ranking information and selects the product they wish to purchase. The selected product information is used for the next process.

[2021] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2022] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2023] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2024] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2025] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2026] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2027] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2028] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2029] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2030] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2031] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2032] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2033] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2034] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2035] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2036] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2037] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2038] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2039] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2040] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2041] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2042] The following is further disclosed regarding the above embodiment.

[2043] (Claim 1)

[2044] means for receiving search conditions input from a user terminal;

[2045] means for acquiring current location information related to the search criteria;

[2046] A means for extracting related tourist spots from a database based on the search criteria and location information;

[2047] A means for ranking the extracted tourist spots based on the user's interests and preferences;

[2048] means for generating short videos about the ranked tourist attractions;

[2049] means for transmitting the generated short video and ranking information to a user terminal;

[2050] A system including:

[2051] (Claim 2)

[2052] 2. The system according to claim 1, wherein the search conditions include available time, weather conditions, the user's past search history, and the user's current emotional state.

[2053] (Claim 3)

[2054] The system according to claim 1, wherein the means for generating short videos includes a generative model that generates videos that introduce the main attractions of each tourist spot.

[2055] "Example 1"

[2056] (Claim 1)

[2057] means for receiving search conditions input from a user terminal;

[2058] means for acquiring current location information related to the search criteria;

[2059] A means for extracting related tourist spots from a database based on the search criteria and location information;

[2060] A means for ranking the extracted tourist spots based on the user's interests and preferences;

[2061] a means for generating short videos about the ranked tourist attractions using a generative AI model; and

[2062] means for transmitting the generated short video and ranking information to a user terminal;

[2063] A system including:

[2064] (Claim 2)

[2065] 2. The system according to claim 1, wherein the search conditions include available time, weather conditions, the user's past search history, and the user's current emotional state.

[2066] (Claim 3)

[2067] The system according to claim 1, wherein the means for generating short videos includes a generative model that generates videos that introduce the main attractions of each tourist spot.

[2068] "Application Example 1"

[2069] (Claim 1)

[2070] means for receiving search conditions input from a user terminal;

[2071] means for acquiring current location information related to the search criteria;

[2072] A means for extracting related tourist spots and work patterns from a database based on the search conditions and location information;

[2073] A means for ranking the extracted tourist spots and work patterns based on the user's preferences and work efficiency;

[2074] means for generating short videos about the ranked tourist spots and work patterns;

[2075] means for transmitting the generated short video and ranking information to a user terminal;

[2076] A system including:

[2077] (Claim 2)

[2078] The system of claim 1, wherein the search conditions include available time, weather conditions, the user's past search history, the worker's past activity history, the user's current emotional state, and the worker's current location information.

[2079] (Claim 3)

[2080] The system of claim 1, characterized in that the means for generating the short video includes a generative model that generates videos with content that introduces the main attractions of tourist spots and work patterns and efficient work procedures.

[2081] "Example 2: Combining Emotion Engines"

[2082] (Claim 1)

[2083] means for receiving search conditions input from a user terminal;

[2084] means for obtaining current location information related to the search criteria;

[2085] A means of acquiring and analyzing emotional data such as voice and facial expressions,

[2086] means for extracting related tourist spots from a database based on the search criteria, location information, and emotion data;

[2087] A means for ranking the extracted tourist spots based on the user's tastes, preferences, and emotional state;

[2088] using a generative AI model to generate short videos about the ranked tourist attractions;

[2089] means for transmitting the generated short video and ranking information to a user terminal;

[2090] A system including:

[2091] (Claim 2)

[2092] 2. The system according to claim 1, wherein the search conditions include available time for activities, weather conditions, the user's past activity history, and the user's current emotional state.

[2093] (Claim 3)

[2094] The system described in claim 1, characterized in that the means for generating the short video inputs a prompt sentence and the generation AI model generates a video introducing the main attractions of each tourist spot.

[2095] "Application example 2 when combining emotion engines"

[2096] (Claim 1)

[2097] means for receiving search conditions input from a user terminal;

[2098] means for acquiring current location information related to the search criteria;

[2099] A means for utilizing an emotion engine that recognizes the user's emotional state from their voice and facial expressions;

[2100] A means for extracting related tourist spots from a database based on the search criteria and location information;

[2101] A means for ranking the extracted tourist spots based on the user's tastes, preferences, and emotional state;

[2102] means for generating short videos about the ranked tourist attractions;

[2103] means for transmitting the generated short video and ranking information to a user terminal;

[2104] A system including:

[2105] (Claim 2)

[2106] 2. The system according to claim 1, wherein the search conditions include available time, weather conditions, the user's past search history, and the user's current emotional state.

[2107] (Claim 3)

[2108] The system according to claim 1, wherein the means for generating short videos includes a generative model that generates videos that introduce the main attractions of each tourist spot. [Explanation of symbols]

[2109] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving search conditions input from a user terminal; means for acquiring current location information related to the search criteria; A means for extracting related tourist spots from a database based on the search criteria and location information; A means for ranking the extracted tourist spots based on the user's interests and preferences; means for generating short videos about the ranked tourist attractions; means for transmitting the generated short video and ranking information to a user terminal; A system including:

2. 2. The system according to claim 1, wherein the search conditions include available time, weather conditions, the user's past search history, and the user's current emotional state.

3. 2. The system according to claim 1, wherein the means for generating short videos includes a generative model for generating videos that introduce the main attractions of each tourist spot.

Images