system

Abstract

Provide a system. A means for inputting regional culture information; means for transmitting the input information together with metadata to a server; a means for the server to store the transmitted information in a database and perform classification; means for a user to search and view the stored information; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Regional cultures and customs are highly diverse, each with its own unique characteristics. However, with modern globalization and changing lifestyles, these traditions and cultures are gradually being lost. In particular, regional food cultures, traditional games, and events are deeply connected to the region's unique history, making them difficult to restore once lost. Therefore, it is extremely important to record these cultures and pass them on to future generations. However, existing systems require a method for systematically collecting, preserving, and easily accessing this information. Against this background, the problem to be solved by this invention is to provide a system that effectively records and preserves regional cultures, food, and games, preventing their erosion. [Means for solving the problem]

[0005] The present invention provides a system including a means for inputting information about regional culture, a means for transmitting the input information together with metadata to a server, a means for the server to store and classify the transmitted information in a database, and a means for a user to search for and view the stored information. In this system, a user inputs video data or text data about regional culture, and the server appropriately classifies the received data and stores it in the database. The system also allows a user to search for information based on specific keywords or categories, and the server provides the relevant information. This allows regional culture to be systematically preserved and easily accessible for anyone to learn about, thereby preventing traditional culture from fading away.

[0006] "Regional culture" includes the history, customs, practices, and unique skills and knowledge that are passed down in a particular region.

[0007] "Means for inputting information" refers to providing users with the ability to input data about local culture using their own devices (such as smartphones or PCs).

[0008] "Metadata" is additional information used to describe and categorize entered information, including titles, descriptions, categories, tags, and the like.

[0009] A "server" is a remote computer system that receives, stores, and manages information submitted by users.

[0010] A "database" is a system for organizing information stored on a server and efficiently searching and managing it.

[0011] The "means for classifying" is a function that automatically classifies the information received by the server based on categories and tags, and stores it in an appropriate database location.

[0012] "Search means" is a function that allows a user to retrieve stored information by entering a specific keyword or category.

[0013] The "browsing means" is a function that allows the user to visually check the information acquired through a search. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention relates to a system for recording, preserving, and preventing the erosion of information about local culture, food, and games (hand games and songs). This system has a series of functions that allow users to input information, send it to a server, and the server classifies, preserves, and provides the information. Specific embodiments for implementing the present invention are described below.

[0036] System Overview

[0037] This system consists of three main components: a user terminal, a server, and a database. Users use their terminals to input information about local culture and send it to the server. The server then appropriately categorizes the received information and stores it in the database. Users can also search for information using specific keywords or categories and view the information retrieved from the database.

[0038] Program processing

[0039] When a user enters information and sends it to the server

[0040] Subject: User and Device

[0041] 1. Users input information about their local culture, events, and games on their devices. For example, they can upload a recipe for a particular local dish or a video of a traditional hand game.

[0042] 2. The device converts the input information into a video file or text file and adds metadata such as title, description, category, and tags.

[0043] 3. The device makes a request to send the converted information and metadata to the server.

[0044] When the server stores and organizes information

[0045] Subject: Server

[0046] 1. The server receives an upload request from the device and retrieves the submitted data and metadata.

[0047] 2. The server stores the received information in a database and categorizes it into appropriate categories and tags based on the metadata.

[0048] 3. The server indexes the newly stored information and makes it available for searching.

[0049] When a user searches or browses information

[0050] Subject: User, device, server

[0051] 1. The user searches from their device by specifying specific keywords or categories.

[0052] 2. The device sends the user's search query to the server.

[0053] 3. The server searches the database based on the search query, retrieves and organizes the relevant information.

[0054] 4. The server sends the search results to the device in an appropriate format.

[0055] 5. The user views the search results on their device and checks for more information.

[0056] Specific examples

[0057] For example, let's consider a case where a user searches for "traditional hand games." When the user enters "otedama" and presses the search button, the device sends this query to the server. The server retrieves video files and text information related to "otedama" from the database, organizes the results, and sends them back to the device. The user can view the results and learn how to play the traditional hand game "otedama."

[0058] summary

[0059] This system aims to prevent the erosion of culture by recording and storing local culture, food, and entertainment, and making that data easily searchable and viewable. The basic functions required for this are implemented between user devices and a server. This allows anyone to easily learn about local traditions and culture and pass them on to the next generation.

[0060] The processing flow will be explained below.

[0061] When a user uploads information

[0062] Step 1:

[0063] Users input information about local culture (e.g., video files or text information) from their own devices (smartphones, PCs, etc.).

[0064] Step 2:

[0065] The system sets metadata such as an appropriate title, description, category, and tag for the information entered by the user.

[0066] Step 3:

[0067] The user presses the "upload" button to issue a request to the terminal to transmit information.

[0068] Step 4:

[0069] The device verifies the information entered and creates a data package containing the necessary metadata.

[0070] Step 5:

[0071] The terminal requests that a data package be sent to the server.

[0072] When the server stores and organizes information

[0073] Step 6:

[0074] The server receives the data package sent from the terminal.

[0075] Step 7:

[0076] The server parses and separates the information and metadata from the received data packages.

[0077] Step 8:

[0078] The server prepares to store the parsed information in a database.

[0079] Step 9:

[0080] The server categorizes the information based on appropriate categories and tags and stores it in a database.

[0081] Step 10:

[0082] The server indexes the newly stored information and makes it searchable.

[0083] Step 11:

[0084] The server returns a confirmation message to the device that the information has been saved and classified.

[0085] When a user searches or browses information

[0086] Step 12:

[0087] A user enters a search query on their device, specifying specific keywords and categories.

[0088] Step 13:

[0089] The terminal analyzes the user's search query and sends a search request to the server.

[0090] Step 14:

[0091] The server searches the database based on the received search query and retrieves the relevant information.

[0092] Step 15:

[0093] The server properly organizes the acquired information and sends it to the terminal.

[0094] Step 16:

[0095] The user views the search results on their device and checks for more information.

[0096] When providing recipes or event information

[0097] Step 17:

[0098] A user requests information about a particular local event or dish.

[0099] Step 18:

[0100] The terminal analyzes the user's request and sends it to the server.

[0101] Step 19:

[0102] The server retrieves relevant information from the database, organizes it, and provides it to the user's terminal.

[0103] Step 20:

[0104] The user views the provided information on the device and deepens their understanding of the events and cuisine.

[0105] Through these processing steps, the system is able to effectively record and preserve local culture and provide it to users in an easily accessible form.

[0106] Example 1

[0107] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0108] Information about a region's unique culture, traditions, food, and pastimes is a valuable asset that should be passed on to future generations. However, if this information is not properly recorded and preserved, it is likely to fade away and be lost. Furthermore, current systems make it difficult to classify input information, making it difficult to search and view. Therefore, there is a need for a system that enables efficient recording, classification, search, and viewing of regional cultural information.

[0109] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0110] In this invention, the server includes means for inputting regional cultural information, means for transmitting the input information together with metadata, means for the server to store the transmitted information in a database and classify it based on the metadata, means for the server to create an index of the stored information and organize the information in a searchable form, means for a user to search for information based on specific keywords or categories, and means for the server to acquire, organize, and transmit the relevant information in an appropriate format to a terminal, and means for a user to view search results and check detailed information. This enables efficient recording, classification, search, and viewing of regional cultural information.

[0111] "Regional culture" is a general term for cultural elements such as events, food, and games that have traditionally been passed down in a particular region.

[0112] "Means for inputting information" refers to a device or interface that allows a user to input information about local culture in digital form.

[0113] "Metadata" refers to additional information such as title, description, category, and tag that is added to input information.

[0114] "Means for sending to the server" refers to the communications technology or protocol used to transfer digital information from the terminal to the server.

[0115] "Means of storing in a database" refers to the systems and technologies used by the server to permanently store the digital information it receives.

[0116] "Indexing means" refers to systems or techniques that generate tables of contents or indexes to facilitate efficient searching of stored digital information.

[0117] "Means of making information searchable" refers to database management and search engine technologies that enable quick and accurate responses to user search queries.

[0118] A "search query" refers to a keyword or phrase that a user enters when searching for information.

[0119] "Methods of organization" refers to techniques and processes for systematically sorting acquired information based on certain criteria.

[0120] "Means for transmitting to the terminal in an appropriate format" refers to the communication technology and protocols that allow the server to provide organized information to the user terminal in an appropriate format.

[0121] "Means for viewing search results" refers to the interface and display technology that allows users to visually check search results using a terminal.

[0122] This invention is a system that effectively records, stores, and easily searches and views information about local culture, food, and entertainment. This system sends information entered by users to a server, which then classifies and stores the information in a database, allowing users to easily search and view it.

[0123] System configuration

[0124] The system mainly consists of the following components:

[0125] 1. User Device

[0126] 2. Server

[0127] 3. Database

[0128] Entering information

[0129] Users input information about their local culture, events, and games into their devices. Specific examples include recipes for local dishes and videos of traditional hand games. The device saves this information as video or text files and adds metadata such as titles, descriptions, categories, and tags.

[0130] Sending data

[0131] The device uses a file upload function to send the converted information and metadata to the server, for example, by using an HTTP POST request to send the data to the server, which receives the request and stores the information and metadata in a database.

[0132] Data classification and storage

[0133] The server stores the received information in a database and categorizes it appropriately based on the metadata. Specifically, it stores the video file in storage and stores the metadata (title, description, category, tags, etc.) in the appropriate tables in the database.

[0134] Making the site searchable

[0135] The server then indexes the newly stored information and makes it searchable, which includes using a full-text search engine to generate the index.

[0136] Searching and browsing information

[0137] Users search for information by specifying specific keywords or categories on their devices. At that time, the device sends the user's search query to the server. The server searches the database based on the search query, retrieves and organizes the relevant information, and then sends it to the device in an appropriate format (for example, JSON format). Finally, the user can view the search results and check detailed information.

[0138] Example prompt

[0139] As a concrete example, let's consider the case where a user searches for "traditional hand games." The user enters "otedama" and presses the search button. The device sends this query to the server via an HTTP GET request. The server executes an SQL query, retrieves video files and text information related to "otedama" from the database, and returns the results organized in JSON format to the device. The device displays the search results in a list, and the user can view detailed information about "how to play otedama" in a pop-up display.

[0140] Example prompts to input to a generative AI model:

[0141] "I would like to know how to play otedama, a traditional Japanese hand game. What are the steps and what materials are needed?"

[0142] This system will enable efficient storage, classification, searching, and viewing of local cultural information, and is expected to help prevent culture from fading away.

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Step 1:

[0145] The user enters information

[0146] Description: Users input information about their local culture, events, and pastimes from their devices.

[0147] Input: The user enters free-form text information and video data into the input form.

[0148] Data processing / calculation: Text data is stored as is via the input device, and video data is stored in digital format.

[0149] Output: Input information temporarily stored on the device.

[0150] Specific operation: The user enters a description of a traditional hand game into a form, then takes a video of the game using their smartphone and uploads it.

[0151] Step 2:

[0152] Device adds metadata

[0153] Description: The device adds metadata to the information entered, such as title, description, category, and tags.

[0154] Input: Text and video data provided by the user in Step 1.

[0155] Data processing / calculation: The process of converting data into a storage format for databases and adding metadata.

[0156] Output: Information with metadata added.

[0157] Specific operation: For a video titled "How to play with beanbags," the device automatically sets the title "Beanbags," the description "Traditional Japanese hand game," the category "Game," and the tags "Hand game, beanbags."

[0158] Step 3:

[0159] The device sends data to the server

[0160] Description: The device sends the converted information and metadata to the server.

[0161] Input: Information with metadata.

[0162] Data processing / computation: Packaging data and generating transmission requests.

[0163] Output: The data sent to the server.

[0164] What it does: Sends data to the server as an HTTP POST request, including metadata in the data packet.

[0165] Step 4:

[0166] The server receives and stores the data

[0167] Description: The server receives upload requests from devices and stores the submitted data and metadata in a database.

[0168] Input: Data and metadata sent from the device.

[0169] Data processing / calculation: Inserting data into the database and saving it to storage.

[0170] Output: Information stored in a database.

[0171] Specific operation: The video file is stored in server-side storage, and the metadata is inserted into the appropriate table in the database.

[0172] Step 5:

[0173] The server categorizes and organizes the information

[0174] Description: The server categorizes information based on metadata, indexes newly stored information, and makes it searchable.

[0175] Input: Information and metadata stored in a database.

[0176] Data processing / computation: Classification and indexing of information using indexing algorithms.

[0177] Output: Indexed and searchable information.

[0178] Specific operation: Tags such as "hand games" and "beanbags" are added to the category "play," and an index is created using a full-text search engine.

[0179] Step 6:

[0180] The user enters a search query

[0181] Description: Users search for information from their devices by specifying specific keywords or categories.

[0182] Input: User-specified keywords and categories.

[0183] Data processing / calculation: Generating search queries entered by users.

[0184] Output: The search query sent from the device to the server.

[0185] Specific behavior: The user enters "Beanbag" into the search form and presses the search button.

[0186] Step 7:

[0187] The device sends the search query to the server

[0188] Description: The device sends the user's search query to the server.

[0189] Input: The search query specified by the user.

[0190] Data processing / computation: Generating and submitting search requests.

[0191] Output: The search query sent to the server.

[0192] Specific behavior: Sends a search query to the server via an HTTP GET request.

[0193] Step 8:

[0194] The server searches the database

[0195] Description: The server searches the database based on the search query, retrieves and organizes relevant information.

[0196] Input: The search query sent from the device.

[0197] Data processing / calculation: Execute SQL queries and extract relevant information from the database.

[0198] Output: Organized search results.

[0199] Specific operation: The server executes an SQL query to retrieve video files and text information related to "Beanbag."

[0200] Step 9:

[0201] The server sends the search results to the device.

[0202] Description: The server sends the search results in the appropriate format to the device.

[0203] Input: Organized search results.

[0204] Data processing / calculation: Packaging search results in an appropriate format such as JSON.

[0205] Output: Search results sent to your device.

[0206] Specific operation: The server returns the search results to the device in JSON format.

[0207] Step 10:

[0208] Your device will display the search results.

[0209] Description: The device displays the search results received from the server in the user interface.

[0210] Input: Search results sent from the server.

[0211] Data processing / calculation: Search results are displayed in list format, and detailed information about the items selected by the user is displayed in a pop-up.

[0212] Output: Search results displayed in the user interface.

[0213] Specific behavior: The device displays the search results in a list format, and the user checks detailed information about "How to play beanbag."

[0214] (Application example 1)

[0215] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0216] Information about regional culture easily fades with time, making it particularly difficult to pass on to younger generations. Furthermore, manually entering information is time-consuming, and the lack of a voice input method makes it difficult to use. The objective of this invention is to provide a system that efficiently collects regional cultural information and utilizes voice recognition technology to allow users to easily input, search, and view information.

[0217] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0218] In this invention, the server includes means for inputting regional culture information, means for transmitting the input information together with metadata to the server, means for the server to store the transmitted information in a database and classify it, means for a user to input regional culture information by voice using voice recognition technology, means for converting the voice-input information into text data, and means for a user to search and view the stored information. This allows users to utilize voice input to easily record information and pass on regional culture.

[0219] "Regional culture" refers to cultural characteristics including events, food, games, traditions, customs, and practices unique to a region.

[0220] "Means for inputting information" means the means by which a user can directly or indirectly input data into a system, including voice, text, video, or other formats.

[0221] "Means for sending to the server" refers to the communication means or protocol for converting the input information into a certain format and sending it to the server.

[0222] "Server" refers to a computer system that processes, stores, and appropriately manages received information.

[0223] A "database" refers to a structured collection of data that stores information and makes it easy to retrieve when needed.

[0224] A "means for classification" refers to a means for organizing input information based on specific criteria or tags.

[0225] "Speech recognition technology" refers to technology for converting voice data into text data.

[0226] The term "means for inputting regional culture information by voice" refers to a means for a user to input regional culture information into the system by voice.

[0227] "Means for converting into text data" refers to means for converting information input by voice into text format.

[0228] "Means for searching and viewing information" refers to means by which a user can search for stored information based on specific keywords or categories and display the results.

[0229] This invention is a system for efficiently collecting, recording, and storing information about local culture, food, entertainment, etc., and preventing it from fading away. The invention is composed of three main components: a user terminal, a server, and a database. Specific embodiments of the invention are described below.

[0230] Input information via user terminal

[0231] First, the user uses voice recognition technology to input information about local culture. Voice recognition software such as Google® Speech-to-Text API is installed on the user's device, and the voice data is converted into text data. For example, if the user says, "The local dish of this region is okonomiyaki," the content is converted into text data. At the same time, metadata such as title, description, category, and tags are also generated.

[0232] Data processing and storage by server

[0233] Text data and metadata generated on user devices are sent via the Internet to a server. The server receives this data and stores it in a database, where it is automatically categorized and organized based on the metadata. The server also creates an index of the newly stored information, making it instantly searchable.

[0234] Users search for and view information

[0235] Users can search for information by specifying specific keywords or categories. When a user enters a search query using a terminal, the query is sent to the server. The server searches for relevant information from the database and sends the results to the user's terminal. For example, if a user searches for "traditional hand games," the server will provide the user with video files and text information related to "beanbags."

[0236] Hardware and software used

[0237] The hardware used includes user devices (such as smartphones), servers, database servers, etc. The software used includes the Google Speech-to-Text API, a voice recognition technology, the HTTP communication protocol, and a database management system.

[0238] For example:

[0239] As users tour the factory, they can talk about local cultures, and their voices are recorded by their smartphones. A speech recognition system converts the speech into text, adds metadata, and sends it to a server, where it can be stored, categorized, and made searchable by other users if desired.

[0240] Prompt Sentence Examples

[0241] Here are some answers to the open-ended question that begins, "What is the local cuisine of your region?"

[0242] Example: Question: "What is the local dish of this region?"

[0243] Answer: "The local dish of this area is okonomiyaki."

[0244] As a result, the system of the present invention provides users with the ability to easily record, search, and view information about local culture, making it possible to prevent local culture from fading away.

[0245] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0246] Step 1:

[0247] The user inputs information about local culture into the device by voice. Using voice recognition technology, the user says, "The local dish of this region is okonomiyaki." The device collects this voice data. As a result, the voice data is obtained as input data.

[0248] Step 2:

[0249] Convert voice data into text data. The input voice data is converted into text data using the Google Speech-to-Text API installed on the device. The output of this process is the text data, "The local dish of this region is okonomiyaki."

[0250] Step 3:

[0251] Metadata is added to the text data. The device generates and adds metadata such as title, description, category, and tag to the converted text data. For example, metadata such as title "Local Cuisine," category "Meal," and tag "Okonomiyaki" are added. This process outputs data that combines the text data and metadata.

[0252] Step 4:

[0253] The text data with metadata is sent to the server. The terminal sends the generated data to the server using the HTTP communication protocol. In this step, data is sent from the user terminal to the server.

[0254] Step 5:

[0255] The server stores and classifies the received data. The server stores the received data in a database and automatically classifies it based on metadata. For example, it may be classified with the tag "Okonomiyaki" under the category "Food." The output is the classified data stored in the database.

[0256] Step 6:

[0257] A user searches for information on a device. The user enters a search query (e.g., "local cuisine"), and the device sends this search query to a server. The input data is the search query, and the output is a request for search results.

[0258] Step 7:

[0259] The server searches the database and organizes the results. The server searches the database based on the received search query and organizes the relevant information. The input to this process is the search query, and the output is the search result data.

[0260] Step 8:

[0261] The server sends the search results. The server sends the organized search results to the user terminal. The output of this process is the search result data sent to the user terminal.

[0262] Step 9:

[0263] The user views the search results on their device and checks detailed information. The user then views the search results sent from the server via their device. As a result, the user can view the recipe for "okonomiyaki" and related information.

[0264] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0265] The present invention combines a system for inputting, saving, searching, and viewing regional cultural information with an emotion engine that recognizes user emotions, and provides information based on the user's emotions. This system is composed of four main components: a user terminal, a server, a database, and the emotion engine. Specific embodiments for implementing the present invention are described below.

[0266] System Overview

[0267] This system has a series of functions in which users input information, send it to a server, and the server stores, categorizes, and provides it to users as needed.Furthermore, it uses an emotion engine to analyze the user's emotions and customize the information displayed based on the results.

[0268] Program processing

[0269] When a user enters information and sends it to the server

[0270] Subject: User and Device

[0271] 1. Users input information about their local culture, events, and games on their devices. For example, they can upload a recipe for a particular local dish or a video of a traditional hand game.

[0272] 2. The device sets appropriate metadata such as title, description, category, and tags for the entered information.

[0273] 3. The device creates a data package of the information and sends it to the server.

[0274] When the server stores and organizes information

[0275] Subject: Server

[0276] 1. The server receives the data package from the device and analyzes and separates the information and metadata.

[0277] 2. The server classifies the information with appropriate categories and tags and stores it in a database.

[0278] 3. The server indexes the newly stored information and makes it searchable.

[0279] When the emotion engine recognizes the user's emotions

[0280] Subject: Emotion engine and device

[0281] 1. The emotion engine uses facial recognition and text analysis to identify the user's current emotion.

[0282] 2. The device receives the user's emotion data from the emotion engine and sends it to the server.

[0283] 3. The server analyzes the emotion data and prepares to provide the user with information on the local culture that best suits them.

[0284] When a user searches or browses information

[0285] Subject: User, device, server

[0286] 1. The user enters specific keywords or categories into the device and sends a search request to the server.

[0287] 2. The server analyzes the user's search query and searches the database.

[0288] 3. The server compares the search results with the emotion engine data and sets priorities.

[0289] 4. The server sends the organized search results to the device, where the user can view them.

[0290] Specific examples

[0291] For example, if a user searches for "traditional hand games," the user can enter "otedama" (beanbag) and the device will send this query to the server. If the emotion engine recognizes that the user is relaxed, the server will prioritize providing videos and information related to "otedama" that have a relaxing effect. The user can browse the results, learn how to play the traditional hand game "otedama," and obtain the most appropriate information according to their emotional state at the time.

[0292] summary

[0293] This system not only records and preserves local culture, but also uses an emotion engine to provide information that responds to the user's emotions, allowing users to learn about and experience local culture more effectively and comfortably.

[0294] The processing flow will be explained below.

[0295] When a user uploads information

[0296] Step 1:

[0297] Users input information about local culture (e.g., video files or text information) from their own devices (smartphones, PCs, etc.).

[0298] Step 2:

[0299] The system sets metadata such as an appropriate title, description, category, and tag for the information entered by the user.

[0300] Step 3:

[0301] The user presses the "upload" button to issue a request to the terminal to transmit information.

[0302] Step 4:

[0303] The device verifies the information entered and creates a data package containing the necessary metadata.

[0304] Step 5:

[0305] The terminal requests that a data package be sent to the server.

[0306] When the server stores and organizes information

[0307] Step 6:

[0308] The server receives the data package sent from the terminal.

[0309] Step 7:

[0310] The server parses and separates the information and metadata from the received data packages.

[0311] Step 8:

[0312] The server prepares to store the parsed information in a database.

[0313] Step 9:

[0314] The server categorizes the information based on appropriate categories and tags and stores it in a database.

[0315] Step 10:

[0316] The server indexes the newly stored information and makes it searchable.

[0317] Step 11:

[0318] The server returns a confirmation message to the device that the information has been saved and classified.

[0319] When the emotion engine recognizes the user's emotions

[0320] Step 12:

[0321] The emotion engine uses facial recognition and text analysis to recognize the user's current emotions.

[0322] Step 13:

[0323] The terminal receives the user's emotion data from the emotion engine and transmits it to the server.

[0324] When a user searches or browses information

[0325] Step 14:

[0326] A user enters a search query on their device, specifying specific keywords and categories.

[0327] Step 15:

[0328] The terminal analyzes the user's search query and sends a search request to the server.

[0329] Step 16:

[0330] The server searches the database based on the received search query and retrieves the relevant information.

[0331] Step 17:

[0332] The server compares the search results with the emotion data and preferentially selects information that matches the user's emotion.

[0333] Step 18:

[0334] The server sends the organized search results to the device.

[0335] Step 19:

[0336] The user views the search results on their device and checks for more information.

[0337] When providing recipes or event information

[0338] Step 20:

[0339] A user requests information about a particular local event or dish.

[0340] Step 21:

[0341] The terminal analyzes the user's request and sends it to the server.

[0342] Step 22:

[0343] The server retrieves relevant information from the database, organizes it, and provides it to the user's terminal.

[0344] Step 23:

[0345] The user views the provided information on the device and deepens their understanding of the events and cuisine.

[0346] Using an emotion engine to customize information

[0347] Step 24:

[0348] The emotion engine records the user's emotion history and uses it to help with the next information search.

[0349] Step 25:

[0350] The emotion engine customizes the default display priority based on the user's current emotions.

[0351] This allows the system to effectively record and preserve local culture and provide optimal information based on the user's feelings, allowing users to comfortably learn and experience cultural information.

[0352] Example 2

[0353] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0354] Conventional regional cultural information systems allowed users to input, save, search, and view information, but lacked the ability to provide information based on the user's emotions. This made it difficult for users to obtain information that matched their emotional state, resulting in a decrease in satisfaction. Furthermore, conventional systems were limited to providing simple text and video data, making it difficult to gain a deep understanding of regional culture or to learn effectively.

[0355] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0356] In this invention, the server includes means for inputting regional culture information, means for transmitting the input information together with metadata to a transmission device, means for the transmission device to store and classify the transmitted information in a storage device, means for a user to search and view the stored information, and means for an emotion recognition device to analyze the user's emotions and provide information based on the user's emotions. This makes it possible to provide optimal regional culture information according to the user's emotional state, improving user satisfaction and enabling effective learning of regional culture.

[0357] "Local culture" refers to the collection of traditions, customs, events, food, games, etc. that are unique to a particular region or community.

[0358] "Means for inputting information" refers to an interface that allows users to provide data (text, video, images, etc.) related to local culture to the system.

[0359] "Metadata" refers to data (title, description, category, tags, etc.) that describes the attributes and structure of the information itself.

[0360] "Transmitting device" refers to a device or system for transmitting information and metadata input from a user terminal to a server.

[0361] "Storage" refers to a database or storage that stores the transmitted information and metadata so that it can be easily searched and classified later.

[0362] "Classification methods" refers to the process of organizing received information into categories or tags to make it easily searchable.

[0363] "Search and browsing means" refers to an interface that allows a user to search for stored information using specific keywords or categories and view the results.

[0364] An "emotion recognition device" refers to technology that identifies a user's emotions through facial recognition or text analysis, and customizes information based on that.

[0365] "Means for providing information based on user emotions" refers to a process for providing the most appropriate local cultural information by taking into account the perceived emotional state of the user.

[0366] This invention combines a system for inputting, saving, searching, and viewing regional cultural information with an emotion engine that recognizes the user's emotions, and provides information based on the user's emotions. This system consists of four main components: a user terminal, a server, a database, and an emotion engine.

[0367] System Overview

[0368] This system has a series of functions in which the user inputs information, sends it to a server, and the server stores, categorizes, and provides it to the user as needed. Furthermore, it uses an emotion engine to analyze the user's emotions and customizes and displays information based on the results. Specifically, it uses the following hardware and software:

[0369] User devices: smartphones, tablets, computers, etc.

[0370] Server: Cloud server (AWS (registered trademark), Google Cloud, Microsoft (registered trademark), Azure (registered trademark), etc.)

[0371] Database: MySQL (registered trademark), PostgreSQL, MongoDB, etc.

[0372] Emotion engine: Commonly used emotion recognition technologies (e.g., Microsoft Azure Emotion Recognition API, Google Cloud Vision API, etc.)

[0373] Specific software: Django (Python framework), React (front-end)

[0374] The information and metadata entered by the user through the device are sent as a series of data packages to the server, which analyzes, classifies, and stores them in a database. The emotion engine also recognizes the user's emotional state and provides information based on that emotion.

[0375] Specific examples

[0376] For example, if a user searches for "traditional hand games," the user can input "otedama" into the device and the query will be sent to the server. If the emotion engine recognizes that the user is relaxed, the server can provide images and information related to "otedama" that have a relaxing effect preferentially. The user can browse these results and obtain the most appropriate information while learning how to play the traditional hand game "otedama." Also, the following prompt sentence can be used:

[0377] "Please tell me about a local festival that suits my mood today."

[0378] "I'd like to know some local recipes. Do you have any recommendations?"

[0379] Results of the implementation

[0380] This system not only allows users to efficiently record, save, search, and view information about local culture, but also utilizes an emotion engine to provide optimal information according to the user's emotional state, thereby improving user satisfaction and enabling effective learning and experience of local culture.

[0381] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0382] Below, a specific explanation is given, broken down into the processing steps of the system.

[0383] Step 1:

[0384] A user inputs information about the culture of their region (e.g., a recipe for a traditional dish or a video of a hand game) into a terminal. The information input can be text, images, or videos. Specifically, consider the case where a user inputs an "okonomiyaki recipe."

[0385] Step 2:

[0386] The device automatically assigns appropriate metadata such as title, description, category, and tags to the information entered. For example, the title might be "Okonomiyaki Recipe," the category might be "Cuisine," and the tags might be "Japan" and "Local Cuisine." This information is then compiled into a data package.

[0387] Step 3:

[0388] The device creates a data package and sends it to the server, which contains the entered information and metadata and is transmitted via a secure communication protocol (e.g., HTTPS).

[0389] Step 4:

[0390] The server receives the data package sent from the device, analyzes the received data package, and separates the information body (e.g., recipes and videos) from the metadata (e.g., titles and tags).

[0391] Step 5:

[0392] The server categorizes the separated information body and metadata based on appropriate categories and tags and stores them in a database. For example, information on "okonomiyaki recipes" is stored in the "cooking" category.

[0393] Step 6:

[0394] The server creates an index for the newly stored information and indexes it for fast searching. This process makes the information searchable.

[0395] Step 7:

[0396] The emotion engine uses facial recognition and text analysis to recognize the user's emotions. For example, the emotion engine analyzes the user's facial expressions and voice through a dedicated camera or microphone to identify an emotional state such as "relaxed."

[0397] Step 8:

[0398] The device receives the user's emotion data from the emotion engine and transmits it to the server via a secure communication protocol.

[0399] Step 9:

[0400] The server analyzes the received emotional data and reprioritizes information based on the results. For example, a user who is "relaxed" will be given priority for videos that have a relaxing effect.

[0401] Step 10:

[0402] A user inputs a specific keyword or category into the device and sends an information search request to the server. For example, the user inputs "otenjaku" (beanbag).

[0403] Step 11:

[0404] The server analyzes the user's search query, searches the database, and compares the search results with data from the emotion engine to prioritize search results based on the user's emotional state.

[0405] Step 12:

[0406] The server sends the search results to the device. If the user is feeling relaxed, the search results are adjusted so that information with a relaxing effect is displayed at the top.

[0407] Step 13:

[0408] Users can view search results on their devices and use the displayed information to learn more about local culture, for example, how to juggle.

[0409] These specific processing steps enable users to effectively input, save, search, and view local cultural information, and further enable the provision of customized information based on emotions.

[0410] (Application example 2)

[0411] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0412] Modern society demands systems that can efficiently input, store, search, and browse information about local culture. However, conventional systems have difficulty providing information that adapts to the user's emotions. For example, the optimal information varies depending on the user's emotions, such as when they want to relax or when they are excited. However, only uniform information is provided. Therefore, there is a need for information that can be customized based on the user's emotions.

[0413] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting information on regional culture, means for transmitting the input information together with metadata to the server, means for the server to store the transmitted information in a database and classify it, means for the user to search and view the stored information, means for acquiring emotion data using an emotion engine that recognizes the user's emotions, and means for customizing information to be provided to the user based on the emotion data. This makes it possible to provide personalized information according to the user's emotions.

[0414] "Regional cultural information" refers to information about events, festivals, cuisine, traditional games, etc. related to a particular region.

[0415] "Means of input" refers to the means by which a user inputs information into the system, and includes methods such as text input, image upload, and voice input.

[0416] "Means for transmitting to the server" refers to a method for transmitting information input from a user terminal to the server via a network, and includes internet connection, wireless communication, and the like.

[0417] "Means for storing and categorizing" refers to the way in which the server stores the received information in a database and organizes it by categories and tags.

[0418] "Means for searching and viewing" refers to the interface or functionality that allows a user to search the database for stored information and view the results on a device.

[0419] An "emotion engine" refers to algorithms or software that identify and acquire data on a user's emotions through facial recognition, voice analysis, and text analysis.

[0420] "Emotion data" refers to information about the user's emotions acquired by the emotion engine, and includes, for example, "joy," "sadness," and "surprise."

[0421] "Means for customization" refers to methods or processes for optimizing and personalizing information provided to users based on acquired emotional data.

[0422] "Product information" refers to detailed product information provided in the virtual store, including product name, price, description, and image.

[0423] This invention combines a system for inputting, saving, searching, and viewing regional cultural information with an emotion engine that recognizes the user's emotions, thereby providing information based on the user's emotions. This system is primarily composed of four main components: a user terminal, a server, a database, and an emotion engine.

[0424] ● System Overview

[0425] The system is based on a series of processes in which users input information about local culture and then send that information to a server. Specifically, the process involves the following steps:

[0426] 1. User Device

[0427] Users can input information about their local culture, events, and pastimes from their devices in a variety of ways, including text entry, image uploading, and video recording.

[0428] The device automatically sets metadata such as appropriate titles, descriptions, categories, and tags for the information entered.

[0429] The terminal creates a data package of information and sends it to the server.

[0430] 2. Server

[0431] The server receives the data packages from the devices and analyzes and classifies the information and metadata.

[0432] The server classifies the information with appropriate categories and tags and stores it in a database.

[0433] The server creates an index of the newly stored information and makes it searchable.

[0434] 3. Emotion Engine

[0435] The emotion engine uses facial recognition, voice analysis, and text analysis to identify the user's current emotion.

[0436] The terminal receives the user's emotion data from the emotion engine and transmits it to the server.

[0437] The server analyzes the emotional data and prepares to provide the user with information on the local culture that is most appropriate for them.

[0438] 4. Searching and browsing information

[0439] The user enters specific keywords or categories into the device and sends a search request to the server.

[0440] The server analyzes the user's search query and searches the database.

[0441] The server compares the search results with data from the emotion engine and sets priorities.

[0442] The server sends the organized search results to the terminal, where the user can view them.

[0443] ● Hardware and software used

[0444] The system of the present invention utilizes the following hardware and software:

[0445] Hardware

[0446] Head-mounted display (HMD)

[0447] Smart Glasses

[0448] Webcam

[0449] software

[0450] OpenCV: Used for face recognition

[0451] EmotionRecognition: Emotion analysis library

[0452] ProductDatabase: Product database

[0453] RecommendationEngine: Product recommendation engine

[0454] ● Examples of operations and prompt statements

[0455] For example, if a user searches for "traditional hand games," the user can input "beanbag" and the device will send this query to the server. If the emotion engine recognizes that the user is relaxed, the server will prioritize providing images and information related to "beanbag" that have a relaxing effect.

[0456] Another example of a prompt might be:

[0457] If you type "I feel happy," you'll see recommendations for fun products (e.g., new board games).

[0458] If you enter "feeling angry," products with a relaxing effect (e.g., scented candles) will be recommended.

[0459] This system allows users to receive personalized information tailored to their specific emotions, enabling them to learn and experience local culture more effectively and comfortably.

[0460] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0461] Step 1:

[0462] The user enters information and sends it to the terminal

[0463] A user uses a terminal to input information about local culture in the form of text, images, audio, video, etc. The terminal automatically sets appropriate metadata (e.g., title, description, category, tags, etc.) for the input information. This information packet is organized into a data package and sent to the server.

[0464] Input: User-entered local cultural information and metadata

[0465] Data processing and calculation: Composing input information into a data package with metadata

[0466] Output: Composed data package

[0467] Step 2:

[0468] The server receives the information, stores it in a database, and categorizes it.

[0469] The server receives the data package sent from the device, analyzes and separates the information and metadata, classifies the analyzed information with appropriate categories and tags, and stores it in a database. At the same time, a search index is also generated.

[0470] Input: Data package

[0471] Data processing and calculation: information and metadata analysis, separation, categorization, storage, and indexing

[0472] Output: Information stored in the database and the generated search index

[0473] Step 3:

[0474] The device acquires the user's emotional data from the emotion engine and sends it to the server.

[0475] The device uses its built-in emotion engine to identify the user's current emotion through facial recognition and voice analysis, and this emotion data is sent to the server as a data package.

[0476] Input: User's facial image and voice data

[0477] Data processing and calculation: Emotion identification by emotion engine, generation of data packages

[0478] Output: Emotion data package

[0479] Step 4:

[0480] The server analyzes the emotion data and prepares to provide information.

[0481] The server analyzes the transmitted emotional data and customizes the information provided based on the user's emotional state. For example, if the user is in a relaxed state, it will prioritize information that has a relaxing effect.

[0482] Input: Emotion data package

[0483] Data processing and calculation: Emotional data analysis, customization of information provision

[0484] Output: Preparing customized informational presentations

[0485] Step 5:

[0486] The user enters specific keywords or categories into the device and sends a search request.

[0487] The user enters specific keywords and categories into the device to send a search request, which is then sent to the server.

[0488] Input: User's search query (keywords or category)

[0489] Data processing and calculation: receiving search queries and generating requests

[0490] Output: Search request to the server

[0491] Step 6:

[0492] The server processes search requests and prioritizes them based on sentiment data.

[0493] The server receives a user's search query, searches the database, matches the search results with sentiment data, and sets priorities based on the user's sentiment.

[0494] Input: search query, sentiment data

[0495] Data processing and calculation: database search, matching with emotion data, setting priorities

[0496] Output: Prioritized search results

[0497] Step 7:

[0498] The server sends the organized search results to the device, where the user can view them.

[0499] The server sends the prioritized search results to the terminal, where the user can browse them to obtain the most appropriate local cultural information according to their emotional state.

[0500] Input: prioritized search results

[0501] Data processing and calculation: formatting and sending results

[0502] Output: Search results displayed on the device

[0503] This series of steps results in a system that allows users to efficiently search and view information about local culture that is optimized to suit their own emotions.

[0504] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0505] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0506] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0507] [Second embodiment]

[0508] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0509] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0510] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0511] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0512] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0513] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0514] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0515] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0516] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0517] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0518] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0519] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0520] The present invention relates to a system for recording, preserving, and preventing the erosion of information about local culture, food, and games (hand games and songs). This system has a series of functions that allow users to input information, send it to a server, and the server classifies, preserves, and provides the information. Specific embodiments for implementing the present invention are described below.

[0521] System Overview

[0522] This system consists of three main components: a user terminal, a server, and a database. Users use their terminals to input information about local culture and send it to the server. The server then appropriately categorizes the received information and stores it in the database. Users can also search for information using specific keywords or categories and view the information retrieved from the database.

[0523] Program processing

[0524] When a user enters information and sends it to the server

[0525] Subject: User and Device

[0526] 1. Users input information about their local culture, events, and games on their devices. For example, they can upload a recipe for a particular local dish or a video of a traditional hand game.

[0527] 2. The device converts the input information into a video file or text file and adds metadata such as title, description, category, and tags.

[0528] 3. The device makes a request to send the converted information and metadata to the server.

[0529] When the server stores and organizes information

[0530] Subject: Server

[0531] 1. The server receives an upload request from the device and retrieves the submitted data and metadata.

[0532] 2. The server stores the received information in a database and categorizes it into appropriate categories and tags based on the metadata.

[0533] 3. The server indexes the newly stored information and makes it available for searching.

[0534] When a user searches or browses information

[0535] Subject: User, device, server

[0536] 1. The user searches from their device by specifying specific keywords or categories.

[0537] 2. The device sends the user's search query to the server.

[0538] 3. The server searches the database based on the search query, retrieves and organizes the relevant information.

[0539] 4. The server sends the search results to the device in an appropriate format.

[0540] 5. The user views the search results on their device and checks for more information.

[0541] Specific examples

[0542] For example, let's consider a case where a user searches for "traditional hand games." When the user enters "otedama" and presses the search button, the device sends this query to the server. The server retrieves video files and text information related to "otedama" from the database, organizes the results, and sends them back to the device. The user can view the results and learn how to play the traditional hand game "otedama."

[0543] summary

[0544] This system aims to prevent the erosion of culture by recording and storing local culture, food, and entertainment, and making that data easily searchable and viewable. The basic functions required for this are implemented between user devices and a server. This allows anyone to easily learn about local traditions and culture and pass them on to the next generation.

[0545] The processing flow will be explained below.

[0546] When a user uploads information

[0547] Step 1:

[0548] Users input information about local culture (e.g., video files or text information) from their own devices (smartphones, PCs, etc.).

[0549] Step 2:

[0550] The system sets metadata such as an appropriate title, description, category, and tag for the information entered by the user.

[0551] Step 3:

[0552] The user presses the "upload" button to issue a request to the terminal to transmit information.

[0553] Step 4:

[0554] The device verifies the information entered and creates a data package containing the necessary metadata.

[0555] Step 5:

[0556] The terminal requests that a data package be sent to the server.

[0557] When the server stores and organizes information

[0558] Step 6:

[0559] The server receives the data package sent from the terminal.

[0560] Step 7:

[0561] The server parses and separates the information and metadata from the received data packages.

[0562] Step 8:

[0563] The server prepares to store the parsed information in a database.

[0564] Step 9:

[0565] The server categorizes the information based on appropriate categories and tags and stores it in a database.

[0566] Step 10:

[0567] The server indexes the newly stored information and makes it searchable.

[0568] Step 11:

[0569] The server returns a confirmation message to the device that the information has been saved and classified.

[0570] When a user searches or browses information

[0571] Step 12:

[0572] A user enters a search query on their device, specifying specific keywords and categories.

[0573] Step 13:

[0574] The terminal analyzes the user's search query and sends a search request to the server.

[0575] Step 14:

[0576] The server searches the database based on the received search query and retrieves the relevant information.

[0577] Step 15:

[0578] The server properly organizes the acquired information and sends it to the terminal.

[0579] Step 16:

[0580] The user views the search results on their device and checks for more information.

[0581] When providing recipes or event information

[0582] Step 17:

[0583] A user requests information about a particular local event or dish.

[0584] Step 18:

[0585] The terminal analyzes the user's request and sends it to the server.

[0586] Step 19:

[0587] The server retrieves relevant information from the database, organizes it, and provides it to the user's terminal.

[0588] Step 20:

[0589] The user views the provided information on the device and deepens their understanding of the events and cuisine.

[0590] Through these processing steps, the system is able to effectively record and preserve local culture and provide it to users in an easily accessible form.

[0591] Example 1

[0592] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0593] Information about a region's unique culture, traditions, food, and pastimes is a valuable asset that should be passed on to future generations. However, if this information is not properly recorded and preserved, it is likely to fade away and be lost. Furthermore, current systems make it difficult to classify input information, making it difficult to search and view. Therefore, there is a need for a system that enables efficient recording, classification, search, and viewing of regional cultural information.

[0594] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0595] In this invention, the server includes means for inputting regional cultural information, means for transmitting the input information together with metadata, means for the server to store the transmitted information in a database and classify it based on the metadata, means for the server to create an index of the stored information and organize the information in a searchable form, means for a user to search for information based on specific keywords or categories, and means for the server to acquire, organize, and transmit the relevant information in an appropriate format to a terminal, and means for a user to view search results and check detailed information. This enables efficient recording, classification, search, and viewing of regional cultural information.

[0596] "Regional culture" is a general term for cultural elements such as events, food, and games that have traditionally been passed down in a particular region.

[0597] "Means for inputting information" refers to a device or interface that allows a user to input information about local culture in digital form.

[0598] "Metadata" refers to additional information such as title, description, category, and tag that is added to input information.

[0599] "Means for sending to the server" refers to the communications technology or protocol used to transfer digital information from the terminal to the server.

[0600] "Means of storing in a database" refers to the systems and technologies used by the server to permanently store the digital information it receives.

[0601] "Indexing means" refers to systems or techniques that generate tables of contents or indexes to facilitate efficient searching of stored digital information.

[0602] "Means of making information searchable" refers to database management and search engine technologies that enable quick and accurate responses to user search queries.

[0603] A "search query" refers to a keyword or phrase that a user enters when searching for information.

[0604] "Methods of organization" refers to techniques and processes for systematically sorting acquired information based on certain criteria.

[0605] "Means for transmitting to the terminal in an appropriate format" refers to the communication technology and protocols that allow the server to provide organized information to the user terminal in an appropriate format.

[0606] "Means for viewing search results" refers to the interface and display technology that allows users to visually check search results using a terminal.

[0607] This invention is a system that effectively records, stores, and easily searches and views information about local culture, food, and entertainment. This system sends information entered by users to a server, which then classifies and stores the information in a database, allowing users to easily search and view it.

[0608] System configuration

[0609] The system mainly consists of the following components:

[0610] 1. User Device

[0611] 2. Server

[0612] 3. Database

[0613] Entering information

[0614] Users input information about their local culture, events, and games into their devices. Specific examples include recipes for local dishes and videos of traditional hand games. The device saves this information as video or text files and adds metadata such as titles, descriptions, categories, and tags.

[0615] Sending data

[0616] The device uses a file upload function to send the converted information and metadata to the server, for example, by using an HTTP POST request to send the data to the server, which receives the request and stores the information and metadata in a database.

[0617] Data classification and storage

[0618] The server stores the received information in a database and categorizes it appropriately based on the metadata. Specifically, it stores the video file in storage and stores the metadata (title, description, category, tags, etc.) in the appropriate tables in the database.

[0619] Making the site searchable

[0620] The server then indexes the newly stored information and makes it searchable, which includes using a full-text search engine to generate the index.

[0621] Searching and browsing information

[0622] Users search for information by specifying specific keywords or categories on their devices. At that time, the device sends the user's search query to the server. The server searches the database based on the search query, retrieves and organizes the relevant information, and then sends it to the device in an appropriate format (for example, JSON format). Finally, the user can view the search results and check detailed information.

[0623] Example prompt

[0624] As a concrete example, let's consider the case where a user searches for "traditional hand games." The user enters "otedama" and presses the search button. The device sends this query to the server via an HTTP GET request. The server executes an SQL query, retrieves video files and text information related to "otedama" from the database, and returns the results organized in JSON format to the device. The device displays the search results in a list, and the user can view detailed information about "how to play otedama" in a pop-up display.

[0625] Example prompts to input to a generative AI model:

[0626] "I would like to know how to play otedama, a traditional Japanese hand game. What are the steps and what materials are needed?"

[0627] This system will enable efficient storage, classification, searching, and viewing of local cultural information, and is expected to help prevent culture from fading away.

[0628] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0629] Step 1:

[0630] The user enters information

[0631] Description: Users input information about their local culture, events, and pastimes from their devices.

[0632] Input: The user enters free-form text information and video data into the input form.

[0633] Data processing / calculation: Text data is stored as is via the input device, and video data is stored in digital format.

[0634] Output: Input information temporarily stored on the device.

[0635] Specific operation: The user enters a description of a traditional hand game into a form, then takes a video of the game using their smartphone and uploads it.

[0636] Step 2:

[0637] Device adds metadata

[0638] Description: The device adds metadata to the information entered, such as title, description, category, and tags.

[0639] Input: Text and video data provided by the user in Step 1.

[0640] Data processing / calculation: The process of converting data into a storage format for databases and adding metadata.

[0641] Output: Information with metadata added.

[0642] Specific operation: For a video titled "How to play with beanbags," the device automatically sets the title "Beanbags," the description "Traditional Japanese hand game," the category "Game," and the tags "Hand game, beanbags."

[0643] Step 3:

[0644] The device sends data to the server

[0645] Description: The device sends the converted information and metadata to the server.

[0646] Input: Information with metadata.

[0647] Data processing / computation: Packaging data and generating transmission requests.

[0648] Output: The data sent to the server.

[0649] What it does: Sends data to the server as an HTTP POST request, including metadata in the data packet.

[0650] Step 4:

[0651] The server receives and stores the data

[0652] Description: The server receives upload requests from devices and stores the submitted data and metadata in a database.

[0653] Input: Data and metadata sent from the device.

[0654] Data processing / calculation: Inserting data into the database and saving it to storage.

[0655] Output: Information stored in a database.

[0656] Specific operation: The video file is stored in server-side storage, and the metadata is inserted into the appropriate table in the database.

[0657] Step 5:

[0658] The server categorizes and organizes the information

[0659] Description: The server categorizes information based on metadata, indexes newly stored information, and makes it searchable.

[0660] Input: Information and metadata stored in a database.

[0661] Data processing / computation: Classification and indexing of information using indexing algorithms.

[0662] Output: Indexed and searchable information.

[0663] Specific operation: Tags such as "hand games" and "beanbags" are added to the category "play," and an index is created using a full-text search engine.

[0664] Step 6:

[0665] The user enters a search query

[0666] Description: Users search for information from their devices by specifying specific keywords or categories.

[0667] Input: User-specified keywords and categories.

[0668] Data processing / calculation: Generating search queries entered by users.

[0669] Output: The search query sent from the device to the server.

[0670] Specific behavior: The user enters "Beanbag" into the search form and presses the search button.

[0671] Step 7:

[0672] The device sends the search query to the server

[0673] Description: The device sends the user's search query to the server.

[0674] Input: The search query specified by the user.

[0675] Data processing / computation: Generating and submitting search requests.

[0676] Output: The search query sent to the server.

[0677] Specific behavior: Sends a search query to the server via an HTTP GET request.

[0678] Step 8:

[0679] The server searches the database

[0680] Description: The server searches the database based on the search query, retrieves and organizes relevant information.

[0681] Input: The search query sent from the device.

[0682] Data processing / calculation: Execute SQL queries and extract relevant information from the database.

[0683] Output: Organized search results.

[0684] Specific operation: The server executes an SQL query to retrieve video files and text information related to "Beanbag."

[0685] Step 9:

[0686] The server sends the search results to the device.

[0687] Description: The server sends the search results in the appropriate format to the device.

[0688] Input: Organized search results.

[0689] Data processing / calculation: Packaging search results in an appropriate format such as JSON.

[0690] Output: Search results sent to your device.

[0691] Specific operation: The server returns the search results to the device in JSON format.

[0692] Step 10:

[0693] Your device will display the search results.

[0694] Description: The device displays the search results received from the server in the user interface.

[0695] Input: Search results sent from the server.

[0696] Data processing / calculation: Search results are displayed in list format, and detailed information about the items selected by the user is displayed in a pop-up.

[0697] Output: Search results displayed in the user interface.

[0698] Specific behavior: The device displays the search results in a list format, and the user checks detailed information about "How to play beanbag."

[0699] (Application example 1)

[0700] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0701] Information about regional culture easily fades with time, making it particularly difficult to pass on to younger generations. Furthermore, manually entering information is time-consuming, and the lack of a voice input method makes it difficult to use. The objective of this invention is to provide a system that efficiently collects regional cultural information and utilizes voice recognition technology to allow users to easily input, search, and view information.

[0702] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0703] In this invention, the server includes means for inputting regional culture information, means for transmitting the input information together with metadata to the server, means for the server to store the transmitted information in a database and classify it, means for a user to input regional culture information by voice using voice recognition technology, means for converting the voice-input information into text data, and means for a user to search and view the stored information. This allows users to utilize voice input to easily record information and pass on regional culture.

[0704] "Regional culture" refers to cultural characteristics including events, food, games, traditions, customs, and practices unique to a region.

[0705] "Means for inputting information" means the means by which a user can directly or indirectly input data into a system, including voice, text, video, or other formats.

[0706] "Means for sending to the server" refers to the communication means or protocol for converting the input information into a certain format and sending it to the server.

[0707] "Server" refers to a computer system that processes, stores, and appropriately manages received information.

[0708] A "database" refers to a structured collection of data that stores information and makes it easy to retrieve when needed.

[0709] A "means for classification" refers to a means for organizing input information based on specific criteria or tags.

[0710] "Speech recognition technology" refers to technology for converting voice data into text data.

[0711] The term "means for inputting regional culture information by voice" refers to a means for a user to input regional culture information into the system by voice.

[0712] "Means for converting into text data" refers to means for converting information input by voice into text format.

[0713] "Means for searching and viewing information" refers to means by which a user can search for stored information based on specific keywords or categories and display the results.

[0714] This invention is a system for efficiently collecting, recording, and storing information about local culture, food, entertainment, etc., and preventing it from fading away. The invention is composed of three main components: a user terminal, a server, and a database. Specific embodiments of the invention are described below.

[0715] Input information via user terminal

[0716] First, the user uses voice recognition technology to input information about local culture. Voice recognition software such as Google Speech-to-Text API is installed on the user's device, and the voice data is converted into text data. For example, if a user says, "The local dish of this region is okonomiyaki," the content is converted into text data. At the same time, metadata such as title, description, category, and tags are also generated.

[0717] Data processing and storage by server

[0718] Text data and metadata generated on user devices are sent via the Internet to a server. The server receives this data and stores it in a database, where it is automatically categorized and organized based on the metadata. The server also creates an index of the newly stored information, making it instantly searchable.

[0719] Users search for and view information

[0720] Users can search for information by specifying specific keywords or categories. When a user enters a search query using a terminal, the query is sent to the server. The server searches for relevant information from the database and sends the results to the user's terminal. For example, if a user searches for "traditional hand games," the server will provide the user with video files and text information related to "beanbags."

[0721] Hardware and software used

[0722] The hardware used includes user devices (such as smartphones), servers, database servers, etc. The software used includes the Google Speech-to-Text API, a voice recognition technology, the HTTP communication protocol, and a database management system.

[0723] For example:

[0724] As users tour the factory, they can talk about local cultures, and their voices are recorded by their smartphones. A speech recognition system converts the speech into text, adds metadata, and sends it to a server, where it can be stored, categorized, and made searchable by other users if desired.

[0725] Prompt Sentence Examples

[0726] Here are some answers to the open-ended question that begins, "What is the local cuisine of your region?"

[0727] Example: Question: "What is the local dish of this region?"

[0728] Answer: "The local dish of this area is okonomiyaki."

[0729] As a result, the system of the present invention provides users with the ability to easily record, search, and view information about local culture, making it possible to prevent local culture from fading away.

[0730] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0731] Step 1:

[0732] The user inputs information about local culture into the device by voice. Using voice recognition technology, the user says, "The local dish of this region is okonomiyaki." The device collects this voice data. As a result, the voice data is obtained as input data.

[0733] Step 2:

[0734] Convert voice data into text data. The input voice data is converted into text data using the Google Speech-to-Text API installed on the device. The output of this process is the text data, "The local dish of this region is okonomiyaki."

[0735] Step 3:

[0736] Metadata is added to the text data. The device generates and adds metadata such as title, description, category, and tag to the converted text data. For example, metadata such as title "Local Cuisine," category "Meal," and tag "Okonomiyaki" are added. This process outputs data that combines the text data and metadata.

[0737] Step 4:

[0738] The text data with metadata is sent to the server. The terminal sends the generated data to the server using the HTTP communication protocol. In this step, data is sent from the user terminal to the server.

[0739] Step 5:

[0740] The server stores and classifies the received data. The server stores the received data in a database and automatically classifies it based on metadata. For example, it may be classified with the tag "Okonomiyaki" under the category "Food." The output is the classified data stored in the database.

[0741] Step 6:

[0742] A user searches for information on a device. The user enters a search query (e.g., "local cuisine"), and the device sends this search query to a server. The input data is the search query, and the output is a request for search results.

[0743] Step 7:

[0744] The server searches the database and organizes the results. The server searches the database based on the received search query and organizes the relevant information. The input to this process is the search query, and the output is the search result data.

[0745] Step 8:

[0746] The server sends the search results. The server sends the organized search results to the user terminal. The output of this process is the search result data sent to the user terminal.

[0747] Step 9:

[0748] The user views the search results on their device and checks detailed information. The user then views the search results sent from the server via their device. As a result, the user can view the recipe for "okonomiyaki" and related information.

[0749] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0750] The present invention combines a system for inputting, saving, searching, and viewing regional cultural information with an emotion engine that recognizes user emotions, and provides information based on the user's emotions. This system is composed of four main components: a user terminal, a server, a database, and the emotion engine. Specific embodiments for implementing the present invention are described below.

[0751] System Overview

[0752] This system has a series of functions in which users input information, send it to a server, and the server stores, categorizes, and provides it to users as needed.Furthermore, it uses an emotion engine to analyze the user's emotions and customize the information displayed based on the results.

[0753] Program processing

[0754] When a user enters information and sends it to the server

[0755] Subject: User and Device

[0756] 1. Users input information about their local culture, events, and games on their devices. For example, they can upload a recipe for a particular local dish or a video of a traditional hand game.

[0757] 2. The device sets appropriate metadata such as title, description, category, and tags for the entered information.

[0758] 3. The device creates a data package of the information and sends it to the server.

[0759] When the server stores and organizes information

[0760] Subject: Server

[0761] 1. The server receives the data package from the device and analyzes and separates the information and metadata.

[0762] 2. The server classifies the information with appropriate categories and tags and stores it in a database.

[0763] 3. The server indexes the newly stored information and makes it searchable.

[0764] When the emotion engine recognizes the user's emotions

[0765] Subject: Emotion engine and device

[0766] 1. The emotion engine uses facial recognition and text analysis to identify the user's current emotion.

[0767] 2. The device receives the user's emotion data from the emotion engine and sends it to the server.

[0768] 3. The server analyzes the emotion data and prepares to provide the user with information on the local culture that best suits them.

[0769] When a user searches or browses information

[0770] Subject: User, device, server

[0771] 1. The user enters specific keywords or categories into the device and sends a search request to the server.

[0772] 2. The server analyzes the user's search query and searches the database.

[0773] 3. The server compares the search results with the emotion engine data and sets priorities.

[0774] 4. The server sends the organized search results to the device, where the user can view them.

[0775] Specific examples

[0776] For example, if a user searches for "traditional hand games," the user can enter "otedama" (beanbag) and the device will send this query to the server. If the emotion engine recognizes that the user is relaxed, the server will prioritize providing videos and information related to "otedama" that have a relaxing effect. The user can browse the results, learn how to play the traditional hand game "otedama," and obtain the most appropriate information according to their emotional state at the time.

[0777] summary

[0778] This system not only records and preserves local culture, but also uses an emotion engine to provide information that responds to the user's emotions, allowing users to learn about and experience local culture more effectively and comfortably.

[0779] The processing flow will be explained below.

[0780] When a user uploads information

[0781] Step 1:

[0782] Users input information about local culture (e.g., video files or text information) from their own devices (smartphones, PCs, etc.).

[0783] Step 2:

[0784] The system sets metadata such as an appropriate title, description, category, and tag for the information entered by the user.

[0785] Step 3:

[0786] The user presses the "upload" button to issue a request to the terminal to transmit information.

[0787] Step 4:

[0788] The device verifies the information entered and creates a data package containing the necessary metadata.

[0789] Step 5:

[0790] The terminal requests that a data package be sent to the server.

[0791] When the server stores and organizes information

[0792] Step 6:

[0793] The server receives the data package sent from the terminal.

[0794] Step 7:

[0795] The server parses and separates the information and metadata from the received data packages.

[0796] Step 8:

[0797] The server prepares to store the parsed information in a database.

[0798] Step 9:

[0799] The server categorizes the information based on appropriate categories and tags and stores it in a database.

[0800] Step 10:

[0801] The server indexes the newly stored information and makes it searchable.

[0802] Step 11:

[0803] The server returns a confirmation message to the device that the information has been saved and classified.

[0804] When the emotion engine recognizes the user's emotions

[0805] Step 12:

[0806] The emotion engine uses facial recognition and text analysis to recognize the user's current emotions.

[0807] Step 13:

[0808] The terminal receives the user's emotion data from the emotion engine and transmits it to the server.

[0809] When a user searches or browses information

[0810] Step 14:

[0811] A user enters a search query on their device, specifying specific keywords and categories.

[0812] Step 15:

[0813] The terminal analyzes the user's search query and sends a search request to the server.

[0814] Step 16:

[0815] The server searches the database based on the received search query and retrieves the relevant information.

[0816] Step 17:

[0817] The server compares the search results with the emotion data and preferentially selects information that matches the user's emotion.

[0818] Step 18:

[0819] The server sends the organized search results to the device.

[0820] Step 19:

[0821] The user views the search results on their device and checks for more information.

[0822] When providing recipes or event information

[0823] Step 20:

[0824] A user requests information about a particular local event or dish.

[0825] Step 21:

[0826] The terminal analyzes the user's request and sends it to the server.

[0827] Step 22:

[0828] The server retrieves relevant information from the database, organizes it, and provides it to the user's terminal.

[0829] Step 23:

[0830] The user views the provided information on the device and deepens their understanding of the events and cuisine.

[0831] Using an emotion engine to customize information

[0832] Step 24:

[0833] The emotion engine records the user's emotion history and uses it to help with the next information search.

[0834] Step 25:

[0835] The emotion engine customizes the default display priority based on the user's current emotions.

[0836] This allows the system to effectively record and preserve local culture and provide optimal information based on the user's feelings, allowing users to comfortably learn and experience cultural information.

[0837] Example 2

[0838] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0839] Conventional regional cultural information systems allowed users to input, save, search, and view information, but lacked the ability to provide information based on the user's emotions. This made it difficult for users to obtain information that matched their emotional state, resulting in a decrease in satisfaction. Furthermore, conventional systems were limited to providing simple text and video data, making it difficult to gain a deep understanding of regional culture or to learn effectively.

[0840] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0841] In this invention, the server includes means for inputting regional culture information, means for transmitting the input information together with metadata to a transmission device, means for the transmission device to store and classify the transmitted information in a storage device, means for a user to search and view the stored information, and means for an emotion recognition device to analyze the user's emotions and provide information based on the user's emotions. This makes it possible to provide optimal regional culture information according to the user's emotional state, improving user satisfaction and enabling effective learning of regional culture.

[0842] "Local culture" refers to the collection of traditions, customs, events, food, games, etc. that are unique to a particular region or community.

[0843] "Means for inputting information" refers to an interface that allows users to provide data (text, video, images, etc.) related to local culture to the system.

[0844] "Metadata" refers to data (title, description, category, tags, etc.) that describes the attributes and structure of the information itself.

[0845] "Transmitting device" refers to a device or system for transmitting information and metadata input from a user terminal to a server.

[0846] "Storage" refers to a database or storage that stores the transmitted information and metadata so that it can be easily searched and classified later.

[0847] "Classification methods" refers to the process of organizing received information into categories or tags to make it easily searchable.

[0848] "Search and browsing means" refers to an interface that allows a user to search for stored information using specific keywords or categories and view the results.

[0849] An "emotion recognition device" refers to technology that identifies a user's emotions through facial recognition or text analysis, and customizes information based on that.

[0850] "Means for providing information based on user emotions" refers to a process for providing the most appropriate local cultural information by taking into account the perceived emotional state of the user.

[0851] This invention combines a system for inputting, saving, searching, and viewing regional cultural information with an emotion engine that recognizes the user's emotions, and provides information based on the user's emotions. This system consists of four main components: a user terminal, a server, a database, and an emotion engine.

[0852] System Overview

[0853] This system has a series of functions in which the user inputs information, sends it to a server, and the server stores, categorizes, and provides it to the user as needed. Furthermore, it uses an emotion engine to analyze the user's emotions and customizes and displays information based on the results. Specifically, it uses the following hardware and software:

[0854] User devices: smartphones, tablets, computers, etc.

[0855] Server: Cloud server (AWS, Google Cloud, Microsoft Azure, etc.)

[0856] Database: MySQL, PostgreSQL, MongoDB, etc.

[0857] Emotion engine: Commonly used emotion recognition technologies (e.g., Microsoft Azure Emotion Recognition API, Google Cloud Vision API, etc.)

[0858] Specific software: Django (Python framework), React (front-end)

[0859] The information and metadata entered by the user through the device are sent as a series of data packages to the server, which analyzes, classifies, and stores them in a database. The emotion engine also recognizes the user's emotional state and provides information based on that emotion.

[0860] Specific examples

[0861] For example, if a user searches for "traditional hand games," the user can input "otedama" into the device and the query will be sent to the server. If the emotion engine recognizes that the user is relaxed, the server can provide images and information related to "otedama" that have a relaxing effect preferentially. The user can browse these results and obtain the most appropriate information while learning how to play the traditional hand game "otedama." Also, the following prompt sentence can be used:

[0862] "Please tell me about a local festival that suits my mood today."

[0863] "I'd like to know some local recipes. Do you have any recommendations?"

[0864] Results of the implementation

[0865] This system not only allows users to efficiently record, save, search, and view information about local culture, but also utilizes an emotion engine to provide optimal information according to the user's emotional state, thereby improving user satisfaction and enabling effective learning and experience of local culture.

[0866] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0867] Below, a specific explanation is given, broken down into the processing steps of the system.

[0868] Step 1:

[0869] A user inputs information about the culture of their region (e.g., a recipe for a traditional dish or a video of a hand game) into a terminal. The information input can be text, images, or videos. Specifically, consider the case where a user inputs an "okonomiyaki recipe."

[0870] Step 2:

[0871] The device automatically assigns appropriate metadata such as title, description, category, and tags to the information entered. For example, the title might be "Okonomiyaki Recipe," the category might be "Cuisine," and the tags might be "Japan" and "Local Cuisine." This information is then compiled into a data package.

[0872] Step 3:

[0873] The device creates a data package and sends it to the server, which contains the entered information and metadata and is transmitted via a secure communication protocol (e.g., HTTPS).

[0874] Step 4:

[0875] The server receives the data package sent from the device, analyzes the received data package, and separates the information body (e.g., recipes and videos) from the metadata (e.g., titles and tags).

[0876] Step 5:

[0877] The server categorizes the separated information body and metadata based on appropriate categories and tags and stores them in a database. For example, information on "okonomiyaki recipes" is stored in the "cooking" category.

[0878] Step 6:

[0879] The server creates an index for the newly stored information and indexes it for fast searching. This process makes the information searchable.

[0880] Step 7:

[0881] The emotion engine uses facial recognition and text analysis to recognize the user's emotions. For example, the emotion engine analyzes the user's facial expressions and voice through a dedicated camera or microphone to identify an emotional state such as "relaxed."

[0882] Step 8:

[0883] The device receives the user's emotion data from the emotion engine and transmits it to the server via a secure communication protocol.

[0884] Step 9:

[0885] The server analyzes the received emotional data and reprioritizes information based on the results. For example, a user who is "relaxed" will be given priority for videos that have a relaxing effect.

[0886] Step 10:

[0887] A user inputs a specific keyword or category into the device and sends an information search request to the server. For example, the user inputs "otenjaku" (beanbag).

[0888] Step 11:

[0889] The server analyzes the user's search query, searches the database, and compares the search results with data from the emotion engine to prioritize search results based on the user's emotional state.

[0890] Step 12:

[0891] The server sends the search results to the device. If the user is feeling relaxed, the search results are adjusted so that information with a relaxing effect is displayed at the top.

[0892] Step 13:

[0893] Users can view search results on their devices and use the displayed information to learn more about local culture, for example, how to juggle.

[0894] These specific processing steps enable users to effectively input, save, search, and view local cultural information, and further enable the provision of customized information based on emotions.

[0895] (Application example 2)

[0896] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0897] Modern society demands systems that can efficiently input, store, search, and browse information about local culture. However, conventional systems have difficulty providing information that adapts to the user's emotions. For example, the optimal information varies depending on the user's emotions, such as when they want to relax or when they are excited. However, only uniform information is provided. Therefore, there is a need for information that can be customized based on the user's emotions.

[0898] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting information on regional culture, means for transmitting the input information together with metadata to the server, means for the server to store the transmitted information in a database and classify it, means for the user to search and view the stored information, means for acquiring emotion data using an emotion engine that recognizes the user's emotions, and means for customizing information to be provided to the user based on the emotion data. This makes it possible to provide personalized information according to the user's emotions.

[0899] "Regional cultural information" refers to information about events, festivals, cuisine, traditional games, etc. related to a particular region.

[0900] "Means of input" refers to the means by which a user inputs information into the system, and includes methods such as text input, image upload, and voice input.

[0901] "Means for transmitting to the server" refers to a method for transmitting information input from a user terminal to the server via a network, and includes internet connection, wireless communication, and the like.

[0902] "Means for storing and categorizing" refers to the way in which the server stores the received information in a database and organizes it by categories and tags.

[0903] "Means for searching and viewing" refers to the interface or functionality that allows a user to search the database for stored information and view the results on a device.

[0904] An "emotion engine" refers to algorithms or software that identify and acquire data on a user's emotions through facial recognition, voice analysis, and text analysis.

[0905] "Emotion data" refers to information about the user's emotions acquired by the emotion engine, and includes, for example, "joy," "sadness," and "surprise."

[0906] "Means for customization" refers to methods or processes for optimizing and personalizing information provided to users based on acquired emotional data.

[0907] "Product information" refers to detailed product information provided in the virtual store, including product name, price, description, and image.

[0908] This invention combines a system for inputting, saving, searching, and viewing regional cultural information with an emotion engine that recognizes the user's emotions, thereby providing information based on the user's emotions. This system is primarily composed of four main components: a user terminal, a server, a database, and an emotion engine.

[0909] ● System Overview

[0910] The system is based on a series of processes in which users input information about local culture and then send that information to a server. Specifically, the process involves the following steps:

[0911] 1. User Device

[0912] Users can input information about their local culture, events, and pastimes from their devices in a variety of ways, including text entry, image uploading, and video recording.

[0913] The device automatically sets metadata such as appropriate titles, descriptions, categories, and tags for the information entered.

[0914] The terminal creates a data package of information and sends it to the server.

[0915] 2. Server

[0916] The server receives the data packages from the devices and analyzes and classifies the information and metadata.

[0917] The server classifies the information with appropriate categories and tags and stores it in a database.

[0918] The server creates an index of the newly stored information and makes it searchable.

[0919] 3. Emotion Engine

[0920] The emotion engine uses facial recognition, voice analysis, and text analysis to identify the user's current emotion.

[0921] The terminal receives the user's emotion data from the emotion engine and transmits it to the server.

[0922] The server analyzes the emotional data and prepares to provide the user with information on the local culture that is most appropriate for them.

[0923] 4. Searching and browsing information

[0924] The user enters specific keywords or categories into the device and sends a search request to the server.

[0925] The server analyzes the user's search query and searches the database.

[0926] The server compares the search results with data from the emotion engine and sets priorities.

[0927] The server sends the organized search results to the terminal, where the user can view them.

[0928] ● Hardware and software used

[0929] The system of the present invention utilizes the following hardware and software:

[0930] Hardware

[0931] Head-mounted display (HMD)

[0932] Smart Glasses

[0933] Webcam

[0934] software

[0935] OpenCV: Used for face recognition

[0936] EmotionRecognition: Emotion analysis library

[0937] ProductDatabase: Product database

[0938] RecommendationEngine: Product recommendation engine

[0939] ● Examples of operations and prompt statements

[0940] For example, if a user searches for "traditional hand games," the user can input "beanbag" and the device will send this query to the server. If the emotion engine recognizes that the user is relaxed, the server will prioritize providing images and information related to "beanbag" that have a relaxing effect.

[0941] Another example of a prompt might be:

[0942] If you type "I feel happy," you'll see recommendations for fun products (e.g., new board games).

[0943] If you enter "feeling angry," products with a relaxing effect (e.g., scented candles) will be recommended.

[0944] This system allows users to receive personalized information tailored to their specific emotions, enabling them to learn and experience local culture more effectively and comfortably.

[0945] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0946] Step 1:

[0947] The user enters information and sends it to the terminal

[0948] A user uses a terminal to input information about local culture in the form of text, images, audio, video, etc. The terminal automatically sets appropriate metadata (e.g., title, description, category, tags, etc.) for the input information. This information packet is organized into a data package and sent to the server.

[0949] Input: User-entered local cultural information and metadata

[0950] Data processing and calculation: Composing input information into a data package with metadata

[0951] Output: Composed data package

[0952] Step 2:

[0953] The server receives the information, stores it in a database, and categorizes it.

[0954] The server receives the data package sent from the device, analyzes and separates the information and metadata, classifies the analyzed information with appropriate categories and tags, and stores it in a database. At the same time, a search index is also generated.

[0955] Input: Data package

[0956] Data processing and calculation: information and metadata analysis, separation, categorization, storage, and indexing

[0957] Output: Information stored in the database and the generated search index

[0958] Step 3:

[0959] The device acquires the user's emotional data from the emotion engine and sends it to the server.

[0960] The device uses its built-in emotion engine to identify the user's current emotion through facial recognition and voice analysis, and this emotion data is sent to the server as a data package.

[0961] Input: User's facial image and voice data

[0962] Data processing and calculation: Emotion identification by emotion engine, generation of data packages

[0963] Output: Emotion data package

[0964] Step 4:

[0965] The server analyzes the emotion data and prepares to provide information.

[0966] The server analyzes the transmitted emotional data and customizes the information provided based on the user's emotional state. For example, if the user is in a relaxed state, it will prioritize information that has a relaxing effect.

[0967] Input: Emotion data package

[0968] Data processing and calculation: Emotional data analysis, customization of information provision

[0969] Output: Preparing customized informational presentations

[0970] Step 5:

[0971] The user enters specific keywords or categories into the device and sends a search request.

[0972] The user enters specific keywords and categories into the device to send a search request, which is then sent to the server.

[0973] Input: User's search query (keywords or category)

[0974] Data processing and calculation: receiving search queries and generating requests

[0975] Output: Search request to the server

[0976] Step 6:

[0977] The server processes search requests and prioritizes them based on sentiment data.

[0978] The server receives a user's search query, searches the database, matches the search results with sentiment data, and sets priorities based on the user's sentiment.

[0979] Input: search query, sentiment data

[0980] Data processing and calculation: database search, matching with emotion data, setting priorities

[0981] Output: Prioritized search results

[0982] Step 7:

[0983] The server sends the organized search results to the device, where the user can view them.

[0984] The server sends the prioritized search results to the terminal, where the user can browse them to obtain the most appropriate local cultural information according to their emotional state.

[0985] Input: prioritized search results

[0986] Data processing and calculation: formatting and sending results

[0987] Output: Search results displayed on the device

[0988] This series of steps results in a system that allows users to efficiently search and view information about local culture that is optimized to suit their own emotions.

[0989] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0990] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0991] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0992] [Third embodiment]

[0993] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0994] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0995] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0996] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0997] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0998] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0999] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1000] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1001] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1002] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1003] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1004] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1005] The present invention relates to a system for recording, preserving, and preventing the erosion of information about local culture, food, and games (hand games and songs). This system has a series of functions that allow users to input information, send it to a server, and the server classifies, preserves, and provides the information. Specific embodiments for implementing the present invention are described below.

[1006] System Overview

[1007] This system consists of three main components: a user terminal, a server, and a database. Users use their terminals to input information about local culture and send it to the server. The server then appropriately categorizes the received information and stores it in the database. Users can also search for information using specific keywords or categories and view the information retrieved from the database.

[1008] Program processing

[1009] When a user enters information and sends it to the server

[1010] Subject: User and Device

[1011] 1. Users input information about their local culture, events, and games on their devices. For example, they can upload a recipe for a particular local dish or a video of a traditional hand game.

[1012] 2. The device converts the input information into a video file or text file and adds metadata such as title, description, category, and tags.

[1013] 3. The device makes a request to send the converted information and metadata to the server.

[1014] When the server stores and organizes information

[1015] Subject: Server

[1016] 1. The server receives an upload request from the device and retrieves the submitted data and metadata.

[1017] 2. The server stores the received information in a database and categorizes it into appropriate categories and tags based on the metadata.

[1018] 3. The server indexes the newly stored information and makes it available for searching.

[1019] When a user searches or browses information

[1020] Subject: User, device, server

[1021] 1. The user searches from their device by specifying specific keywords or categories.

[1022] 2. The device sends the user's search query to the server.

[1023] 3. The server searches the database based on the search query, retrieves and organizes the relevant information.

[1024] 4. The server sends the search results to the device in an appropriate format.

[1025] 5. The user views the search results on their device and checks for more information.

[1026] Specific examples

[1027] For example, let's consider a case where a user searches for "traditional hand games." When the user enters "otedama" and presses the search button, the device sends this query to the server. The server retrieves video files and text information related to "otedama" from the database, organizes the results, and sends them back to the device. The user can view the results and learn how to play the traditional hand game "otedama."

[1028] summary

[1029] This system aims to prevent the erosion of culture by recording and storing local culture, food, and entertainment, and making that data easily searchable and viewable. The basic functions required for this are implemented between user devices and a server. This allows anyone to easily learn about local traditions and culture and pass them on to the next generation.

[1030] The processing flow will be explained below.

[1031] When a user uploads information

[1032] Step 1:

[1033] Users input information about local culture (e.g., video files or text information) from their own devices (smartphones, PCs, etc.).

[1034] Step 2:

[1035] The system sets metadata such as an appropriate title, description, category, and tag for the information entered by the user.

[1036] Step 3:

[1037] The user presses the "upload" button to issue a request to the terminal to transmit information.

[1038] Step 4:

[1039] The device verifies the information entered and creates a data package containing the necessary metadata.

[1040] Step 5:

[1041] The terminal requests that a data package be sent to the server.

[1042] When the server stores and organizes information

[1043] Step 6:

[1044] The server receives the data package sent from the terminal.

[1045] Step 7:

[1046] The server parses and separates the information and metadata from the received data packages.

[1047] Step 8:

[1048] The server prepares to store the parsed information in a database.

[1049] Step 9:

[1050] The server categorizes the information based on appropriate categories and tags and stores it in a database.

[1051] Step 10:

[1052] The server indexes the newly stored information and makes it searchable.

[1053] Step 11:

[1054] The server returns a confirmation message to the device that the information has been saved and classified.

[1055] When a user searches or browses information

[1056] Step 12:

[1057] A user enters a search query on their device, specifying specific keywords and categories.

[1058] Step 13:

[1059] The terminal analyzes the user's search query and sends a search request to the server.

[1060] Step 14:

[1061] The server searches the database based on the received search query and retrieves the relevant information.

[1062] Step 15:

[1063] The server properly organizes the acquired information and sends it to the terminal.

[1064] Step 16:

[1065] The user views the search results on their device and checks for more information.

[1066] When providing recipes or event information

[1067] Step 17:

[1068] A user requests information about a particular local event or dish.

[1069] Step 18:

[1070] The terminal analyzes the user's request and sends it to the server.

[1071] Step 19:

[1072] The server retrieves relevant information from the database, organizes it, and provides it to the user's terminal.

[1073] Step 20:

[1074] The user views the provided information on the device and deepens their understanding of the events and cuisine.

[1075] Through these processing steps, the system is able to effectively record and preserve local culture and provide it to users in an easily accessible form.

[1076] Example 1

[1077] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1078] Information about a region's unique culture, traditions, food, and pastimes is a valuable asset that should be passed on to future generations. However, if this information is not properly recorded and preserved, it is likely to fade away and be lost. Furthermore, current systems make it difficult to classify input information, making it difficult to search and view. Therefore, there is a need for a system that enables efficient recording, classification, search, and viewing of regional cultural information.

[1079] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1080] In this invention, the server includes means for inputting regional cultural information, means for transmitting the input information together with metadata, means for the server to store the transmitted information in a database and classify it based on the metadata, means for the server to create an index of the stored information and organize the information in a searchable form, means for a user to search for information based on specific keywords or categories, and means for the server to acquire, organize, and transmit the relevant information in an appropriate format to a terminal, and means for a user to view search results and check detailed information. This enables efficient recording, classification, search, and viewing of regional cultural information.

[1081] "Regional culture" is a general term for cultural elements such as events, food, and games that have traditionally been passed down in a particular region.

[1082] "Means for inputting information" refers to a device or interface that allows a user to input information about local culture in digital form.

[1083] "Metadata" refers to additional information such as title, description, category, and tag that is added to input information.

[1084] "Means for sending to the server" refers to the communications technology or protocol used to transfer digital information from the terminal to the server.

[1085] "Means of storing in a database" refers to the systems and technologies used by the server to permanently store the digital information it receives.

[1086] "Indexing means" refers to systems or techniques that generate tables of contents or indexes to facilitate efficient searching of stored digital information.

[1087] "Means of making information searchable" refers to database management and search engine technologies that enable quick and accurate responses to user search queries.

[1088] A "search query" refers to a keyword or phrase that a user enters when searching for information.

[1089] "Methods of organization" refers to techniques and processes for systematically sorting acquired information based on certain criteria.

[1090] "Means for transmitting to the terminal in an appropriate format" refers to the communication technology and protocols that allow the server to provide organized information to the user terminal in an appropriate format.

[1091] "Means for viewing search results" refers to the interface and display technology that allows users to visually check search results using a terminal.

[1092] This invention is a system that effectively records, stores, and easily searches and views information about local culture, food, and entertainment. This system sends information entered by users to a server, which then classifies and stores the information in a database, allowing users to easily search and view it.

[1093] System configuration

[1094] The system mainly consists of the following components:

[1095] 1. User Device

[1096] 2. Server

[1097] 3. Database

[1098] Entering information

[1099] Users input information about their local culture, events, and games into their devices. Specific examples include recipes for local dishes and videos of traditional hand games. The device saves this information as video or text files and adds metadata such as titles, descriptions, categories, and tags.

[1100] Sending data

[1101] The device uses a file upload function to send the converted information and metadata to the server, for example, by using an HTTP POST request to send the data to the server, which receives the request and stores the information and metadata in a database.

[1102] Data classification and storage

[1103] The server stores the received information in a database and categorizes it appropriately based on the metadata. Specifically, it stores the video file in storage and stores the metadata (title, description, category, tags, etc.) in the appropriate tables in the database.

[1104] Making the site searchable

[1105] The server then indexes the newly stored information and makes it searchable, which includes using a full-text search engine to generate the index.

[1106] Searching and browsing information

[1107] Users search for information by specifying specific keywords or categories on their devices. At that time, the device sends the user's search query to the server. The server searches the database based on the search query, retrieves and organizes the relevant information, and then sends it to the device in an appropriate format (for example, JSON format). Finally, the user can view the search results and check detailed information.

[1108] Example prompt

[1109] As a concrete example, let's consider the case where a user searches for "traditional hand games." The user enters "otedama" and presses the search button. The device sends this query to the server via an HTTP GET request. The server executes an SQL query, retrieves video files and text information related to "otedama" from the database, and returns the results organized in JSON format to the device. The device displays the search results in a list, and the user can view detailed information about "how to play otedama" in a pop-up display.

[1110] Example prompts to input to a generative AI model:

[1111] "I would like to know how to play otedama, a traditional Japanese hand game. What are the steps and what materials are needed?"

[1112] This system will enable efficient storage, classification, searching, and viewing of local cultural information, and is expected to help prevent culture from fading away.

[1113] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1114] Step 1:

[1115] The user enters information

[1116] Description: Users input information about their local culture, events, and pastimes from their devices.

[1117] Input: The user enters free-form text information and video data into the input form.

[1118] Data processing / calculation: Text data is stored as is via the input device, and video data is stored in digital format.

[1119] Output: Input information temporarily stored on the device.

[1120] Specific operation: The user enters a description of a traditional hand game into a form, then takes a video of the game using their smartphone and uploads it.

[1121] Step 2:

[1122] Device adds metadata

[1123] Description: The device adds metadata to the information entered, such as title, description, category, and tags.

[1124] Input: Text and video data provided by the user in Step 1.

[1125] Data processing / calculation: The process of converting data into a storage format for databases and adding metadata.

[1126] Output: Information with metadata added.

[1127] Specific operation: For a video titled "How to play with beanbags," the device automatically sets the title "Beanbags," the description "Traditional Japanese hand game," the category "Game," and the tags "Hand game, beanbags."

[1128] Step 3:

[1129] The device sends data to the server

[1130] Description: The device sends the converted information and metadata to the server.

[1131] Input: Information with metadata.

[1132] Data processing / computation: Packaging data and generating transmission requests.

[1133] Output: The data sent to the server.

[1134] What it does: Sends data to the server as an HTTP POST request, including metadata in the data packet.

[1135] Step 4:

[1136] The server receives and stores the data

[1137] Description: The server receives upload requests from devices and stores the submitted data and metadata in a database.

[1138] Input: Data and metadata sent from the device.

[1139] Data processing / calculation: Inserting data into the database and saving it to storage.

[1140] Output: Information stored in a database.

[1141] Specific operation: The video file is stored in server-side storage, and the metadata is inserted into the appropriate table in the database.

[1142] Step 5:

[1143] The server categorizes and organizes the information

[1144] Description: The server categorizes information based on metadata, indexes newly stored information, and makes it searchable.

[1145] Input: Information and metadata stored in a database.

[1146] Data processing / computation: Classification and indexing of information using indexing algorithms.

[1147] Output: Indexed and searchable information.

[1148] Specific operation: Tags such as "hand games" and "beanbags" are added to the category "play," and an index is created using a full-text search engine.

[1149] Step 6:

[1150] The user enters a search query

[1151] Description: Users search for information from their devices by specifying specific keywords or categories.

[1152] Input: User-specified keywords and categories.

[1153] Data processing / calculation: Generating search queries entered by users.

[1154] Output: The search query sent from the device to the server.

[1155] Specific behavior: The user enters "Beanbag" into the search form and presses the search button.

[1156] Step 7:

[1157] The device sends the search query to the server

[1158] Description: The device sends the user's search query to the server.

[1159] Input: The search query specified by the user.

[1160] Data processing / computation: Generating and submitting search requests.

[1161] Output: The search query sent to the server.

[1162] Specific behavior: Sends a search query to the server via an HTTP GET request.

[1163] Step 8:

[1164] The server searches the database

[1165] Description: The server searches the database based on the search query, retrieves and organizes relevant information.

[1166] Input: The search query sent from the device.

[1167] Data processing / calculation: Execute SQL queries and extract relevant information from the database.

[1168] Output: Organized search results.

[1169] Specific operation: The server executes an SQL query to retrieve video files and text information related to "Beanbag."

[1170] Step 9:

[1171] The server sends the search results to the device.

[1172] Description: The server sends the search results in the appropriate format to the device.

[1173] Input: Organized search results.

[1174] Data processing / calculation: Packaging search results in an appropriate format such as JSON.

[1175] Output: Search results sent to your device.

[1176] Specific operation: The server returns the search results to the device in JSON format.

[1177] Step 10:

[1178] Your device will display the search results.

[1179] Description: The device displays the search results received from the server in the user interface.

[1180] Input: Search results sent from the server.

[1181] Data processing / calculation: Search results are displayed in list format, and detailed information about the items selected by the user is displayed in a pop-up.

[1182] Output: Search results displayed in the user interface.

[1183] Specific behavior: The device displays the search results in a list format, and the user checks detailed information about "How to play beanbag."

[1184] (Application example 1)

[1185] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1186] Information about regional culture easily fades with time, making it particularly difficult to pass on to younger generations. Furthermore, manually entering information is time-consuming, and the lack of a voice input method makes it difficult to use. The objective of this invention is to provide a system that efficiently collects regional cultural information and utilizes voice recognition technology to allow users to easily input, search, and view information.

[1187] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1188] In this invention, the server includes means for inputting regional culture information, means for transmitting the input information together with metadata to the server, means for the server to store the transmitted information in a database and classify it, means for a user to input regional culture information by voice using voice recognition technology, means for converting the voice-input information into text data, and means for a user to search and view the stored information. This allows users to utilize voice input to easily record information and pass on regional culture.

[1189] "Regional culture" refers to cultural characteristics including events, food, games, traditions, customs, and practices unique to a region.

[1190] "Means for inputting information" means the means by which a user can directly or indirectly input data into a system, including voice, text, video, or other formats.

[1191] "Means for sending to the server" refers to the communication means or protocol for converting the input information into a certain format and sending it to the server.

[1192] "Server" refers to a computer system that processes, stores, and appropriately manages received information.

[1193] A "database" refers to a structured collection of data that stores information and makes it easy to retrieve when needed.

[1194] A "means for classification" refers to a means for organizing input information based on specific criteria or tags.

[1195] "Speech recognition technology" refers to technology for converting voice data into text data.

[1196] The term "means for inputting regional culture information by voice" refers to a means for a user to input regional culture information into the system by voice.

[1197] "Means for converting into text data" refers to means for converting information input by voice into text format.

[1198] "Means for searching and viewing information" refers to means by which a user can search for stored information based on specific keywords or categories and display the results.

[1199] This invention is a system for efficiently collecting, recording, and storing information about local culture, food, entertainment, etc., and preventing it from fading away. The invention is composed of three main components: a user terminal, a server, and a database. Specific embodiments of the invention are described below.

[1200] Input information via user terminal

[1201] First, the user uses voice recognition technology to input information about local culture. Voice recognition software such as Google Speech-to-Text API is installed on the user's device, and the voice data is converted into text data. For example, if a user says, "The local dish of this region is okonomiyaki," the content is converted into text data. At the same time, metadata such as title, description, category, and tags are also generated.

[1202] Data processing and storage by server

[1203] Text data and metadata generated on user devices are sent via the Internet to a server. The server receives this data and stores it in a database, where it is automatically categorized and organized based on the metadata. The server also creates an index of the newly stored information, making it instantly searchable.

[1204] Users search for and view information

[1205] Users can search for information by specifying specific keywords or categories. When a user enters a search query using a terminal, the query is sent to the server. The server searches for relevant information from the database and sends the results to the user's terminal. For example, if a user searches for "traditional hand games," the server will provide the user with video files and text information related to "beanbags."

[1206] Hardware and software used

[1207] The hardware used includes user devices (such as smartphones), servers, database servers, etc. The software used includes the Google Speech-to-Text API, a voice recognition technology, the HTTP communication protocol, and a database management system.

[1208] For example:

[1209] As users tour the factory, they can talk about local cultures, and their voices are recorded by their smartphones. A speech recognition system converts the speech into text, adds metadata, and sends it to a server, where it can be stored, categorized, and made searchable by other users if desired.

[1210] Prompt Sentence Examples

[1211] Here are some answers to the open-ended question that begins, "What is the local cuisine of your region?"

[1212] Example: Question: "What is the local dish of this region?"

[1213] Answer: "The local dish of this area is okonomiyaki."

[1214] As a result, the system of the present invention provides users with the ability to easily record, search, and view information about local culture, making it possible to prevent local culture from fading away.

[1215] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1216] Step 1:

[1217] The user inputs information about local culture into the device by voice. Using voice recognition technology, the user says, "The local dish of this region is okonomiyaki." The device collects this voice data. As a result, the voice data is obtained as input data.

[1218] Step 2:

[1219] Convert voice data into text data. The input voice data is converted into text data using the Google Speech-to-Text API installed on the device. The output of this process is the text data, "The local dish of this region is okonomiyaki."

[1220] Step 3:

[1221] Metadata is added to the text data. The device generates and adds metadata such as title, description, category, and tag to the converted text data. For example, metadata such as title "Local Cuisine," category "Meal," and tag "Okonomiyaki" are added. This process outputs data that combines the text data and metadata.

[1222] Step 4:

[1223] The text data with metadata is sent to the server. The terminal sends the generated data to the server using the HTTP communication protocol. In this step, data is sent from the user terminal to the server.

[1224] Step 5:

[1225] The server stores and classifies the received data. The server stores the received data in a database and automatically classifies it based on metadata. For example, it may be classified with the tag "Okonomiyaki" under the category "Food." The output is the classified data stored in the database.

[1226] Step 6:

[1227] A user searches for information on a device. The user enters a search query (e.g., "local cuisine"), and the device sends this search query to a server. The input data is the search query, and the output is a request for search results.

[1228] Step 7:

[1229] The server searches the database and organizes the results. The server searches the database based on the received search query and organizes the relevant information. The input to this process is the search query, and the output is the search result data.

[1230] Step 8:

[1231] The server sends the search results. The server sends the organized search results to the user terminal. The output of this process is the search result data sent to the user terminal.

[1232] Step 9:

[1233] The user views the search results on their device and checks detailed information. The user then views the search results sent from the server via their device. As a result, the user can view the recipe for "okonomiyaki" and related information.

[1234] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1235] The present invention combines a system for inputting, saving, searching, and viewing regional cultural information with an emotion engine that recognizes user emotions, and provides information based on the user's emotions. This system is composed of four main components: a user terminal, a server, a database, and the emotion engine. Specific embodiments for implementing the present invention are described below.

[1236] System Overview

[1237] This system has a series of functions in which users input information, send it to a server, and the server stores, categorizes, and provides it to users as needed.Furthermore, it uses an emotion engine to analyze the user's emotions and customize the information displayed based on the results.

[1238] Program processing

[1239] When a user enters information and sends it to the server

[1240] Subject: User and Device

[1241] 1. Users input information about their local culture, events, and games on their devices. For example, they can upload a recipe for a particular local dish or a video of a traditional hand game.

[1242] 2. The device sets appropriate metadata such as title, description, category, and tags for the entered information.

[1243] 3. The device creates a data package of the information and sends it to the server.

[1244] When the server stores and organizes information

[1245] Subject: Server

[1246] 1. The server receives the data package from the device and analyzes and separates the information and metadata.

[1247] 2. The server classifies the information with appropriate categories and tags and stores it in a database.

[1248] 3. The server indexes the newly stored information and makes it searchable.

[1249] When the emotion engine recognizes the user's emotions

[1250] Subject: Emotion engine and device

[1251] 1. The emotion engine uses facial recognition and text analysis to identify the user's current emotion.

[1252] 2. The device receives the user's emotion data from the emotion engine and sends it to the server.

[1253] 3. The server analyzes the emotion data and prepares to provide the user with information on the local culture that best suits them.

[1254] When a user searches or browses information

[1255] Subject: User, device, server

[1256] 1. The user enters specific keywords or categories into the device and sends a search request to the server.

[1257] 2. The server analyzes the user's search query and searches the database.

[1258] 3. The server compares the search results with the emotion engine data and sets priorities.

[1259] 4. The server sends the organized search results to the device, where the user can view them.

[1260] Specific examples

[1261] For example, if a user searches for "traditional hand games," the user can enter "otedama" (beanbag) and the device will send this query to the server. If the emotion engine recognizes that the user is relaxed, the server will prioritize providing videos and information related to "otedama" that have a relaxing effect. The user can browse the results, learn how to play the traditional hand game "otedama," and obtain the most appropriate information according to their emotional state at the time.

[1262] summary

[1263] This system not only records and preserves local culture, but also uses an emotion engine to provide information that responds to the user's emotions, allowing users to learn about and experience local culture more effectively and comfortably.

[1264] The processing flow will be explained below.

[1265] When a user uploads information

[1266] Step 1:

[1267] Users input information about local culture (e.g., video files or text information) from their own devices (smartphones, PCs, etc.).

[1268] Step 2:

[1269] The system sets metadata such as an appropriate title, description, category, and tag for the information entered by the user.

[1270] Step 3:

[1271] The user presses the "upload" button to issue a request to the terminal to transmit information.

[1272] Step 4:

[1273] The device verifies the information entered and creates a data package containing the necessary metadata.

[1274] Step 5:

[1275] The terminal requests that a data package be sent to the server.

[1276] When the server stores and organizes information

[1277] Step 6:

[1278] The server receives the data package sent from the terminal.

[1279] Step 7:

[1280] The server parses and separates the information and metadata from the received data packages.

[1281] Step 8:

[1282] The server prepares to store the parsed information in a database.

[1283] Step 9:

[1284] The server categorizes the information based on appropriate categories and tags and stores it in a database.

[1285] Step 10:

[1286] The server indexes the newly stored information and makes it searchable.

[1287] Step 11:

[1288] The server returns a confirmation message to the device that the information has been saved and classified.

[1289] When the emotion engine recognizes the user's emotions

[1290] Step 12:

[1291] The emotion engine uses facial recognition and text analysis to recognize the user's current emotions.

[1292] Step 13:

[1293] The terminal receives the user's emotion data from the emotion engine and transmits it to the server.

[1294] When a user searches or browses information

[1295] Step 14:

[1296] A user enters a search query on their device, specifying specific keywords and categories.

[1297] Step 15:

[1298] The terminal analyzes the user's search query and sends a search request to the server.

[1299] Step 16:

[1300] The server searches the database based on the received search query and retrieves the relevant information.

[1301] Step 17:

[1302] The server compares the search results with the emotion data and preferentially selects information that matches the user's emotion.

[1303] Step 18:

[1304] The server sends the organized search results to the device.

[1305] Step 19:

[1306] The user views the search results on their device and checks for more information.

[1307] When providing recipes or event information

[1308] Step 20:

[1309] A user requests information about a particular local event or dish.

[1310] Step 21:

[1311] The terminal analyzes the user's request and sends it to the server.

[1312] Step 22:

[1313] The server retrieves relevant information from the database, organizes it, and provides it to the user's terminal.

[1314] Step 23:

[1315] The user views the provided information on the device and deepens their understanding of the events and cuisine.

[1316] Using an emotion engine to customize information

[1317] Step 24:

[1318] The emotion engine records the user's emotion history and uses it to help with the next information search.

[1319] Step 25:

[1320] The emotion engine customizes the default display priority based on the user's current emotions.

[1321] This allows the system to effectively record and preserve local culture and provide optimal information based on the user's feelings, allowing users to comfortably learn and experience cultural information.

[1322] Example 2

[1323] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1324] Conventional regional cultural information systems allowed users to input, save, search, and view information, but lacked the ability to provide information based on the user's emotions. This made it difficult for users to obtain information that matched their emotional state, resulting in a decrease in satisfaction. Furthermore, conventional systems were limited to providing simple text and video data, making it difficult to gain a deep understanding of regional culture or to learn effectively.

[1325] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1326] In this invention, the server includes means for inputting regional culture information, means for transmitting the input information together with metadata to a transmission device, means for the transmission device to store and classify the transmitted information in a storage device, means for a user to search and view the stored information, and means for an emotion recognition device to analyze the user's emotions and provide information based on the user's emotions. This makes it possible to provide optimal regional culture information according to the user's emotional state, improving user satisfaction and enabling effective learning of regional culture.

[1327] "Local culture" refers to the collection of traditions, customs, events, food, games, etc. that are unique to a particular region or community.

[1328] "Means for inputting information" refers to an interface that allows users to provide data (text, video, images, etc.) related to local culture to the system.

[1329] "Metadata" refers to data (title, description, category, tags, etc.) that describes the attributes and structure of the information itself.

[1330] "Transmitting device" refers to a device or system for transmitting information and metadata input from a user terminal to a server.

[1331] "Storage" refers to a database or storage that stores the transmitted information and metadata so that it can be easily searched and classified later.

[1332] "Classification methods" refers to the process of organizing received information into categories or tags to make it easily searchable.

[1333] "Search and browsing means" refers to an interface that allows a user to search for stored information using specific keywords or categories and view the results.

[1334] An "emotion recognition device" refers to technology that identifies a user's emotions through facial recognition or text analysis, and customizes information based on that.

[1335] "Means for providing information based on user emotions" refers to a process for providing the most appropriate local cultural information by taking into account the perceived emotional state of the user.

[1336] This invention combines a system for inputting, saving, searching, and viewing regional cultural information with an emotion engine that recognizes the user's emotions, and provides information based on the user's emotions. This system consists of four main components: a user terminal, a server, a database, and an emotion engine.

[1337] System Overview

[1338] This system has a series of functions in which the user inputs information, sends it to a server, and the server stores, categorizes, and provides it to the user as needed. Furthermore, it uses an emotion engine to analyze the user's emotions and customizes and displays information based on the results. Specifically, it uses the following hardware and software:

[1339] User devices: smartphones, tablets, computers, etc.

[1340] Server: Cloud server (AWS, Google Cloud, Microsoft Azure, etc.)

[1341] Database: MySQL, PostgreSQL, MongoDB, etc.

[1342] Emotion engine: Commonly used emotion recognition technologies (e.g., Microsoft Azure Emotion Recognition API, Google Cloud Vision API, etc.)

[1343] Specific software: Django (Python framework), React (front-end)

[1344] The information and metadata entered by the user through the device are sent as a series of data packages to the server, which analyzes, classifies, and stores them in a database. The emotion engine also recognizes the user's emotional state and provides information based on that emotion.

[1345] Specific examples

[1346] For example, if a user searches for "traditional hand games," the user can input "otedama" into the device and the query will be sent to the server. If the emotion engine recognizes that the user is relaxed, the server can provide images and information related to "otedama" that have a relaxing effect preferentially. The user can browse these results and obtain the most appropriate information while learning how to play the traditional hand game "otedama." Also, the following prompt sentence can be used:

[1347] "Please tell me about a local festival that suits my mood today."

[1348] "I'd like to know some local recipes. Do you have any recommendations?"

[1349] Results of the implementation

[1350] This system not only allows users to efficiently record, save, search, and view information about local culture, but also utilizes an emotion engine to provide optimal information according to the user's emotional state, thereby improving user satisfaction and enabling effective learning and experience of local culture.

[1351] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1352] Below, a specific explanation is given, broken down into the processing steps of the system.

[1353] Step 1:

[1354] A user inputs information about the culture of their region (e.g., a recipe for a traditional dish or a video of a hand game) into a terminal. The information input can be text, images, or videos. Specifically, consider the case where a user inputs an "okonomiyaki recipe."

[1355] Step 2:

[1356] The device automatically assigns appropriate metadata such as title, description, category, and tags to the information entered. For example, the title might be "Okonomiyaki Recipe," the category might be "Cuisine," and the tags might be "Japan" and "Local Cuisine." This information is then compiled into a data package.

[1357] Step 3:

[1358] The device creates a data package and sends it to the server, which contains the entered information and metadata and is transmitted via a secure communication protocol (e.g., HTTPS).

[1359] Step 4:

[1360] The server receives the data package sent from the device, analyzes the received data package, and separates the information body (e.g., recipes and videos) from the metadata (e.g., titles and tags).

[1361] Step 5:

[1362] The server categorizes the separated information body and metadata based on appropriate categories and tags and stores them in a database. For example, information on "okonomiyaki recipes" is stored in the "cooking" category.

[1363] Step 6:

[1364] The server creates an index for the newly stored information and indexes it for fast searching. This process makes the information searchable.

[1365] Step 7:

[1366] The emotion engine uses facial recognition and text analysis to recognize the user's emotions. For example, the emotion engine analyzes the user's facial expressions and voice through a dedicated camera or microphone to identify an emotional state such as "relaxed."

[1367] Step 8:

[1368] The device receives the user's emotion data from the emotion engine and transmits it to the server via a secure communication protocol.

[1369] Step 9:

[1370] The server analyzes the received emotional data and reprioritizes information based on the results. For example, a user who is "relaxed" will be given priority for videos that have a relaxing effect.

[1371] Step 10:

[1372] A user inputs a specific keyword or category into the device and sends an information search request to the server. For example, the user inputs "otenjaku" (beanbag).

[1373] Step 11:

[1374] The server analyzes the user's search query, searches the database, and compares the search results with data from the emotion engine to prioritize search results based on the user's emotional state.

[1375] Step 12:

[1376] The server sends the search results to the device. If the user is feeling relaxed, the search results are adjusted so that information with a relaxing effect is displayed at the top.

[1377] Step 13:

[1378] Users can view search results on their devices and use the displayed information to learn more about local culture, for example, how to juggle.

[1379] These specific processing steps enable users to effectively input, save, search, and view local cultural information, and further enable the provision of customized information based on emotions.

[1380] (Application example 2)

[1381] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1382] Modern society demands systems that can efficiently input, store, search, and browse information about local culture. However, conventional systems have difficulty providing information that adapts to the user's emotions. For example, the optimal information varies depending on the user's emotions, such as when they want to relax or when they are excited. However, only uniform information is provided. Therefore, there is a need for information that can be customized based on the user's emotions.

[1383] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting information on regional culture, means for transmitting the input information together with metadata to the server, means for the server to store the transmitted information in a database and classify it, means for the user to search and view the stored information, means for acquiring emotion data using an emotion engine that recognizes the user's emotions, and means for customizing information to be provided to the user based on the emotion data. This makes it possible to provide personalized information according to the user's emotions.

[1384] "Regional cultural information" refers to information about events, festivals, cuisine, traditional games, etc. related to a particular region.

[1385] "Means of input" refers to the means by which a user inputs information into the system, and includes methods such as text input, image upload, and voice input.

[1386] "Means for transmitting to the server" refers to a method for transmitting information input from a user terminal to the server via a network, and includes internet connection, wireless communication, and the like.

[1387] "Means for storing and categorizing" refers to the way in which the server stores the received information in a database and organizes it by categories and tags.

[1388] "Means for searching and viewing" refers to the interface or functionality that allows a user to search the database for stored information and view the results on a device.

[1389] An "emotion engine" refers to algorithms or software that identify and acquire data on a user's emotions through facial recognition, voice analysis, and text analysis.

[1390] "Emotion data" refers to information about the user's emotions acquired by the emotion engine, and includes, for example, "joy," "sadness," and "surprise."

[1391] "Means for customization" refers to methods or processes for optimizing and personalizing information provided to users based on acquired emotional data.

[1392] "Product information" refers to detailed product information provided in the virtual store, including product name, price, description, and image.

[1393] This invention combines a system for inputting, saving, searching, and viewing regional cultural information with an emotion engine that recognizes the user's emotions, thereby providing information based on the user's emotions. This system is primarily composed of four main components: a user terminal, a server, a database, and an emotion engine.

[1394] ● System Overview

[1395] The system is based on a series of processes in which users input information about local culture and then send that information to a server. Specifically, the process involves the following steps:

[1396] 1. User Device

[1397] Users can input information about their local culture, events, and pastimes from their devices in a variety of ways, including text entry, image uploading, and video recording.

[1398] The device automatically sets metadata such as appropriate titles, descriptions, categories, and tags for the information entered.

[1399] The terminal creates a data package of information and sends it to the server.

[1400] 2. Server

[1401] The server receives the data packages from the devices and analyzes and classifies the information and metadata.

[1402] The server classifies the information with appropriate categories and tags and stores it in a database.

[1403] The server creates an index of the newly stored information and makes it searchable.

[1404] 3. Emotion Engine

[1405] The emotion engine uses facial recognition, voice analysis, and text analysis to identify the user's current emotion.

[1406] The terminal receives the user's emotion data from the emotion engine and transmits it to the server.

[1407] The server analyzes the emotional data and prepares to provide the user with information on the local culture that is most appropriate for them.

[1408] 4. Searching and browsing information

[1409] The user enters specific keywords or categories into the device and sends a search request to the server.

[1410] The server analyzes the user's search query and searches the database.

[1411] The server compares the search results with data from the emotion engine and sets priorities.

[1412] The server sends the organized search results to the terminal, where the user can view them.

[1413] ● Hardware and software used

[1414] The system of the present invention utilizes the following hardware and software:

[1415] Hardware

[1416] Head-mounted display (HMD)

[1417] Smart Glasses

[1418] Webcam

[1419] software

[1420] OpenCV: Used for face recognition

[1421] EmotionRecognition: Emotion analysis library

[1422] ProductDatabase: Product database

[1423] RecommendationEngine: Product recommendation engine

[1424] ● Examples of operations and prompt statements

[1425] For example, if a user searches for "traditional hand games," the user can input "beanbag" and the device will send this query to the server. If the emotion engine recognizes that the user is relaxed, the server will prioritize providing images and information related to "beanbag" that have a relaxing effect.

[1426] Another example of a prompt might be:

[1427] If you type "I feel happy," you'll see recommendations for fun products (e.g., new board games).

[1428] If you enter "feeling angry," products with a relaxing effect (e.g., scented candles) will be recommended.

[1429] This system allows users to receive personalized information tailored to their specific emotions, enabling them to learn and experience local culture more effectively and comfortably.

[1430] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1431] Step 1:

[1432] The user enters information and sends it to the terminal

[1433] A user uses a terminal to input information about local culture in the form of text, images, audio, video, etc. The terminal automatically sets appropriate metadata (e.g., title, description, category, tags, etc.) for the input information. This information packet is organized into a data package and sent to the server.

[1434] Input: User-entered local cultural information and metadata

[1435] Data processing and calculation: Composing input information into a data package with metadata

[1436] Output: Composed data package

[1437] Step 2:

[1438] The server receives the information, stores it in a database, and categorizes it.

[1439] The server receives the data package sent from the device, analyzes and separates the information and metadata, classifies the analyzed information with appropriate categories and tags, and stores it in a database. At the same time, a search index is also generated.

[1440] Input: Data package

[1441] Data processing and calculation: information and metadata analysis, separation, categorization, storage, and indexing

[1442] Output: Information stored in the database and the generated search index

[1443] Step 3:

[1444] The device acquires the user's emotional data from the emotion engine and sends it to the server.

[1445] The device uses its built-in emotion engine to identify the user's current emotion through facial recognition and voice analysis, and this emotion data is sent to the server as a data package.

[1446] Input: User's facial image and voice data

[1447] Data processing and calculation: Emotion identification by emotion engine, generation of data packages

[1448] Output: Emotion data package

[1449] Step 4:

[1450] The server analyzes the emotion data and prepares to provide information.

[1451] The server analyzes the transmitted emotional data and customizes the information provided based on the user's emotional state. For example, if the user is in a relaxed state, it will prioritize information that has a relaxing effect.

[1452] Input: Emotion data package

[1453] Data processing and calculation: Emotional data analysis, customization of information provision

[1454] Output: Preparing customized informational presentations

[1455] Step 5:

[1456] The user enters specific keywords or categories into the device and sends a search request.

[1457] The user enters specific keywords and categories into the device to send a search request, which is then sent to the server.

[1458] Input: User's search query (keywords or category)

[1459] Data processing and calculation: receiving search queries and generating requests

[1460] Output: Search request to the server

[1461] Step 6:

[1462] The server processes search requests and prioritizes them based on sentiment data.

[1463] The server receives a user's search query, searches the database, matches the search results with sentiment data, and sets priorities based on the user's sentiment.

[1464] Input: search query, sentiment data

[1465] Data processing and calculation: database search, matching with emotion data, setting priorities

[1466] Output: Prioritized search results

[1467] Step 7:

[1468] The server sends the organized search results to the device, where the user can view them.

[1469] The server sends the prioritized search results to the terminal, where the user can browse them to obtain the most appropriate local cultural information according to their emotional state.

[1470] Input: prioritized search results

[1471] Data processing and calculation: formatting and sending results

[1472] Output: Search results displayed on the device

[1473] This series of steps results in a system that allows users to efficiently search and view information about local culture that is optimized to suit their own emotions.

[1474] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1475] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1476] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1477] [Fourth embodiment]

[1478] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1479] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1480] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1481] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1482] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1483] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1484] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1485] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1486] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1487] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1488] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1489] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1490] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1491] The present invention relates to a system for recording, preserving, and preventing the erosion of information about local culture, food, and games (hand games and songs). This system has a series of functions that allow users to input information, send it to a server, and the server classifies, preserves, and provides the information. Specific embodiments for implementing the present invention are described below.

[1492] System Overview

[1493] This system consists of three main components: a user terminal, a server, and a database. Users use their terminals to input information about local culture and send it to the server. The server then appropriately categorizes the received information and stores it in the database. Users can also search for information using specific keywords or categories and view the information retrieved from the database.

[1494] Program processing

[1495] When a user enters information and sends it to the server

[1496] Subject: User and Device

[1497] 1. Users input information about their local culture, events, and games on their devices. For example, they can upload a recipe for a particular local dish or a video of a traditional hand game.

[1498] 2. The device converts the input information into a video file or text file and adds metadata such as title, description, category, and tags.

[1499] 3. The device makes a request to send the converted information and metadata to the server.

[1500] When the server stores and organizes information

[1501] Subject: Server

[1502] 1. The server receives an upload request from the device and retrieves the submitted data and metadata.

[1503] 2. The server stores the received information in a database and categorizes it into appropriate categories and tags based on the metadata.

[1504] 3. The server indexes the newly stored information and makes it available for searching.

[1505] When a user searches or browses information

[1506] Subject: User, device, server

[1507] 1. The user searches from their device by specifying specific keywords or categories.

[1508] 2. The device sends the user's search query to the server.

[1509] 3. The server searches the database based on the search query, retrieves and organizes the relevant information.

[1510] 4. The server sends the search results to the device in an appropriate format.

[1511] 5. The user views the search results on their device and checks for more information.

[1512] Specific examples

[1513] For example, let's consider a case where a user searches for "traditional hand games." When the user enters "otedama" and presses the search button, the device sends this query to the server. The server retrieves video files and text information related to "otedama" from the database, organizes the results, and sends them back to the device. The user can view the results and learn how to play the traditional hand game "otedama."

[1514] summary

[1515] This system aims to prevent the erosion of culture by recording and storing local culture, food, and entertainment, and making that data easily searchable and viewable. The basic functions required for this are implemented between user devices and a server. This allows anyone to easily learn about local traditions and culture and pass them on to the next generation.

[1516] The processing flow will be explained below.

[1517] When a user uploads information

[1518] Step 1:

[1519] Users input information about local culture (e.g., video files or text information) from their own devices (smartphones, PCs, etc.).

[1520] Step 2:

[1521] The system sets metadata such as an appropriate title, description, category, and tag for the information entered by the user.

[1522] Step 3:

[1523] The user presses the "upload" button to issue a request to the terminal to transmit information.

[1524] Step 4:

[1525] The device verifies the information entered and creates a data package containing the necessary metadata.

[1526] Step 5:

[1527] The terminal requests that a data package be sent to the server.

[1528] When the server stores and organizes information

[1529] Step 6:

[1530] The server receives the data package sent from the terminal.

[1531] Step 7:

[1532] The server parses and separates the information and metadata from the received data packages.

[1533] Step 8:

[1534] The server prepares to store the parsed information in a database.

[1535] Step 9:

[1536] The server categorizes the information based on appropriate categories and tags and stores it in a database.

[1537] Step 10:

[1538] The server indexes the newly stored information and makes it searchable.

[1539] Step 11:

[1540] The server returns a confirmation message to the device that the information has been saved and classified.

[1541] When a user searches or browses information

[1542] Step 12:

[1543] A user enters a search query on their device, specifying specific keywords and categories.

[1544] Step 13:

[1545] The terminal analyzes the user's search query and sends a search request to the server.

[1546] Step 14:

[1547] The server searches the database based on the received search query and retrieves the relevant information.

[1548] Step 15:

[1549] The server properly organizes the acquired information and sends it to the terminal.

[1550] Step 16:

[1551] The user views the search results on their device and checks for more information.

[1552] When providing recipes or event information

[1553] Step 17:

[1554] A user requests information about a particular local event or dish.

[1555] Step 18:

[1556] The terminal analyzes the user's request and sends it to the server.

[1557] Step 19:

[1558] The server retrieves relevant information from the database, organizes it, and provides it to the user's terminal.

[1559] Step 20:

[1560] The user views the provided information on the device and deepens their understanding of the events and cuisine.

[1561] Through these processing steps, the system is able to effectively record and preserve local culture and provide it to users in an easily accessible form.

[1562] Example 1

[1563] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1564] Information about a region's unique culture, traditions, food, and pastimes is a valuable asset that should be passed on to future generations. However, if this information is not properly recorded and preserved, it is likely to fade away and be lost. Furthermore, current systems make it difficult to classify input information, making it difficult to search and view. Therefore, there is a need for a system that enables efficient recording, classification, search, and viewing of regional cultural information.

[1565] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1566] In this invention, the server includes means for inputting regional cultural information, means for transmitting the input information together with metadata, means for the server to store the transmitted information in a database and classify it based on the metadata, means for the server to create an index of the stored information and organize the information in a searchable form, means for a user to search for information based on specific keywords or categories, and means for the server to acquire, organize, and transmit the relevant information in an appropriate format to a terminal, and means for a user to view search results and check detailed information. This enables efficient recording, classification, search, and viewing of regional cultural information.

[1567] "Regional culture" is a general term for cultural elements such as events, food, and games that have traditionally been passed down in a particular region.

[1568] "Means for inputting information" refers to a device or interface that allows a user to input information about local culture in digital form.

[1569] "Metadata" refers to additional information such as title, description, category, and tag that is added to input information.

[1570] "Means for sending to the server" refers to the communications technology or protocol used to transfer digital information from the terminal to the server.

[1571] "Means of storing in a database" refers to the systems and technologies used by the server to permanently store the digital information it receives.

[1572] "Indexing means" refers to systems or techniques that generate tables of contents or indexes to facilitate efficient searching of stored digital information.

[1573] "Means of making information searchable" refers to database management and search engine technologies that enable quick and accurate responses to user search queries.

[1574] A "search query" refers to a keyword or phrase that a user enters when searching for information.

[1575] "Methods of organization" refers to techniques and processes for systematically sorting acquired information based on certain criteria.

[1576] "Means for transmitting to the terminal in an appropriate format" refers to the communication technology and protocols that allow the server to provide organized information to the user terminal in an appropriate format.

[1577] "Means for viewing search results" refers to the interface and display technology that allows users to visually check search results using a terminal.

[1578] This invention is a system that effectively records, stores, and easily searches and views information about local culture, food, and entertainment. This system sends information entered by users to a server, which then classifies and stores the information in a database, allowing users to easily search and view it.

[1579] System configuration

[1580] The system mainly consists of the following components:

[1581] 1. User Device

[1582] 2. Server

[1583] 3. Database

[1584] Entering information

[1585] Users input information about their local culture, events, and games into their devices. Specific examples include recipes for local dishes and videos of traditional hand games. The device saves this information as video or text files and adds metadata such as titles, descriptions, categories, and tags.

[1586] Sending data

[1587] The device uses a file upload function to send the converted information and metadata to the server, for example, by using an HTTP POST request to send the data to the server, which receives the request and stores the information and metadata in a database.

[1588] Data classification and storage

[1589] The server stores the received information in a database and categorizes it appropriately based on the metadata. Specifically, it stores the video file in storage and stores the metadata (title, description, category, tags, etc.) in the appropriate tables in the database.

[1590] Making the site searchable

[1591] The server then indexes the newly stored information and makes it searchable, which includes using a full-text search engine to generate the index.

[1592] Searching and browsing information

[1593] Users search for information by specifying specific keywords or categories on their devices. At that time, the device sends the user's search query to the server. The server searches the database based on the search query, retrieves and organizes the relevant information, and then sends it to the device in an appropriate format (for example, JSON format). Finally, the user can view the search results and check detailed information.

[1594] Example prompt

[1595] As a concrete example, let's consider the case where a user searches for "traditional hand games." The user enters "otedama" and presses the search button. The device sends this query to the server via an HTTP GET request. The server executes an SQL query, retrieves video files and text information related to "otedama" from the database, and returns the results organized in JSON format to the device. The device displays the search results in a list, and the user can view detailed information about "how to play otedama" in a pop-up display.

[1596] Example prompts to input to a generative AI model:

[1597] "I would like to know how to play otedama, a traditional Japanese hand game. What are the steps and what materials are needed?"

[1598] This system will enable efficient storage, classification, searching, and viewing of local cultural information, and is expected to help prevent culture from fading away.

[1599] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1600] Step 1:

[1601] The user enters information

[1602] Description: Users input information about their local culture, events, and pastimes from their devices.

[1603] Input: The user enters free-form text information and video data into the input form.

[1604] Data processing / calculation: Text data is stored as is via the input device, and video data is stored in digital format.

[1605] Output: Input information temporarily stored on the device.

[1606] Specific operation: The user enters a description of a traditional hand game into a form, then takes a video of the game using their smartphone and uploads it.

[1607] Step 2:

[1608] Device adds metadata

[1609] Description: The device adds metadata to the information entered, such as title, description, category, and tags.

[1610] Input: Text and video data provided by the user in Step 1.

[1611] Data processing / calculation: The process of converting data into a storage format for databases and adding metadata.

[1612] Output: Information with metadata added.

[1613] Specific operation: For a video titled "How to play with beanbags," the device automatically sets the title "Beanbags," the description "Traditional Japanese hand game," the category "Game," and the tags "Hand game, beanbags."

[1614] Step 3:

[1615] The device sends data to the server

[1616] Description: The device sends the converted information and metadata to the server.

[1617] Input: Information with metadata.

[1618] Data processing / computation: Packaging data and generating transmission requests.

[1619] Output: The data sent to the server.

[1620] What it does: Sends data to the server as an HTTP POST request, including metadata in the data packet.

[1621] Step 4:

[1622] The server receives and stores the data

[1623] Description: The server receives upload requests from devices and stores the submitted data and metadata in a database.

[1624] Input: Data and metadata sent from the device.

[1625] Data processing / calculation: Inserting data into the database and saving it to storage.

[1626] Output: Information stored in a database.

[1627] Specific operation: The video file is stored in server-side storage, and the metadata is inserted into the appropriate table in the database.

[1628] Step 5:

[1629] The server categorizes and organizes the information

[1630] Description: The server categorizes information based on metadata, indexes newly stored information, and makes it searchable.

[1631] Input: Information and metadata stored in a database.

[1632] Data processing / computation: Classification and indexing of information using indexing algorithms.

[1633] Output: Indexed and searchable information.

[1634] Specific operation: Tags such as "hand games" and "beanbags" are added to the category "play," and an index is created using a full-text search engine.

[1635] Step 6:

[1636] The user enters a search query

[1637] Description: Users search for information from their devices by specifying specific keywords or categories.

[1638] Input: User-specified keywords and categories.

[1639] Data processing / calculation: Generating search queries entered by users.

[1640] Output: The search query sent from the device to the server.

[1641] Specific behavior: The user enters "Beanbag" into the search form and presses the search button.

[1642] Step 7:

[1643] The device sends the search query to the server

[1644] Description: The device sends the user's search query to the server.

[1645] Input: The search query specified by the user.

[1646] Data processing / computation: Generating and submitting search requests.

[1647] Output: The search query sent to the server.

[1648] Specific behavior: Sends a search query to the server via an HTTP GET request.

[1649] Step 8:

[1650] The server searches the database

[1651] Description: The server searches the database based on the search query, retrieves and organizes relevant information.

[1652] Input: The search query sent from the device.

[1653] Data processing / calculation: Execute SQL queries and extract relevant information from the database.

[1654] Output: Organized search results.

[1655] Specific operation: The server executes an SQL query to retrieve video files and text information related to "Beanbag."

[1656] Step 9:

[1657] The server sends the search results to the device.

[1658] Description: The server sends the search results in the appropriate format to the device.

[1659] Input: Organized search results.

[1660] Data processing / calculation: Packaging search results in an appropriate format such as JSON.

[1661] Output: Search results sent to your device.

[1662] Specific operation: The server returns the search results to the device in JSON format.

[1663] Step 10:

[1664] Your device will display the search results.

[1665] Description: The device displays the search results received from the server in the user interface.

[1666] Input: Search results sent from the server.

[1667] Data processing / calculation: Search results are displayed in list format, and detailed information about the items selected by the user is displayed in a pop-up.

[1668] Output: Search results displayed in the user interface.

[1669] Specific behavior: The device displays the search results in a list format, and the user checks detailed information about "How to play beanbag."

[1670] (Application example 1)

[1671] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1672] Information about regional culture easily fades with time, making it particularly difficult to pass on to younger generations. Furthermore, manually entering information is time-consuming, and the lack of a voice input method makes it difficult to use. The objective of this invention is to provide a system that efficiently collects regional cultural information and utilizes voice recognition technology to allow users to easily input, search, and view information.

[1673] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1674] In this invention, the server includes means for inputting regional culture information, means for transmitting the input information together with metadata to the server, means for the server to store the transmitted information in a database and classify it, means for a user to input regional culture information by voice using voice recognition technology, means for converting the voice-input information into text data, and means for a user to search and view the stored information. This allows users to utilize voice input to easily record information and pass on regional culture.

[1675] "Regional culture" refers to cultural characteristics including events, food, games, traditions, customs, and practices unique to a region.

[1676] "Means for inputting information" means the means by which a user can directly or indirectly input data into a system, including voice, text, video, or other formats.

[1677] "Means for sending to the server" refers to the communication means or protocol for converting the input information into a certain format and sending it to the server.

[1678] "Server" refers to a computer system that processes, stores, and appropriately manages received information.

[1679] A "database" refers to a structured collection of data that stores information and makes it easy to retrieve when needed.

[1680] A "means for classification" refers to a means for organizing input information based on specific criteria or tags.

[1681] "Speech recognition technology" refers to technology for converting voice data into text data.

[1682] The term "means for inputting regional culture information by voice" refers to a means for a user to input regional culture information into the system by voice.

[1683] "Means for converting into text data" refers to means for converting information input by voice into text format.

[1684] "Means for searching and viewing information" refers to means by which a user can search for stored information based on specific keywords or categories and display the results.

[1685] This invention is a system for efficiently collecting, recording, and storing information about local culture, food, entertainment, etc., and preventing it from fading away. The invention is composed of three main components: a user terminal, a server, and a database. Specific embodiments of the invention are described below.

[1686] Input information via user terminal

[1687] First, the user uses voice recognition technology to input information about local culture. Voice recognition software such as Google Speech-to-Text API is installed on the user's device, and the voice data is converted into text data. For example, if a user says, "The local dish of this region is okonomiyaki," the content is converted into text data. At the same time, metadata such as title, description, category, and tags are also generated.

[1688] Data processing and storage by server

[1689] Text data and metadata generated on user devices are sent via the Internet to a server. The server receives this data and stores it in a database, where it is automatically categorized and organized based on the metadata. The server also creates an index of the newly stored information, making it instantly searchable.

[1690] Users search for and view information

[1691] Users can search for information by specifying specific keywords or categories. When a user enters a search query using a terminal, the query is sent to the server. The server searches for relevant information from the database and sends the results to the user's terminal. For example, if a user searches for "traditional hand games," the server will provide the user with video files and text information related to "beanbags."

[1692] Hardware and software used

[1693] The hardware used includes user devices (such as smartphones), servers, database servers, etc. The software used includes the Google Speech-to-Text API, a voice recognition technology, the HTTP communication protocol, and a database management system.

[1694] For example:

[1695] As users tour the factory, they can talk about local cultures, and their voices are recorded by their smartphones. A speech recognition system converts the speech into text, adds metadata, and sends it to a server, where it can be stored, categorized, and made searchable by other users if desired.

[1696] Prompt Sentence Examples

[1697] Here are some answers to the open-ended question that begins, "What is the local cuisine of your region?"

[1698] Example: Question: "What is the local dish of this region?"

[1699] Answer: "The local dish of this area is okonomiyaki."

[1700] As a result, the system of the present invention provides users with the ability to easily record, search, and view information about local culture, making it possible to prevent local culture from fading away.

[1701] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1702] Step 1:

[1703] The user inputs information about local culture into the device by voice. Using voice recognition technology, the user says, "The local dish of this region is okonomiyaki." The device collects this voice data. As a result, the voice data is obtained as input data.

[1704] Step 2:

[1705] Convert voice data into text data. The input voice data is converted into text data using the Google Speech-to-Text API installed on the device. The output of this process is the text data, "The local dish of this region is okonomiyaki."

[1706] Step 3:

[1707] Metadata is added to the text data. The device generates and adds metadata such as title, description, category, and tag to the converted text data. For example, metadata such as title "Local Cuisine," category "Meal," and tag "Okonomiyaki" are added. This process outputs data that combines the text data and metadata.

[1708] Step 4:

[1709] The text data with metadata is sent to the server. The terminal sends the generated data to the server using the HTTP communication protocol. In this step, data is sent from the user terminal to the server.

[1710] Step 5:

[1711] The server stores and classifies the received data. The server stores the received data in a database and automatically classifies it based on metadata. For example, it may be classified with the tag "Okonomiyaki" under the category "Food." The output is the classified data stored in the database.

[1712] Step 6:

[1713] A user searches for information on a device. The user enters a search query (e.g., "local cuisine"), and the device sends this search query to a server. The input data is the search query, and the output is a request for search results.

[1714] Step 7:

[1715] The server searches the database and organizes the results. The server searches the database based on the received search query and organizes the relevant information. The input to this process is the search query, and the output is the search result data.

[1716] Step 8:

[1717] The server sends the search results. The server sends the organized search results to the user terminal. The output of this process is the search result data sent to the user terminal.

[1718] Step 9:

[1719] The user views the search results on their device and checks detailed information. The user then views the search results sent from the server via their device. As a result, the user can view the recipe for "okonomiyaki" and related information.

[1720] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1721] The present invention combines a system for inputting, saving, searching, and viewing regional cultural information with an emotion engine that recognizes user emotions, and provides information based on the user's emotions. This system is composed of four main components: a user terminal, a server, a database, and the emotion engine. Specific embodiments for implementing the present invention are described below.

[1722] System Overview

[1723] This system has a series of functions in which users input information, send it to a server, and the server stores, categorizes, and provides it to users as needed.Furthermore, it uses an emotion engine to analyze the user's emotions and customize the information displayed based on the results.

[1724] Program processing

[1725] When a user enters information and sends it to the server

[1726] Subject: User and Device

[1727] 1. Users input information about their local culture, events, and games on their devices. For example, they can upload a recipe for a particular local dish or a video of a traditional hand game.

[1728] 2. The device sets appropriate metadata such as title, description, category, and tags for the entered information.

[1729] 3. The device creates a data package of the information and sends it to the server.

[1730] When the server stores and organizes information

[1731] Subject: Server

[1732] 1. The server receives the data package from the device and analyzes and separates the information and metadata.

[1733] 2. The server classifies the information with appropriate categories and tags and stores it in a database.

[1734] 3. The server indexes the newly stored information and makes it searchable.

[1735] When the emotion engine recognizes the user's emotions

[1736] Subject: Emotion engine and device

[1737] 1. The emotion engine uses facial recognition and text analysis to identify the user's current emotion.

[1738] 2. The device receives the user's emotion data from the emotion engine and sends it to the server.

[1739] 3. The server analyzes the emotion data and prepares to provide the user with information on the local culture that best suits them.

[1740] When a user searches or browses information

[1741] Subject: User, device, server

[1742] 1. The user enters specific keywords or categories into the device and sends a search request to the server.

[1743] 2. The server analyzes the user's search query and searches the database.

[1744] 3. The server compares the search results with the emotion engine data and sets priorities.

[1745] 4. The server sends the organized search results to the device, where the user can view them.

[1746] Specific examples

[1747] For example, if a user searches for "traditional hand games," the user can enter "otedama" (beanbag) and the device will send this query to the server. If the emotion engine recognizes that the user is relaxed, the server will prioritize providing videos and information related to "otedama" that have a relaxing effect. The user can browse the results, learn how to play the traditional hand game "otedama," and obtain the most appropriate information according to their emotional state at the time.

[1748] summary

[1749] This system not only records and preserves local culture, but also uses an emotion engine to provide information that responds to the user's emotions, allowing users to learn about and experience local culture more effectively and comfortably.

[1750] The processing flow will be explained below.

[1751] When a user uploads information

[1752] Step 1:

[1753] Users input information about local culture (e.g., video files or text information) from their own devices (smartphones, PCs, etc.).

[1754] Step 2:

[1755] The system sets metadata such as an appropriate title, description, category, and tag for the information entered by the user.

[1756] Step 3:

[1757] The user presses the "upload" button to issue a request to the terminal to transmit information.

[1758] Step 4:

[1759] The device verifies the information entered and creates a data package containing the necessary metadata.

[1760] Step 5:

[1761] The terminal requests that a data package be sent to the server.

[1762] When the server stores and organizes information

[1763] Step 6:

[1764] The server receives the data package sent from the terminal.

[1765] Step 7:

[1766] The server parses and separates the information and metadata from the received data packages.

[1767] Step 8:

[1768] The server prepares to store the parsed information in a database.

[1769] Step 9:

[1770] The server categorizes the information based on appropriate categories and tags and stores it in a database.

[1771] Step 10:

[1772] The server indexes the newly stored information and makes it searchable.

[1773] Step 11:

[1774] The server returns a confirmation message to the device that the information has been saved and classified.

[1775] When the emotion engine recognizes the user's emotions

[1776] Step 12:

[1777] The emotion engine uses facial recognition and text analysis to recognize the user's current emotions.

[1778] Step 13:

[1779] The terminal receives the user's emotion data from the emotion engine and transmits it to the server.

[1780] When a user searches or browses information

[1781] Step 14:

[1782] A user enters a search query on their device, specifying specific keywords and categories.

[1783] Step 15:

[1784] The terminal analyzes the user's search query and sends a search request to the server.

[1785] Step 16:

[1786] The server searches the database based on the received search query and retrieves the relevant information.

[1787] Step 17:

[1788] The server compares the search results with the emotion data and preferentially selects information that matches the user's emotion.

[1789] Step 18:

[1790] The server sends the organized search results to the device.

[1791] Step 19:

[1792] The user views the search results on their device and checks for more information.

[1793] When providing recipes or event information

[1794] Step 20:

[1795] A user requests information about a particular local event or dish.

[1796] Step 21:

[1797] The terminal analyzes the user's request and sends it to the server.

[1798] Step 22:

[1799] The server retrieves relevant information from the database, organizes it, and provides it to the user's terminal.

[1800] Step 23:

[1801] The user views the provided information on the device and deepens their understanding of the events and cuisine.

[1802] Using an emotion engine to customize information

[1803] Step 24:

[1804] The emotion engine records the user's emotion history and uses it to help with the next information search.

[1805] Step 25:

[1806] The emotion engine customizes the default display priority based on the user's current emotions.

[1807] This allows the system to effectively record and preserve local culture and provide optimal information based on the user's feelings, allowing users to comfortably learn and experience cultural information.

[1808] Example 2

[1809] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1810] Conventional regional cultural information systems allowed users to input, save, search, and view information, but lacked the ability to provide information based on the user's emotions. This made it difficult for users to obtain information that matched their emotional state, resulting in a decrease in satisfaction. Furthermore, conventional systems were limited to providing simple text and video data, making it difficult to gain a deep understanding of regional culture or to learn effectively.

[1811] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1812] In this invention, the server includes means for inputting regional culture information, means for transmitting the input information together with metadata to a transmission device, means for the transmission device to store and classify the transmitted information in a storage device, means for a user to search and view the stored information, and means for an emotion recognition device to analyze the user's emotions and provide information based on the user's emotions. This makes it possible to provide optimal regional culture information according to the user's emotional state, improving user satisfaction and enabling effective learning of regional culture.

[1813] "Local culture" refers to the collection of traditions, customs, events, food, games, etc. that are unique to a particular region or community.

[1814] "Means for inputting information" refers to an interface that allows users to provide data (text, video, images, etc.) related to local culture to the system.

[1815] "Metadata" refers to data (title, description, category, tags, etc.) that describes the attributes and structure of the information itself.

[1816] "Transmitting device" refers to a device or system for transmitting information and metadata input from a user terminal to a server.

[1817] "Storage" refers to a database or storage that stores the transmitted information and metadata so that it can be easily searched and classified later.

[1818] "Classification methods" refers to the process of organizing received information into categories or tags to make it easily searchable.

[1819] "Search and browsing means" refers to an interface that allows a user to search for stored information using specific keywords or categories and view the results.

[1820] An "emotion recognition device" refers to technology that identifies a user's emotions through facial recognition or text analysis, and customizes information based on that.

[1821] "Means for providing information based on user emotions" refers to a process for providing the most appropriate local cultural information by taking into account the perceived emotional state of the user.

[1822] This invention combines a system for inputting, saving, searching, and viewing regional cultural information with an emotion engine that recognizes the user's emotions, and provides information based on the user's emotions. This system consists of four main components: a user terminal, a server, a database, and an emotion engine.

[1823] System Overview

[1824] This system has a series of functions in which the user inputs information, sends it to a server, and the server stores, categorizes, and provides it to the user as needed. Furthermore, it uses an emotion engine to analyze the user's emotions and customizes and displays information based on the results. Specifically, it uses the following hardware and software:

[1825] User devices: smartphones, tablets, computers, etc.

[1826] Server: Cloud server (AWS, Google Cloud, Microsoft Azure, etc.)

[1827] Database: MySQL, PostgreSQL, MongoDB, etc.

[1828] Emotion engine: Commonly used emotion recognition technologies (e.g., Microsoft Azure Emotion Recognition API, Google Cloud Vision API, etc.)

[1829] Specific software: Django (Python framework), React (front-end)

[1830] The information and metadata entered by the user through the device are sent as a series of data packages to the server, which analyzes, classifies, and stores them in a database. The emotion engine also recognizes the user's emotional state and provides information based on that emotion.

[1831] Specific examples

[1832] For example, if a user searches for "traditional hand games," the user can input "otedama" into the device and the query will be sent to the server. If the emotion engine recognizes that the user is relaxed, the server can provide images and information related to "otedama" that have a relaxing effect preferentially. The user can browse these results and obtain the most appropriate information while learning how to play the traditional hand game "otedama." Also, the following prompt sentence can be used:

[1833] "Please tell me about a local festival that suits my mood today."

[1834] "I'd like to know some local recipes. Do you have any recommendations?"

[1835] Results of the implementation

[1836] This system not only allows users to efficiently record, save, search, and view information about local culture, but also utilizes an emotion engine to provide optimal information according to the user's emotional state, thereby improving user satisfaction and enabling effective learning and experience of local culture.

[1837] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1838] Below, a specific explanation is given, broken down into the processing steps of the system.

[1839] Step 1:

[1840] A user inputs information about the culture of their region (e.g., a recipe for a traditional dish or a video of a hand game) into a terminal. The information input can be text, images, or videos. Specifically, consider the case where a user inputs an "okonomiyaki recipe."

[1841] Step 2:

[1842] The device automatically assigns appropriate metadata such as title, description, category, and tags to the information entered. For example, the title might be "Okonomiyaki Recipe," the category might be "Cuisine," and the tags might be "Japan" and "Local Cuisine." This information is then compiled into a data package.

[1843] Step 3:

[1844] The device creates a data package and sends it to the server, which contains the entered information and metadata and is transmitted via a secure communication protocol (e.g., HTTPS).

[1845] Step 4:

[1846] The server receives the data package sent from the device, analyzes the received data package, and separates the information body (e.g., recipes and videos) from the metadata (e.g., titles and tags).

[1847] Step 5:

[1848] The server categorizes the separated information body and metadata based on appropriate categories and tags and stores them in a database. For example, information on "okonomiyaki recipes" is stored in the "cooking" category.

[1849] Step 6:

[1850] The server creates an index for the newly stored information and indexes it for fast searching. This process makes the information searchable.

[1851] Step 7:

[1852] The emotion engine uses facial recognition and text analysis to recognize the user's emotions. For example, the emotion engine analyzes the user's facial expressions and voice through a dedicated camera or microphone to identify an emotional state such as "relaxed."

[1853] Step 8:

[1854] The device receives the user's emotion data from the emotion engine and transmits it to the server via a secure communication protocol.

[1855] Step 9:

[1856] The server analyzes the received emotional data and reprioritizes information based on the results. For example, a user who is "relaxed" will be given priority for videos that have a relaxing effect.

[1857] Step 10:

[1858] A user inputs a specific keyword or category into the device and sends an information search request to the server. For example, the user inputs "otenjaku" (beanbag).

[1859] Step 11:

[1860] The server analyzes the user's search query, searches the database, and compares the search results with data from the emotion engine to prioritize search results based on the user's emotional state.

[1861] Step 12:

[1862] The server sends the search results to the device. If the user is feeling relaxed, the search results are adjusted so that information with a relaxing effect is displayed at the top.

[1863] Step 13:

[1864] Users can view search results on their devices and use the displayed information to learn more about local culture, for example, how to juggle.

[1865] These specific processing steps enable users to effectively input, save, search, and view local cultural information, and further enable the provision of customized information based on emotions.

[1866] (Application example 2)

[1867] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1868] Modern society demands systems that can efficiently input, store, search, and browse information about local culture. However, conventional systems have difficulty providing information that adapts to the user's emotions. For example, the optimal information varies depending on the user's emotions, such as when they want to relax or when they are excited. However, only uniform information is provided. Therefore, there is a need for information that can be customized based on the user's emotions.

[1869] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting information on regional culture, means for transmitting the input information together with metadata to the server, means for the server to store the transmitted information in a database and classify it, means for the user to search and view the stored information, means for acquiring emotion data using an emotion engine that recognizes the user's emotions, and means for customizing information to be provided to the user based on the emotion data. This makes it possible to provide personalized information according to the user's emotions.

[1870] "Regional cultural information" refers to information about events, festivals, cuisine, traditional games, etc. related to a particular region.

[1871] "Means of input" refers to the means by which a user inputs information into the system, and includes methods such as text input, image upload, and voice input.

[1872] "Means for transmitting to the server" refers to a method for transmitting information input from a user terminal to the server via a network, and includes internet connection, wireless communication, and the like.

[1873] "Means for storing and categorizing" refers to the way in which the server stores the received information in a database and organizes it by categories and tags.

[1874] "Means for searching and viewing" refers to the interface or functionality that allows a user to search the database for stored information and view the results on a device.

[1875] An "emotion engine" refers to algorithms or software that identify and acquire data on a user's emotions through facial recognition, voice analysis, and text analysis.

[1876] "Emotion data" refers to information about the user's emotions acquired by the emotion engine, and includes, for example, "joy," "sadness," and "surprise."

[1877] "Means for customization" refers to methods or processes for optimizing and personalizing information provided to users based on acquired emotional data.

[1878] "Product information" refers to detailed product information provided in the virtual store, including product name, price, description, and image.

[1879] This invention combines a system for inputting, saving, searching, and viewing regional cultural information with an emotion engine that recognizes the user's emotions, thereby providing information based on the user's emotions. This system is primarily composed of four main components: a user terminal, a server, a database, and an emotion engine.

[1880] ● System Overview

[1881] The system is based on a series of processes in which users input information about local culture and then send that information to a server. Specifically, the process involves the following steps:

[1882] 1. User Device

[1883] Users can input information about their local culture, events, and pastimes from their devices in a variety of ways, including text entry, image uploading, and video recording.

[1884] The device automatically sets metadata such as appropriate titles, descriptions, categories, and tags for the information entered.

[1885] The terminal creates a data package of information and sends it to the server.

[1886] 2. Server

[1887] The server receives the data packages from the devices and analyzes and classifies the information and metadata.

[1888] The server classifies the information with appropriate categories and tags and stores it in a database.

[1889] The server creates an index of the newly stored information and makes it searchable.

[1890] 3. Emotion Engine

[1891] The emotion engine uses facial recognition, voice analysis, and text analysis to identify the user's current emotion.

[1892] The terminal receives the user's emotion data from the emotion engine and transmits it to the server.

[1893] The server analyzes the emotional data and prepares to provide the user with information on the local culture that is most appropriate for them.

[1894] 4. Searching and browsing information

[1895] The user enters specific keywords or categories into the device and sends a search request to the server.

[1896] The server analyzes the user's search query and searches the database.

[1897] The server compares the search results with data from the emotion engine and sets priorities.

[1898] The server sends the organized search results to the terminal, where the user can view them.

[1899] ● Hardware and software used

[1900] The system of the present invention utilizes the following hardware and software:

[1901] Hardware

[1902] Head-mounted display (HMD)

[1903] Smart Glasses

[1904] Webcam

[1905] software

[1906] OpenCV: Used for face recognition

[1907] EmotionRecognition: Emotion analysis library

[1908] ProductDatabase: Product database

[1909] RecommendationEngine: Product recommendation engine

[1910] ● Examples of operations and prompt statements

[1911] For example, if a user searches for "traditional hand games," the user can input "beanbag" and the device will send this query to the server. If the emotion engine recognizes that the user is relaxed, the server will prioritize providing images and information related to "beanbag" that have a relaxing effect.

[1912] Another example of a prompt might be:

[1913] If you type "I feel happy," you'll see recommendations for fun products (e.g., new board games).

[1914] If you enter "feeling angry," products with a relaxing effect (e.g., scented candles) will be recommended.

[1915] This system allows users to receive personalized information tailored to their specific emotions, enabling them to learn and experience local culture more effectively and comfortably.

[1916] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1917] Step 1:

[1918] The user enters information and sends it to the terminal

[1919] A user uses a terminal to input information about local culture in the form of text, images, audio, video, etc. The terminal automatically sets appropriate metadata (e.g., title, description, category, tags, etc.) for the input information. This information packet is organized into a data package and sent to the server.

[1920] Input: User-entered local cultural information and metadata

[1921] Data processing and calculation: Composing input information into a data package with metadata

[1922] Output: Composed data package

[1923] Step 2:

[1924] The server receives the information, stores it in a database, and categorizes it.

[1925] The server receives the data package sent from the device, analyzes and separates the information and metadata, classifies the analyzed information with appropriate categories and tags, and stores it in a database. At the same time, a search index is also generated.

[1926] Input: Data package

[1927] Data processing and calculation: information and metadata analysis, separation, categorization, storage, and indexing

[1928] Output: Information stored in the database and the generated search index

[1929] Step 3:

[1930] The device acquires the user's emotional data from the emotion engine and sends it to the server.

[1931] The device uses its built-in emotion engine to identify the user's current emotion through facial recognition and voice analysis, and this emotion data is sent to the server as a data package.

[1932] Input: User's facial image and voice data

[1933] Data processing and calculation: Emotion identification by emotion engine, generation of data packages

[1934] Output: Emotion data package

[1935] Step 4:

[1936] The server analyzes the emotion data and prepares to provide information.

[1937] The server analyzes the transmitted emotional data and customizes the information provided based on the user's emotional state. For example, if the user is in a relaxed state, it will prioritize information that has a relaxing effect.

[1938] Input: Emotion data package

[1939] Data processing and calculation: Emotional data analysis, customization of information provision

[1940] Output: Preparing customized informational presentations

[1941] Step 5:

[1942] The user enters specific keywords or categories into the device and sends a search request.

[1943] The user enters specific keywords and categories into the device to send a search request, which is then sent to the server.

[1944] Input: User's search query (keywords or category)

[1945] Data processing and calculation: receiving search queries and generating requests

[1946] Output: Search request to the server

[1947] Step 6:

[1948] The server processes search requests and prioritizes them based on sentiment data.

[1949] The server receives a user's search query, searches the database, matches the search results with sentiment data, and sets priorities based on the user's sentiment.

[1950] Input: search query, sentiment data

[1951] Data processing and calculation: database search, matching with emotion data, setting priorities

[1952] Output: Prioritized search results

[1953] Step 7:

[1954] The server sends the organized search results to the device, where the user can view them.

[1955] The server sends the prioritized search results to the terminal, where the user can browse them to obtain the most appropriate local cultural information according to their emotional state.

[1956] Input: prioritized search results

[1957] Data processing and calculation: formatting and sending results

[1958] Output: Search results displayed on the device

[1959] This series of steps results in a system that allows users to efficiently search and view information about local culture that is optimized to suit their own emotions.

[1960] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1961] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1962] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1963] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1964] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1965] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1966] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1967] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1968] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1969] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1970] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1971] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1972] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1973] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1974] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1975] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1976] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1977] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1978] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1979] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1980] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1981] The following is further disclosed regarding the above embodiment.

[1982] (Claim 1)

[1983] A means of inputting regional cultural information;

[1984] means for transmitting the input information together with metadata to a server;

[1985] a means for the server to store the transmitted information in a database and perform classification;

[1986] means for a user to search and view the stored information;

[1987] A system including:

[1988] (Claim 2)

[1989] 10. The system of claim 1, including video data and text data relating to local cultural events, meals, and pastimes.

[1990] (Claim 3)

[1991] 2. The system according to claim 1, wherein a user searches for information based on a specific keyword or category, and the server provides the relevant information.

[1992] "Example 1"

[1993] (Claim 1)

[1994] A means of inputting regional cultural information;

[1995] means for transmitting the input information together with metadata to a server;

[1996] means for the server to store the transmitted information in a database and classify it based on the metadata;

[1997] a means for the server to index the stored information and make the information searchable;

[1998] A means for a user to search for information based on specific keywords or categories, and a server to retrieve and organize the relevant information and send it to the terminal in an appropriate format;

[1999] A way for users to browse search results and find more information;

[2000] A system including:

[2001] (Claim 2)

[2002] 10. The system of claim 1, including digital data about local cultural events, meals, and pastimes.

[2003] (Claim 3)

[2004] 2. The system according to claim 1, further comprising a process in which the server adds metadata such as a title, description, category, and tag to the information entered by the user.

[2005] "Application Example 1"

[2006] (Claim 1)

[2007] A means of inputting regional cultural information;

[2008] means for transmitting the input information together with metadata to a server;

[2009] a means for the server to store the transmitted information in a database and perform classification;

[2010] A means for users to input regional cultural information by voice using voice recognition technology;

[2011] A means for converting voice-input information into text data;

[2012] means for a user to search and view the stored information;

[2013] A system including:

[2014] (Claim 2)

[2015] 10. The system of claim 1, including video data and text data relating to local cultural events, meals, and pastimes.

[2016] (Claim 3)

[2017] 2. The system according to claim 1, wherein a user searches for information based on a specific keyword or category, and the server provides the relevant information.

[2018] "Example 2: Combining Emotion Engines"

[2019] (Claim 1)

[2020] A means of inputting regional cultural information;

[2021] means for transmitting the input information together with metadata to a transmitting device;

[2022] a means for storing the information transmitted by the transmitting device in a storage device and classifying the information;

[2023] means for a user to search and view the stored information;

[2024] means for analyzing the emotion of a user by an emotion recognition device and providing information based on the emotion of the user;

[2025] A system including:

[2026] (Claim 2)

[2027] The system of claim 1, including motion data and text data relating to local cultural events, cooking, and games.

[2028] (Claim 3)

[2029] 2. The system according to claim 1, wherein a user searches for information based on a specific keyword or category, and a transmitting device provides the corresponding information.

[2030] "Application example 2 when combining emotion engines"

[2031] (Claim 1)

[2032] A means of inputting regional cultural information;

[2033] means for transmitting the input information together with metadata to a server;

[2034] a means for the server to store the transmitted information in a database and perform classification;

[2035] means for a user to search and view the stored information;

[2036] A means for acquiring emotion data using an emotion engine that recognizes the emotion of a user;

[2037] means for customizing information provided to a user based on the emotion data;

[2038] A system including:

[2039] (Claim 2)

[2040] 10. The system of claim 1, including video data and text data relating to local cultural events, meals, and pastimes.

[2041] (Claim 3)

[2042] 2. The system according to claim 1, wherein a user searches for information based on a specific keyword or category, and the server provides the relevant information.

[2043] (Claim 4)

[2044] 2. The system according to claim 1, further comprising means for analyzing a user's emotions in real time and providing product information according to the emotions. [Explanation of symbols]

[2045] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of inputting regional cultural information; means for transmitting the input information together with metadata to a server; a means for the server to store the transmitted information in a database and perform classification; means for a user to search and view the stored information; A system including:

2. 10. The system of claim 1, further comprising video data and text data relating to local cultural events, meals, and pastimes.

3. 2. The system of claim 1, wherein a user searches for information based on a specific keyword or category, and the server provides the relevant information.

Images