system

Abstract

Provide a system. The system includes: a means for authenticating a user by comparing authentication information entered by the user with information stored in a database; a means of retrieving and providing information about multiple cultures from a database; A means of recommending optimal interaction partners based on the user's profile information; A means for translating a message sent by a user in real time, and storing and transmitting the translated message to the terminal of the other user; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, as globalization progresses, misunderstandings and conflicts between people with different cultural backgrounds are increasing. This often makes international cooperation and coexistence difficult. Therefore, there is a need for systems that deepen understanding of different cultures and promote mutual understanding through dialogue. However, there is a lack of effective means to resolve cultural differences in real time and enable smooth communication between users who speak different languages. Technology is needed to solve this problem. [Means for solving the problem]

[0005] The present invention is a system that includes a means for authenticating a user by comparing authentication information entered by the user with a database, a means for retrieving and providing information about multiple cultures from the database, a means for recommending optimal exchange partners based on the user's profile information, and a means for translating messages sent by the user in real time and saving and sending the translated messages to the other user's device. Additionally, by providing a means for executing an algorithm for matching cultural exchange partners based on the information entered by the user, and a means for saving translated messages in a chat log and managing the conversation history, it is possible to reduce cultural misunderstandings and conflicts and promote mutual understanding between users with different cultural backgrounds.

[0006] A "user" is an individual or organization that authenticates to the system and interacts with other users from different cultural backgrounds.

[0007] "Authentication information" refers to information including a username and password that a user enters when accessing a system.

[0008] A "database" is a collection of related information that stores data such as user information, cultural information, and chat logs, and can be used for searching and matching as needed.

[0009] "Cultural information" refers to information that constitutes the lifestyle and culture of people in a particular region or country, such as customs, traditions, history, practices, and festivals.

[0010] "Profile information" is information about a user that the user creates within the system, including name, age, interests, affiliation, etc.

[0011] "Exchange partners" are other users with different cultural backgrounds who are recommended by the system for the user to engage in cultural exchange.

[0012] "Real-time translation" is the process of instantly converting a message sent by a user into a specified language.

[0013] A "chat log" refers to a record of messages exchanged between users, which is saved in chronological order. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The embodiment of the present invention first includes a user authentication process, a cultural information providing process, an exchange partner matching process, and a dialogue translation and storage process.

[0036] User Authentication

[0037] When a user accesses a system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the database, and if it matches, it authenticates the user and allows them access to the system.

[0038] Providing cultural information

[0039] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[0040] Matching exchange partners

[0041] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list of candidates.

[0042] Dialogue translation and storage

[0043] A user starts a conversation session and inputs a message. The device collects the input message and sends it to the server. The server translates the received message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[0044] Specific examples

[0045] For example, user A enters "I want to know about Japanese New Year customs," and as a result, he or she checks detailed information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" on his or her device. User A then finds a cultural exchange partner and is connected with user B. When user A sends a message saying, "How do you celebrate New Year in your country?", the message is translated by the server and converted into user B's language. User B replies, "We set off fireworks." The contents of the conversation are saved in a chat log and displayed on both devices.

[0046] As described above, this system includes functions for user authentication, provision of cultural information, matching of communication partners, and real-time translation and storage of messages, enabling users with different cultural backgrounds to communicate smoothly and promote mutual understanding.

[0047] The processing flow will be explained below.

[0048] Step 1:

[0049] The user accesses the login screen and enters their username and password.

[0050] Step 2:

[0051] The terminal collects the authentication information entered by the user and sends it to the server.

[0052] Step 3:

[0053] The server checks the received authentication information against the database, and if there is a match, authenticates the user and returns the authentication result to the terminal.

[0054] Step 4:

[0055] Users navigate to the Cultural Exchange section and select a cultural topic of interest.

[0056] Step 5:

[0057] The terminal transmits information on the cultural topic selected by the user to the server as a request.

[0058] Step 6:

[0059] Based on the request, the server retrieves the relevant cultural information from the database and returns the information to the terminal.

[0060] Step 7:

[0061] The terminal displays the received culture information to the user.

[0062] Step 8:

[0063] Users click the "Find a Cultural Exchange Partner" button.

[0064] Step 9:

[0065] The terminal sends a partner search request to the server.

[0066] Step 10:

[0067] The server searches a database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the terminal.

[0068] Step 11:

[0069] The user selects a partner from the candidate list and starts a dialogue session.

[0070] Step 12:

[0071] The user inputs a message, and the terminal transmits the input message to the server.

[0072] Step 13:

[0073] The server translates the received message in real time using a translation engine.

[0074] Step 14:

[0075] The server saves the translated message in the chat log and sends the translation result to the other party's device.

[0076] Step 15:

[0077] The terminal displays the translated message from the other party to the user.

[0078] Step 16:

[0079] The user checks the other person's message and continues the conversation.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] In modern society, smooth communication between people of different cultural backgrounds is essential to promoting mutual understanding. However, cultural exchange can be difficult due to language barriers, a lack of cultural information, and a lack of suitable exchange partners. There is a need to develop a system that can solve these issues.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes means for authenticating the user by comparing authentication information entered by the user with information stored in a database, means for retrieving and providing information on multiple cultures from the database, means for recommending optimal communication partners based on the user's profile information, and means for translating messages sent by the user in real time, saving the translated messages in a chat log, and sending them to the other user's device. This enables users with different cultural backgrounds to communicate smoothly and promote mutual understanding.

[0085] "User authentication information" refers to information such as a username and password that a user enters when accessing a system.

[0086] A "database" is a system for organizing and managing information, and stores user authentication information, cultural information, chat logs, etc.

[0087] "Cultural information" is information about the customs, practices, and events of different regions, countries, and societies.

[0088] "Exchange partners" are other users with different cultural backgrounds who are recommended to the user by the system for cultural exchange.

[0089] "Real-time translation" is the process of instantly translating a message sent by a user into another language.

[0090] A "chat log" is a database that records and stores the content of conversations between users.

[0091] "Server" means the central computer system that manages and executes the functions of the entire System.

[0092] "Authentication" refers to the process and technology used to authenticate a user by matching the user's credentials against a database.

[0093] "Providing means" refers to the process and technology for providing cultural information obtained from the database to the user's terminal.

[0094] "Recommendation tools" are processes and techniques for recommending optimal interaction partners based on a user's profile information.

[0095] "Translation means" refers to the process and technology for translating a message sent by a user in real time and sending it to the other user's terminal.

[0096] The "history database" is a database for managing the history of conversations including translated messages.

[0097] To implement this invention, a user authentication process, a cultural information provision process, an exchange partner matching process, and a dialogue translation and storage process are required.

[0098] User Authentication Process

[0099] When a user accesses the system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the database, and if it matches, it authenticates the user and allows them access to the system.

[0100] For example, when a user enters a "user name" and "password," the terminal sends this to the server, which then checks the database for authentication. If authentication is successful, the terminal displays "Authentication successful." An example of a prompt statement is "Please enter your user name and password."

[0101] Cultural information provision process

[0102] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[0103] As a concrete example, when a user selects the topic "Japanese New Year customs" and the device requests this from the server, the server retrieves information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" from the database and sends it to the device. The device then displays this information to the user. An example of a prompt sentence is "Please select a cultural topic that interests you."

[0104] Exchange Partner Matching Process

[0105] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list of candidates.

[0106] As a concrete example, when a user clicks the "Find a cultural exchange partner" button and the device sends a request to the server, the server searches for candidates such as "User B" and "User C" and sends them to the device. The device displays the candidate list to the user, and the user selects "User B." An example of a prompt sentence is "Click the button to find a cultural exchange partner."

[0107] Dialogue translation and storage process

[0108] A user starts a conversation session and inputs a message. The device collects the input message and sends it to the server. The server translates the message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[0109] For example, user A types "How do you celebrate New Year in your country?" and their device sends this message to the server, which translates the message and sends it in the language of "user B." User B replies, "We set off fireworks," and the reply is saved in the chat log and displayed on both devices.

[0110] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0111] User Authentication Process

[0112] Step 1:

[0113] The user enters their username and password into the login screen.

[0114] Input: Username, Password

[0115] Output: Credentials

[0116] Specific operation: The user enters a "user name" and "password."

[0117] Step 2:

[0118] The terminal collects this authentication information and sends it to the server.

[0119] Input: Credentials

[0120] Output: Request to server

[0121] Specific operation: The terminal sends the username and password to the server.

[0122] Step 3:

[0123] The server checks the received authentication information against a database.

[0124] Input: Credentials

[0125] Output: Matching result

[0126] What happens: The server checks the username and password in its database.

[0127] Step 4:

[0128] The server returns the result of the match to the terminal.

[0129] Input: Matching result

[0130] Output: Authentication result

[0131] Specific operation: The server sends the result of "authentication successful" or "authentication failed" to the terminal.

[0132] Step 5:

[0133] The terminal displays the authentication result to the user.

[0134] Input: Authentication result

[0135] Output: Display information

[0136] Specific operation: The device will display "Authentication successful" or "Authentication failed" on the screen.

[0137] Cultural information provision process

[0138] Step 1:

[0139] The user navigates to the Cultural Exchange section and selects a cultural topic of interest.

[0140] Input: Select a cultural topic

[0141] Output: Selection information

[0142] What happens: The user selects a topic, such as "Japanese New Year's Customs."

[0143] Step 2:

[0144] The terminal collects the user's selection information and sends a request to the server.

[0145] Input: Selection information

[0146] Output: Request to server

[0147] Specific operation: The terminal transmits the selection information to the server.

[0148] Step 3:

[0149] The server retrieves the requested cultural information from a database.

[0150] Input: Request information

[0151] Output:Cultural information

[0152] Specific operation: The server retrieves information such as "first shrine visit of the year" and "New Year's cuisine" from the database.

[0153] Step 4:

[0154] The server returns the acquired culture information to the terminal.

[0155] Input:Cultural information

[0156] Output:Return information

[0157] Specific operation: The server sends culture information to the terminal.

[0158] Step 5:

[0159] The terminal displays the returned information to the user.

[0160] Input:Return information

[0161] Output: Display information

[0162] Specific operation: The device displays the cultural information on the screen.

[0163] Exchange Partner Matching Process

[0164] Step 1:

[0165] The user clicks the "Find a Cultural Exchange Partner" button.

[0166] Input: Button click

[0167] Output: Partner search request

[0168] Specific action: The user clicks the "Find a cultural exchange partner" button.

[0169] Step 2:

[0170] The terminal sends a partner search request to the server.

[0171] Input: Search request

[0172] Output: Request to server

[0173] Specific operation: The device sends a search request to the server.

[0174] Step 3:

[0175] The server searches a database for the most suitable partner based on the user's profile information.

[0176] Input: Profile information

[0177] Output: List of potential partners

[0178] Specific operation: The server searches the database for candidates such as "User B" and "User C."

[0179] Step 4:

[0180] The server returns a list of potential partners to the terminal as a search result.

[0181] Input: List of potential partners

[0182] Output:Return information

[0183] Specific operation: The server sends the candidate list to the terminal.

[0184] Step 5:

[0185] The terminal displays the candidate list to the user.

[0186] Input:Return information

[0187] Output: Display information

[0188] Specific operation: The device displays the candidate list on the screen.

[0189] Step 6:

[0190] The user selects the partner with whom they wish to interact from a list of candidates.

[0191] Input: Partner selection

[0192] Output: Selection information

[0193] Specific operation: The user selects "User B."

[0194] Dialogue translation and storage process

[0195] Step 1:

[0196] The user initiates an interactive session and enters a message.

[0197] Input: Message

[0198] Output: The input message

[0199] What happens: User types, "How do you celebrate New Year's in your country?"

[0200] Step 2:

[0201] The terminal collects the input messages and sends them to the server.

[0202] Input: The message entered

[0203] Output: Request to server

[0204] Specific operation: The device sends a message to the server.

[0205] Step 3:

[0206] The server translates the message into the target language using a real-time translation engine.

[0207] Input: The message entered

[0208] Output: Translated message

[0209] Specific behavior: The server translates "How do you celebrate New Year in your country?" into the other person's language.

[0210] Step 4:

[0211] The server stores the translated messages in the chat log.

[0212] Input: Translated message

[0213] Output: Saved messages

[0214] What happens next: The server stores the translated message in a database.

[0215] Step 5:

[0216] The server sends the translated message to the other user's terminal.

[0217] Input: Translated message

[0218] Output: Send to the other terminal

[0219] Specific operation: The server sends the translated message to the other device.

[0220] Step 6:

[0221] The user can check the translated message on their device and continue the conversation.

[0222] Input: Translated message

[0223] Output: The displayed message

[0224] Specific action: The user sees the reply from the other party on their device: "We're going to set off fireworks."

[0225] (Application example 1)

[0226] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0227] For users with different cultural backgrounds to communicate smoothly while shopping online, real-time language translation and cultural and product information are necessary. Conventional systems do not integrate these elements and are unable to provide the diverse information and interaction opportunities users desire. Therefore, the challenge is to enable users from different cultural backgrounds to share cultural and product information while also enabling smooth communication with partners.

[0228] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0229] In this invention, the server includes means for authenticating the user by comparing authentication information entered by the user with information stored in a database, means for retrieving and providing information about multiple cultures from the database, means for recommending optimal communication partners based on the user's profile information, means for translating messages sent by the user in real time and storing and sending the translated messages to the other user's terminal, and means for users to retrieve information about cultures and products of different countries while shopping online and communicate with sales staff and other shopping users in the local language, thereby enabling users from different cultural backgrounds to share cultural and product information and communicate smoothly with partners.

[0230] "User authentication" refers to the process of verifying a user's identity by matching the authentication information entered by the user with information stored in a database.

[0231] "Cultural information provision" refers to the process of retrieving information about multiple cultures from a database and providing it appropriately to the user.

[0232] "Exchange partner recommendation" refers to the process of selecting and recommending the most suitable exchange partner to the user based on the user's profile information.

[0233] "Real-time translation" refers to the process of translating a message sent by a user into a different language in real time, and then storing and sending the translated message to the other user's device.

[0234] "Cultural information provision during online shopping" refers to the process of retrieving and providing information about cultures and products of different countries when users are shopping online.

[0235] "Communication methods" refers to the functions and processes for communicating in real time with salespeople and other shopping users in the local language.

[0236] As an embodiment of this invention, we will explain a system that includes user authentication, cultural information provision, exchange partner recommendation, real-time translation, cultural information provision during online shopping, and communication means. This system is configured using a program that uses the Flask framework and the Google (registered trademark) translation API.

[0237] User Authentication

[0238] When a user accesses the system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the information stored in the database, and if it matches, the user is authenticated and allowed to access the system.

[0239] Providing cultural information

[0240] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[0241] Exchange partner recommendation

[0242] When a user clicks the "Find a cultural exchange partner" button, the device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list.

[0243] Real-time translation

[0244] When a user starts a conversation session and inputs a message, the device collects the input message and sends it to the server. The server translates the received message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[0245] Providing cultural information while shopping online

[0246] When a user selects a specific item to retrieve information about the culture or products of different countries while shopping online, the terminal sends the request to the server, which retrieves the culture and product information from the database and returns it to the terminal, which uses the information to enrich the user's shopping experience.

[0247] Communication tools

[0248] A real-time translation function is used by users to communicate with other users and salespeople while online shopping. The server translates messages and sends the translated messages to each user's device.

[0249] Specific examples

[0250] For example, if User A is interested in French wine and requests cultural information about it, information about the history and brewing methods of wine will be displayed. If User A then communicates with a French salesperson and wants more detailed information, the message will be translated in real time and sent to the salesperson. The salesperson's reply will also be translated and displayed on User A's screen, enabling smooth communication.

[0251] Example prompt sentence:

[0252] 1. Log in:

[0253] Enter your username and password to log in

[0254] "username: user1, password: password1"

[0255] 2. Obtaining cultural information:

[0256] User requests information about "wine"

[0257] "topic: wine"

[0258] 3. Matching exchange partners:

[0259] Provide your profile information to find a partner

[0260] "profile: User Profile Data"

[0261] 4. Message translation and dialogue:

[0262] Translate and communicate messages

[0263] "message: Tell me about French wine, target_language: fr"

[0264] In this way, a system is realized that allows users to obtain information about different cultures while shopping online, and promote communication with their partners.

[0265] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0266] Step 1:

[0267] When a user accesses the system, they enter a username and password on a login screen.

[0268] Input: Username, Password

[0269] Operation: The device collects the authentication information entered by the user and sends it to the server.

[0270] Data processing: The server receives the authentication information and checks it against the information stored in the database.

[0271] Output: Authentication success / failure message

[0272] Step 2:

[0273] The user navigates to the Cultural Exchange section and selects a cultural topic of interest.

[0274] Input: Cultural topic selection information

[0275] Operation: The terminal collects the user's selection information and sends it to the server.

[0276] Data processing: The server retrieves the relevant cultural information from the database.

[0277] Output: Retrieved cultural information

[0278] Step 3:

[0279] The user clicks the "Find a Cultural Exchange Partner" button.

[0280] Input: Partner Search Request

[0281] Operation: The device sends a request to the server.

[0282] Data processing: The server searches the database for the most suitable partner based on the user's profile information.

[0283] Output: List of potential partners

[0284] Step 4:

[0285] The user selects the partner they want to interact with from a list of candidates.

[0286] Input: Partner selection information

[0287] Operation: The device sends the selection information to the server, which then sends a notification to the selected partner.

[0288] Output: Connection status with partner

[0289] Step 5:

[0290] The user initiates an interactive session and enters a message.

[0291] Input: Dialogue message

[0292] Operation: The terminal sends the inputted interaction message to the server.

[0293] Data processing: The server translates the message using a real-time translation engine.

[0294] Output: Save the translated message and send it to the other user's device

[0295] Step 6:

[0296] Users select specific items to retrieve specific cultural or product information while shopping online.

[0297] Input: Request for cultural or product information

[0298] Operation: The device sends a request to the server.

[0299] Data processing: The server retrieves the relevant information from the database and returns it to the terminal.

[0300] Output: Returned culture and product information

[0301] Step 7:

[0302] While a user is shopping online, the user communicates with sales staff and other users in real time.

[0303] Input: Dialogue message

[0304] Operation: The terminal sends the input message to the server, which translates the message and sends it to the other terminal.

[0305] Data processing: Message translation and saving to chat log

[0306] Output: Sends translated message and updates chat log

[0307] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0308] The embodiment of the present invention includes a user authentication process, a cultural information providing process, an exchange partner matching process, a dialogue translation and storage process, and an emotion engine integration process.

[0309] User Authentication

[0310] When a user accesses a system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the database, and if it matches, it authenticates the user and allows them access to the system.

[0311] Providing cultural information

[0312] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[0313] Matching exchange partners

[0314] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list of candidates.

[0315] Dialogue translation and storage

[0316] A user starts a conversation session and inputs a message. The device sends the input message to the server. The server translates the received message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[0317] Emotion engine integration

[0318] An emotion engine is built into messages entered by users. When a device sends a message to the server, it requests that the emotion of the message be analyzed. The server uses the emotion engine to analyze the emotion of the message in real time and adjusts the tone and nuance of the translation based on the results. For example, if user A enters "I'm excited," the emotion engine analyzes the emotion as "excited" and translates it with an appropriate tone. The emotion engine also saves the results in the chat log and manages changes in emotion as a history.

[0319] Specific examples

[0320] For example, User A types, "I want to know about Japanese New Year customs," and checks detailed information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" on his device. User A then finds a cultural exchange partner and is connected with User B. When User A sends a message saying, "How do you celebrate the New Year in your country?", the message is translated in real time by the server and converted into User B's language. At this time, User A's emotions are also analyzed, so User B can receive the translated message, "We will set off fireworks," in a natural tone.

[0321] As described above, this system includes functions for user authentication, provision of cultural information, matching of communication partners, real-time translation and storage of messages, and integration of an emotion engine, enabling users with different cultural backgrounds to communicate smoothly and promote deeper mutual understanding.

[0322] The processing flow will be explained below.

[0323] Step 1:

[0324] The user accesses the login screen and enters their username and password.

[0325] Step 2:

[0326] The terminal collects the authentication information entered by the user and sends it to the server.

[0327] Step 3:

[0328] The server checks the received authentication information against the database, and if there is a match, authenticates the user and returns the authentication result to the terminal.

[0329] Step 4:

[0330] Users navigate to the Cultural Exchange section and select a cultural topic of interest.

[0331] Step 5:

[0332] The terminal transmits information on the cultural topic selected by the user to the server as a request.

[0333] Step 6:

[0334] Based on the request, the server retrieves the relevant cultural information from a database and returns the information to the terminal.

[0335] Step 7:

[0336] The terminal displays the received culture information to the user.

[0337] Step 8:

[0338] Users click the "Find a Cultural Exchange Partner" button.

[0339] Step 9:

[0340] The terminal sends a partner search request to the server.

[0341] Step 10:

[0342] The server searches a database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the terminal.

[0343] Step 11:

[0344] The user selects a partner from the candidate list and starts a dialogue session.

[0345] Step 12:

[0346] The user inputs a message, and the terminal requests emotion analysis by the emotion engine when transmitting the input message to the server.

[0347] Step 13:

[0348] The server analyzes the received message in real time using an emotion engine to identify the emotion of the message.

[0349] Step 14:

[0350] The server adjusts the tone and nuance of the translation as needed based on the emotions identified by the emotion engine.

[0351] Step 15:

[0352] The server saves the adjusted translated message in the chat log and sends the translation result to the other party's device.

[0353] Step 16:

[0354] The terminal displays the translated message from the other party to the user.

[0355] Step 17:

[0356] The device also stores the emotional data of messages in a chat log and manages changes in emotions as a history.

[0357] Step 18:

[0358] The user checks the other person's message and continues the conversation.

[0359] For example, if user A types "I want to know about Japanese New Year customs," the information is returned to the device and displayed. User A then finds a social partner and is connected with user B. If user A sends a message saying "How do you celebrate the New Year in your country?", the server analyzes the message and identifies the emotion "excited." The translation result is provided to user B in an appropriate tone, enabling natural conversation that takes into account each other's emotions.

[0360] Example 2

[0361] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0362] While conventional cultural exchange systems provide basic functions such as user authentication, cultural information provision, partner matching, and message translation and storage, they lack the ability to adjust emotional nuances during translation or the ability to store and manage emotional changes in chat logs. This makes it difficult for users with different cultural backgrounds to communicate smoothly and naturally. The objective of this invention is to resolve these shortcomings and promote deeper mutual understanding and smoother communication between users.

[0363] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0364] means for authenticating a user by matching authentication information entered by the user with information stored in a database;

[0365] a means of retrieving and providing information about multiple cultures from a database;

[0366] A means of recommending optimal interaction partners based on the user's profile information;

[0367] A means for translating a message sent by a user in real time, and storing and transmitting the translated message to the terminal of the other user;

[0368] A means to analyze the message to be translated using an emotion engine and adjust the tone and nuance of the translation,

[0369] A means to save the analysis results of the emotion engine in a chat log and manage changes in emotions as a history,

[0370] This allows users with different cultural backgrounds to communicate smoothly and naturally and manage emotional changes, enabling deeper mutual understanding.

[0371] "User authentication" is the process of verifying the identity of a user who accesses a system by comparing the authentication information entered by the user with the information stored in a database.

[0372] "Cultural information" is information about multiple cultures, and is a collection of knowledge and data that is provided by user selection.

[0373] An "exchange partner" is a user with whom you can engage in cultural exchange, optimally selected based on your profile information.

[0374] "Message translation" is the process of translating messages sent by users into other languages ​​in real time.

[0375] A "translated message" is a message after the original message has been translated by a real-time translation engine.

[0376] The "emotion engine" is an engine that analyzes messages entered by users and adjusts the tone and nuance based on emotions.

[0377] A "chat log" is a database for storing and managing the history of conversations between users.

[0378] "Emotion analysis results" are data related to emotions obtained by the emotion engine analyzing the user's message.

[0379] The embodiment of the present invention mainly includes a user authentication process, a cultural information providing process, an exchange partner matching process, a dialogue translation and storage process, and an emotion engine integration process.

[0380] User Authentication Process

[0381] When a user accesses the system, they enter their username and password on a login screen. This authentication information is collected by the terminal and sent to the server. The server compares the received authentication information with its database, and if it matches, it authenticates the user and allows them access to the system. This series of processes uses AWS (registered trademark) DynamoDB and the HTTP protocol.

[0382] Cultural information provision process

[0383] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device sends the user's selection as a request to the server. The server retrieves the cultural information from the database using MongoDB and returns it to the device. The returned information is displayed by the device for the user to review.

[0384] Exchange Partner Matching Process

[0385] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server uses ElasticSearch (registered trademark) to search the database based on the user's profile information, creates a list of suitable partners, and returns it to the device. The user can then select the desired partner from the list of candidates.

[0386] Dialogue translation and storage process

[0387] A user starts a conversation session and inputs a message. The device sends the input message to the server. The server translates the message in real time using the Google Cloud Translate API, stores the translated message in DynamoDB, and sends it to the other user's device. The user can then view the translated message and continue the conversation.

[0388] Emotion Engine Integration Process

[0389] An emotion engine is built into messages entered by users. When a device sends a message to the server, it requests that the emotion engine be used to analyze the emotion of the message. The server uses IBM Watson (registered trademark)'s NLP emotion analysis API to analyze the emotion of the message in real time and adjusts the tone and nuance of the translation based on the results. The emotion engine's analysis results are also saved in the chat log, and changes in emotion are managed as a history.

[0390] Specific examples

[0391] For example, User A types, "I want to know about Japanese New Year customs," and checks detailed information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" on his device. User A then finds a cultural exchange partner and is connected with User B. When User A sends a message saying, "How do you celebrate the New Year in your country?", the message is translated in real time by the server and converted into User B's language. At this time, User A's emotions are also analyzed, so User B can receive the translated message, "We will set off fireworks," in a natural tone.

[0392] Prompt Sentence Examples

[0393] An example of a prompt sentence to be input to the generative AI model is, "Please explain the process that occurs when a user enters their username and password on the login screen."

[0394] As described above, this system includes functions for user authentication, provision of cultural information, matching of communication partners, real-time translation and storage of messages, and even integration of an emotion engine, enabling users with different cultural backgrounds to communicate smoothly and promote deeper mutual understanding.

[0395] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0396] Step 1:

[0397] A user enters a username and password into a login screen to access the system.

[0398] Specific operation: The user enters a username and password in the input fields and presses the login button.

[0399] Input: Username and Password

[0400] Output: User credentials passed to the terminal

[0401] Step 2:

[0402] The terminal collects the entered username and password and sends them to the server.

[0403] Specific operation: The terminal sends the input data to the server as an HTTP POST request.

[0404] Input: User credentials

[0405] Output: HTTP request passed to the server

[0406] Step 3:

[0407] The server checks the received authentication information against a database, and if there is a match, it authenticates the user and allows them access to the system.

[0408] What happens: The server queries DynamoDB to find a matching record, and if authentication is successful, returns a success message in the HTTP response.

[0409] Input: User credentials

[0410] Output: Authentication result (success / failure)

[0411] Step 4:

[0412] Users navigate to the Cultural Exchange section and select a cultural topic of interest.

[0413] Specific operation: The user selects a cultural topic from the UI and presses the select button.

[0414] Input: Cultural topic selection

[0415] Output: Selection information passed to the terminal

[0416] Step 5:

[0417] The terminal collects the user's selection information and sends a request to the server.

[0418] Specific operation: The terminal sends the selected data to the server as an HTTP GET request.

[0419] Input: Cultural topic selection information

[0420] Output: HTTP request passed to the server

[0421] Step 6:

[0422] Upon receiving the request, the server retrieves the culture information from the database and returns it to the terminal.

[0423] Specific operation: The server searches MongoDB, obtains the corresponding culture information, and returns it in the HTTP response.

[0424] Input: Cultural topic selection information

[0425] Output: Culture information sent back to the terminal

[0426] Step 7:

[0427] The terminal displays the culture information received from the server so that the user can check it.

[0428] Specific operation: The device parses the received JSON data and displays it on the screen.

[0429] Input:Cultural information

[0430] Output: Culture information displayed on the screen

[0431] Step 8:

[0432] The user clicks on the "Find a Cultural Exchange Partner" button.

[0433] Specific Action: The user clicks a button in the UI.

[0434] Input: Button click event

[0435] Output: Click event passed to the terminal

[0436] Step 9:

[0437] The terminal sends a partner search request to the server.

[0438] Specific operation: The terminal sends an HTTP GET request including user information to the server.

[0439] Input: User information, click event

[0440] Output: HTTP request passed to the server

[0441] Step 10:

[0442] The server searches a database based on the user's profile information to find the most suitable partner and returns a list of candidate partners to the terminal.

[0443] What happens: The server uses Elasticsearch to search the database and return a list of matching users.

[0444] Input: User information

[0445] Output: A list of potential partners returned to the terminal

[0446] Step 11:

[0447] The terminal displays a list of candidate partners to the user, allowing the user to make a selection.

[0448] Specific operation: The device displays the received list on the screen and provides a UI that allows the user to select.

[0449] Input: List of potential partners

[0450] Output: Partner list displayed on screen

[0451] Step 12:

[0452] The user initiates an interactive session and enters a message.

[0453] Specific operation: The user enters a message in the text box of the UI and presses the send button.

[0454] Input: Message content

[0455] Output: Message content passed to the terminal

[0456] Step 13:

[0457] The terminal transmits the input message to the server.

[0458] Specific operation: The device sends an HTTP POST request containing a message to the server.

[0459] Input: Message content

[0460] Output: HTTP request passed to the server

[0461] Step 14:

[0462] The server translates the message in real time using the Google Cloud Translate API, stores the translated message in DynamoDB, and sends it to the other user's device.

[0463] Specific operation: The server calls the translation API, stores the translation results in DynamoDB, and sends them to the other device.

[0464] Input: Message content

[0465] Output: translated messages, saved chat logs

[0466] Step 15:

[0467] The other party's terminal displays the translated message received from the server to the user.

[0468] Specific operation: The device parses the received JSON data and displays it on the screen.

[0469] Input: Translated message

[0470] Output: Messages displayed on the screen

[0471] Step 16:

[0472] The device requests the server to analyze the emotions of the sent messages in real time.

[0473] Specific operation: The device adds an emotion analysis flag to an HTTP POST request containing a message and sends it to the server.

[0474] Input: Message content, sentiment analysis flag

[0475] Output: HTTP request passed to the server

[0476] Step 17:

[0477] The server uses IBM Watson's NLP sentiment analysis API to analyze messages in real time and set the tone and nuance of the translation based on the results.

[0478] Specific operation: The server calls the sentiment analysis API and passes the results to the Google Cloud Translate API for translation.

[0479] Input: Message content

[0480] Output: Sentiment analysis results, adjusted translation tone

[0481] Step 18:

[0482] The server stores the emotion analysis results in a chat log and manages changes in emotions as a history.

[0483] Specific operation: The server stores the sentiment analysis results in DynamoDB so that they can be retrieved later.

[0484] Input: Sentiment analysis results

[0485] Output: Saved sentiment analysis results

[0486] Through the above steps, communication between users is facilitated and a dialogue history including changes in emotions is managed.

[0487] (Application example 2)

[0488] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0489] Conventional cultural exchange systems have faced challenges in enabling smooth communication between users with different languages ​​and cultural backgrounds. In particular, they lacked the ability to find suitable exchange partners, perform real-time translation between multiple languages, and analyze emotions to maintain a natural tone in the dialogue. As a result, users not only failed to deepen their understanding of each other, but also lost interest and attention, preventing the effectiveness of cultural exchange from being maximized.

[0490] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for authenticating the user by comparing authentication information entered by the user with information stored in a database, means for retrieving and providing information on multiple cultures from the database, means for recommending optimal communication partners based on the user's profile information, means for translating messages sent by the user in real time and saving and sending the translated messages to the other user's device, means for analyzing the emotions of the messages using an emotion engine and adjusting the tone and nuance of the translation based on the results, and means for providing cultural information and related product information and promoting intercultural exchange. This enables users with different cultural backgrounds to communicate smoothly and foster deeper mutual understanding.

[0491] "User authentication" refers to authenticating a user by checking the authentication information entered by the user against the information stored in a database.

[0492] "Cultural information" is detailed information about different cultures that is retrieved from a database and provided to the user.

[0493] "Interaction Partners" refers to other users who share common interests and concerns and are recommended as suitable partners based on the user's profile information.

[0494] "Real-time translation" means instantly translating a message sent by a user into another language, and then saving and sending the translated message to the other user's terminal.

[0495] The "emotion engine" is a system that analyzes the emotions in messages sent by users and adjusts the tone and nuance of the translation based on the results.

[0496] "Cultural information provision" is a means of promoting intercultural exchange by providing information on cultures that interest users and related product information.

[0497] A "chat log" is a record used to store and manage the history of a conversation, including translated messages and the results of sentiment analysis.

[0498] The present invention is a system that includes functions for user authentication, provision of cultural information, matching of exchange partners, real-time translation and storage of messages, and integration of an emotion engine, and promotes smooth communication between users with different cultural backgrounds.

[0499] First, the server has a means for user authentication. Specifically, it checks the authentication information (user name and password) entered by the user against the information in the database, and if there is a match, it authenticates the user. This authentication process ensures secure access to the system.

[0500] The server then has a means for retrieving and providing information about multiple cultures from a database. When a user selects a cultural topic of interest, the terminal transmits the selection information to the server, which then retrieves the requested cultural information and returns it to the terminal. This allows the user to view detailed cultural information.

[0501] The server also has a means for recommending the most suitable exchange partner based on the user's profile information. When the user clicks the "Find a cultural exchange partner" button, the terminal sends a partner search request to the server, and the server searches the database for the most suitable exchange partner based on the user's profile information and returns the result to the terminal.

[0502] Furthermore, the server has a means for translating messages sent by users in real time, and storing and transmitting the translated messages to the terminals of the other users. When a user starts an interactive session and enters a message, the terminal sends the message to the server, which translates it into the target language using a real-time translation engine. The translated message is stored in a chat log and transmitted to the other users.

[0503] The emotion engine analyzes the emotion of the user's message and adjusts the tone and nuance of the translation based on the results. When the user enters and sends a message, the device requests the server to analyze the emotion of the message. The server uses the emotion engine to analyze the emotion in real time and reflects the appropriate tone and nuance in the translation. The emotion engine also saves the analysis results in the chat log and manages changes in emotion as a history.

[0504] A system for implementing the present invention includes a terminal such as a smartphone or a head-mounted display, a server using a RESTful API, and software such as Python or Flask / Django (server side), and React Native (client side).

[0505] As a concrete example, consider a situation in which a Japanese language enthusiast starts a conversation with a social partner by asking about how to celebrate the New Year. When the user enters a prompt such as, "Please explain about New Year's customs. I'd especially like to know more about hatsumode (first shrine visit of the year) and osechi ryori (New Year's food)," the system uses a generative AI model to provide detailed information. Furthermore, when the user sends a message such as, "How do you celebrate the New Year in your country?", the emotion engine analyzes the sentiment of the message and adjusts the translation tone to achieve natural communication.

[0506] As a result, users with different cultural backgrounds can communicate smoothly and foster deeper mutual understanding.

[0507] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0508] Step 1:

[0509] A user accesses a system using a terminal and enters authentication information (username and password). The terminal sends these credentials to a server, which checks them against information in a database. If there is a match, the server authenticates the user and allows access to the system. This ensures secure access.

[0510] Step 2:

[0511] The user selects a cultural topic of interest through the terminal. The terminal sends the selection information to the server, which then retrieves related cultural information from the database. The retrieved cultural information is then sent back to the terminal and displayed to the user, allowing the user to obtain detailed cultural information.

[0512] Step 3:

[0513] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server, which searches the database for the most suitable exchange partner based on the user's profile information. A list of suitable exchange partner candidates is returned to the device and displayed to the user. The user then selects the partner they would like to exchange with from the list of candidates. This completes the match.

[0514] Step 4:

[0515] A user inputs a message to start a conversation with a social partner. The device sends the message to the server, which translates the message into the target language using a real-time translation engine. The translated message is saved in the chat log and simultaneously sent to the other user's device. This enables real-time multilingual communication.

[0516] Step 5:

[0517] An emotion engine is built into messages entered by users. When a device sends a message to the server, it requests that the emotion of the message be analyzed. The server uses the emotion engine to analyze the emotion of the message in real time and adjusts the tone and nuance of the translation based on the results. For example, the message "I'm excited" is analyzed as "excited" and translated with an appropriate tone. The emotion results are saved in the chat log, and changes in emotion are managed as a history.

[0518] Step 6:

[0519] When a user requests specific cultural information or related product information, the device sends the request to the server. The server uses a generative AI model to generate and retrieve detailed information based on the requested information. The generated information is sent back to the device and displayed to the user. For example, detailed cultural information can be provided based on a prompt such as, "Please explain New Year's customs. I would particularly like to know more about hatsumode (first shrine visit of the year) and osechi ryori (New Year's food)." This allows the user to gain deeper knowledge about the culture they are interested in.

[0520] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0521] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0522] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0523] [Second embodiment]

[0524] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0525] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0526] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0527] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0528] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0529] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0530] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0531] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0532] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0533] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0534] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0535] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0536] The embodiment of the present invention first includes a user authentication process, a cultural information providing process, an exchange partner matching process, and a dialogue translation and storage process.

[0537] User Authentication

[0538] When a user accesses a system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the database, and if it matches, it authenticates the user and allows them access to the system.

[0539] Providing cultural information

[0540] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[0541] Matching exchange partners

[0542] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list of candidates.

[0543] Dialogue translation and storage

[0544] A user starts a conversation session and inputs a message. The device collects the input message and sends it to the server. The server translates the received message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[0545] Specific examples

[0546] For example, user A enters "I want to know about Japanese New Year customs," and as a result, he or she checks detailed information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" on his or her device. User A then finds a cultural exchange partner and is connected with user B. When user A sends a message saying, "How do you celebrate New Year in your country?", the message is translated by the server and converted into user B's language. User B replies, "We set off fireworks." The contents of the conversation are saved in a chat log and displayed on both devices.

[0547] As described above, this system includes functions for user authentication, provision of cultural information, matching of communication partners, and real-time translation and storage of messages, enabling users with different cultural backgrounds to communicate smoothly and promote mutual understanding.

[0548] The processing flow will be explained below.

[0549] Step 1:

[0550] The user accesses the login screen and enters their username and password.

[0551] Step 2:

[0552] The terminal collects the authentication information entered by the user and sends it to the server.

[0553] Step 3:

[0554] The server checks the received authentication information against the database, and if there is a match, authenticates the user and returns the authentication result to the terminal.

[0555] Step 4:

[0556] Users navigate to the Cultural Exchange section and select a cultural topic of interest.

[0557] Step 5:

[0558] The terminal transmits information on the cultural topic selected by the user to the server as a request.

[0559] Step 6:

[0560] Based on the request, the server retrieves the relevant cultural information from the database and returns the information to the terminal.

[0561] Step 7:

[0562] The terminal displays the received culture information to the user.

[0563] Step 8:

[0564] Users click the "Find a Cultural Exchange Partner" button.

[0565] Step 9:

[0566] The terminal sends a partner search request to the server.

[0567] Step 10:

[0568] The server searches a database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the terminal.

[0569] Step 11:

[0570] The user selects a partner from the candidate list and starts a dialogue session.

[0571] Step 12:

[0572] The user inputs a message, and the terminal transmits the input message to the server.

[0573] Step 13:

[0574] The server translates the received message in real time using a translation engine.

[0575] Step 14:

[0576] The server saves the translated message in the chat log and sends the translation result to the other party's device.

[0577] Step 15:

[0578] The terminal displays the translated message from the other party to the user.

[0579] Step 16:

[0580] The user checks the other person's message and continues the conversation.

[0581] Example 1

[0582] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0583] In modern society, smooth communication between people of different cultural backgrounds is essential to promoting mutual understanding. However, cultural exchange can be difficult due to language barriers, a lack of cultural information, and a lack of suitable exchange partners. There is a need to develop a system that can solve these issues.

[0584] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0585] In this invention, the server includes means for authenticating the user by comparing authentication information entered by the user with information stored in a database, means for retrieving and providing information on multiple cultures from the database, means for recommending optimal communication partners based on the user's profile information, and means for translating messages sent by the user in real time, saving the translated messages in a chat log, and sending them to the other user's device. This enables users with different cultural backgrounds to communicate smoothly and promote mutual understanding.

[0586] "User authentication information" refers to information such as a username and password that a user enters when accessing a system.

[0587] A "database" is a system for organizing and managing information, and stores user authentication information, cultural information, chat logs, etc.

[0588] "Cultural information" is information about the customs, practices, and events of different regions, countries, and societies.

[0589] "Exchange partners" are other users with different cultural backgrounds who are recommended to the user by the system for cultural exchange.

[0590] "Real-time translation" is the process of instantly translating a message sent by a user into another language.

[0591] A "chat log" is a database that records and stores the content of conversations between users.

[0592] "Server" means the central computer system that manages and executes the functions of the entire System.

[0593] "Authentication" refers to the process and technology used to authenticate a user by matching the user's credentials against a database.

[0594] "Providing means" refers to the process and technology for providing cultural information obtained from the database to the user's terminal.

[0595] "Recommendation tools" are processes and techniques for recommending optimal interaction partners based on a user's profile information.

[0596] "Translation means" refers to the process and technology for translating a message sent by a user in real time and sending it to the other user's terminal.

[0597] The "history database" is a database for managing the history of conversations including translated messages.

[0598] To implement this invention, a user authentication process, a cultural information provision process, an exchange partner matching process, and a dialogue translation and storage process are required.

[0599] User Authentication Process

[0600] When a user accesses the system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the database, and if it matches, it authenticates the user and allows them access to the system.

[0601] For example, when a user enters a "user name" and "password," the terminal sends this to the server, which then checks the database for authentication. If authentication is successful, the terminal displays "Authentication successful." An example of a prompt statement is "Please enter your user name and password."

[0602] Cultural information provision process

[0603] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[0604] As a concrete example, when a user selects the topic "Japanese New Year customs" and the device requests this from the server, the server retrieves information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" from the database and sends it to the device. The device then displays this information to the user. An example of a prompt sentence is "Please select a cultural topic that interests you."

[0605] Exchange Partner Matching Process

[0606] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list of candidates.

[0607] As a concrete example, when a user clicks the "Find a cultural exchange partner" button and the device sends a request to the server, the server searches for candidates such as "User B" and "User C" and sends them to the device. The device displays the candidate list to the user, and the user selects "User B." An example of a prompt sentence is "Click the button to find a cultural exchange partner."

[0608] Dialogue translation and storage process

[0609] A user starts a conversation session and inputs a message. The device collects the input message and sends it to the server. The server translates the message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[0610] For example, user A types "How do you celebrate New Year in your country?" and their device sends this message to the server, which translates the message and sends it in the language of "user B." User B replies, "We set off fireworks," and the reply is saved in the chat log and displayed on both devices.

[0611] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0612] User Authentication Process

[0613] Step 1:

[0614] The user enters their username and password into the login screen.

[0615] Input: Username, Password

[0616] Output: Credentials

[0617] Specific operation: The user enters a "user name" and "password."

[0618] Step 2:

[0619] The terminal collects this authentication information and sends it to the server.

[0620] Input: Credentials

[0621] Output: Request to server

[0622] Specific operation: The terminal sends the username and password to the server.

[0623] Step 3:

[0624] The server checks the received authentication information against a database.

[0625] Input: Credentials

[0626] Output: Matching result

[0627] What happens: The server checks the username and password in its database.

[0628] Step 4:

[0629] The server returns the result of the match to the terminal.

[0630] Input: Matching result

[0631] Output: Authentication result

[0632] Specific operation: The server sends the result of "authentication successful" or "authentication failed" to the terminal.

[0633] Step 5:

[0634] The terminal displays the authentication result to the user.

[0635] Input: Authentication result

[0636] Output: Display information

[0637] Specific operation: The device will display "Authentication successful" or "Authentication failed" on the screen.

[0638] Cultural information provision process

[0639] Step 1:

[0640] The user navigates to the Cultural Exchange section and selects a cultural topic of interest.

[0641] Input: Select a cultural topic

[0642] Output: Selection information

[0643] What happens: The user selects a topic, such as "Japanese New Year's Customs."

[0644] Step 2:

[0645] The terminal collects the user's selection information and sends a request to the server.

[0646] Input: Selection information

[0647] Output: Request to server

[0648] Specific operation: The terminal transmits the selection information to the server.

[0649] Step 3:

[0650] The server retrieves the requested cultural information from a database.

[0651] Input: Request information

[0652] Output:Cultural information

[0653] Specific operation: The server retrieves information such as "first shrine visit of the year" and "New Year's cuisine" from the database.

[0654] Step 4:

[0655] The server returns the acquired culture information to the terminal.

[0656] Input:Cultural information

[0657] Output:Return information

[0658] Specific operation: The server sends culture information to the terminal.

[0659] Step 5:

[0660] The terminal displays the returned information to the user.

[0661] Input:Return information

[0662] Output: Display information

[0663] Specific operation: The device displays the cultural information on the screen.

[0664] Exchange Partner Matching Process

[0665] Step 1:

[0666] The user clicks the "Find a Cultural Exchange Partner" button.

[0667] Input: Button click

[0668] Output: Partner search request

[0669] Specific action: The user clicks the "Find a cultural exchange partner" button.

[0670] Step 2:

[0671] The terminal sends a partner search request to the server.

[0672] Input: Search request

[0673] Output: Request to server

[0674] Specific operation: The device sends a search request to the server.

[0675] Step 3:

[0676] The server searches a database for the most suitable partner based on the user's profile information.

[0677] Input: Profile information

[0678] Output: List of potential partners

[0679] Specific operation: The server searches the database for candidates such as "User B" and "User C."

[0680] Step 4:

[0681] The server returns a list of potential partners to the terminal as a search result.

[0682] Input: List of potential partners

[0683] Output:Return information

[0684] Specific operation: The server sends the candidate list to the terminal.

[0685] Step 5:

[0686] The terminal displays the candidate list to the user.

[0687] Input:Return information

[0688] Output: Display information

[0689] Specific operation: The device displays the candidate list on the screen.

[0690] Step 6:

[0691] The user selects the partner with whom they wish to interact from a list of candidates.

[0692] Input: Partner selection

[0693] Output: Selection information

[0694] Specific operation: The user selects "User B."

[0695] Dialogue translation and storage process

[0696] Step 1:

[0697] The user initiates an interactive session and enters a message.

[0698] Input: Message

[0699] Output: The input message

[0700] What happens: User types, "How do you celebrate New Year's in your country?"

[0701] Step 2:

[0702] The terminal collects the input messages and sends them to the server.

[0703] Input: The message entered

[0704] Output: Request to server

[0705] Specific operation: The device sends a message to the server.

[0706] Step 3:

[0707] The server translates the message into the target language using a real-time translation engine.

[0708] Input: The message entered

[0709] Output: Translated message

[0710] Specific behavior: The server translates "How do you celebrate New Year in your country?" into the other person's language.

[0711] Step 4:

[0712] The server stores the translated messages in the chat log.

[0713] Input: Translated message

[0714] Output: Saved messages

[0715] What happens next: The server stores the translated message in a database.

[0716] Step 5:

[0717] The server sends the translated message to the other user's terminal.

[0718] Input: Translated message

[0719] Output: Send to the other terminal

[0720] Specific operation: The server sends the translated message to the other device.

[0721] Step 6:

[0722] The user can check the translated message on their device and continue the conversation.

[0723] Input: Translated message

[0724] Output: The displayed message

[0725] Specific action: The user sees the reply from the other party on their device: "We're going to set off fireworks."

[0726] (Application example 1)

[0727] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0728] For users with different cultural backgrounds to communicate smoothly while shopping online, real-time language translation and cultural and product information are necessary. Conventional systems do not integrate these elements and are unable to provide the diverse information and interaction opportunities users desire. Therefore, the challenge is to enable users from different cultural backgrounds to share cultural and product information while also enabling smooth communication with partners.

[0729] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0730] In this invention, the server includes means for authenticating the user by comparing authentication information entered by the user with information stored in a database, means for retrieving and providing information about multiple cultures from the database, means for recommending optimal communication partners based on the user's profile information, means for translating messages sent by the user in real time and storing and sending the translated messages to the other user's terminal, and means for users to retrieve information about cultures and products of different countries while shopping online and communicate with sales staff and other shopping users in the local language, thereby enabling users from different cultural backgrounds to share cultural and product information and communicate smoothly with partners.

[0731] "User authentication" refers to the process of verifying a user's identity by matching the authentication information entered by the user with information stored in a database.

[0732] "Cultural information provision" refers to the process of retrieving information about multiple cultures from a database and providing it appropriately to the user.

[0733] "Exchange partner recommendation" refers to the process of selecting and recommending the most suitable exchange partner to the user based on the user's profile information.

[0734] "Real-time translation" refers to the process of translating a message sent by a user into a different language in real time, and then storing and sending the translated message to the other user's device.

[0735] "Cultural information provision during online shopping" refers to the process of retrieving and providing information about cultures and products of different countries when users are shopping online.

[0736] "Communication methods" refers to the functions and processes for communicating in real time with salespeople and other shopping users in the local language.

[0737] As an embodiment of this invention, we will explain a system that includes user authentication, cultural information provision, exchange partner recommendation, real-time translation, cultural information provision during online shopping, and communication means. This system is configured using a program that uses the Flask framework and the Google Translate API.

[0738] User Authentication

[0739] When a user accesses the system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the information stored in the database, and if it matches, the user is authenticated and allowed to access the system.

[0740] Providing cultural information

[0741] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[0742] Exchange partner recommendation

[0743] When a user clicks the "Find a cultural exchange partner" button, the device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list.

[0744] Real-time translation

[0745] When a user starts a conversation session and inputs a message, the device collects the input message and sends it to the server. The server translates the received message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[0746] Providing cultural information while shopping online

[0747] When a user selects a specific item to retrieve information about the culture or products of different countries while shopping online, the terminal sends the request to the server, which retrieves the culture and product information from the database and returns it to the terminal, which uses the information to enrich the user's shopping experience.

[0748] Communication tools

[0749] A real-time translation function is used by users to communicate with other users and salespeople while online shopping. The server translates messages and sends the translated messages to each user's device.

[0750] Specific examples

[0751] For example, if User A is interested in French wine and requests cultural information about it, information about the history and brewing methods of wine will be displayed. If User A then communicates with a French salesperson and wants more detailed information, the message will be translated in real time and sent to the salesperson. The salesperson's reply will also be translated and displayed on User A's screen, enabling smooth communication.

[0752] Example prompt sentence:

[0753] 1. Log in:

[0754] Enter your username and password to log in

[0755] "username: user1, password: password1"

[0756] 2. Obtaining cultural information:

[0757] User requests information about "wine"

[0758] "topic: wine"

[0759] 3. Matching exchange partners:

[0760] Provide your profile information to find a partner

[0761] "profile: User Profile Data"

[0762] 4. Message translation and dialogue:

[0763] Translate and communicate messages

[0764] "message: Tell me about French wine, target_language: fr"

[0765] In this way, a system is realized that allows users to obtain information about different cultures while shopping online, and promote communication with their partners.

[0766] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0767] Step 1:

[0768] When a user accesses the system, they enter a username and password on a login screen.

[0769] Input: Username, Password

[0770] Operation: The device collects the authentication information entered by the user and sends it to the server.

[0771] Data processing: The server receives the authentication information and checks it against the information stored in the database.

[0772] Output: Authentication success / failure message

[0773] Step 2:

[0774] The user navigates to the Cultural Exchange section and selects a cultural topic of interest.

[0775] Input: Cultural topic selection information

[0776] Operation: The terminal collects the user's selection information and sends it to the server.

[0777] Data processing: The server retrieves the relevant cultural information from the database.

[0778] Output: Retrieved cultural information

[0779] Step 3:

[0780] The user clicks the "Find a Cultural Exchange Partner" button.

[0781] Input: Partner Search Request

[0782] Operation: The device sends a request to the server.

[0783] Data processing: The server searches the database for the most suitable partner based on the user's profile information.

[0784] Output: List of potential partners

[0785] Step 4:

[0786] The user selects the partner they want to interact with from a list of candidates.

[0787] Input: Partner selection information

[0788] Operation: The device sends the selection information to the server, which then sends a notification to the selected partner.

[0789] Output: Connection status with partner

[0790] Step 5:

[0791] The user initiates an interactive session and enters a message.

[0792] Input: Dialogue message

[0793] Operation: The terminal sends the inputted interaction message to the server.

[0794] Data processing: The server translates the message using a real-time translation engine.

[0795] Output: Save the translated message and send it to the other user's device

[0796] Step 6:

[0797] Users select specific items to retrieve specific cultural or product information while shopping online.

[0798] Input: Request for cultural or product information

[0799] Operation: The device sends a request to the server.

[0800] Data processing: The server retrieves the relevant information from the database and returns it to the terminal.

[0801] Output: Returned culture and product information

[0802] Step 7:

[0803] While a user is shopping online, the user communicates with sales staff and other users in real time.

[0804] Input: Dialogue message

[0805] Operation: The terminal sends the input message to the server, which translates the message and sends it to the other terminal.

[0806] Data processing: Message translation and saving to chat log

[0807] Output: Sends translated message and updates chat log

[0808] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0809] The embodiment of the present invention includes a user authentication process, a cultural information providing process, an exchange partner matching process, a dialogue translation and storage process, and an emotion engine integration process.

[0810] User Authentication

[0811] When a user accesses a system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the database, and if it matches, it authenticates the user and allows them access to the system.

[0812] Providing cultural information

[0813] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[0814] Matching exchange partners

[0815] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list of candidates.

[0816] Dialogue translation and storage

[0817] A user starts a conversation session and inputs a message. The device sends the input message to the server. The server translates the received message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[0818] Emotion engine integration

[0819] An emotion engine is built into messages entered by users. When a device sends a message to the server, it requests that the emotion of the message be analyzed. The server uses the emotion engine to analyze the emotion of the message in real time and adjusts the tone and nuance of the translation based on the results. For example, if user A enters "I'm excited," the emotion engine analyzes the emotion as "excited" and translates it with an appropriate tone. The emotion engine also saves the results in the chat log and manages changes in emotion as a history.

[0820] Specific examples

[0821] For example, User A types, "I want to know about Japanese New Year customs," and checks detailed information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" on his device. User A then finds a cultural exchange partner and is connected with User B. When User A sends a message saying, "How do you celebrate the New Year in your country?", the message is translated in real time by the server and converted into User B's language. At this time, User A's emotions are also analyzed, so User B can receive the translated message, "We will set off fireworks," in a natural tone.

[0822] As described above, this system includes functions for user authentication, provision of cultural information, matching of communication partners, real-time translation and storage of messages, and integration of an emotion engine, enabling users with different cultural backgrounds to communicate smoothly and promote deeper mutual understanding.

[0823] The processing flow will be explained below.

[0824] Step 1:

[0825] The user accesses the login screen and enters their username and password.

[0826] Step 2:

[0827] The terminal collects the authentication information entered by the user and sends it to the server.

[0828] Step 3:

[0829] The server checks the received authentication information against the database, and if there is a match, authenticates the user and returns the authentication result to the terminal.

[0830] Step 4:

[0831] Users navigate to the Cultural Exchange section and select a cultural topic of interest.

[0832] Step 5:

[0833] The terminal transmits information on the cultural topic selected by the user to the server as a request.

[0834] Step 6:

[0835] Based on the request, the server retrieves the relevant cultural information from a database and returns the information to the terminal.

[0836] Step 7:

[0837] The terminal displays the received culture information to the user.

[0838] Step 8:

[0839] Users click the "Find a Cultural Exchange Partner" button.

[0840] Step 9:

[0841] The terminal sends a partner search request to the server.

[0842] Step 10:

[0843] The server searches a database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the terminal.

[0844] Step 11:

[0845] The user selects a partner from the candidate list and starts a dialogue session.

[0846] Step 12:

[0847] The user inputs a message, and the terminal requests emotion analysis by the emotion engine when transmitting the input message to the server.

[0848] Step 13:

[0849] The server analyzes the received message in real time using an emotion engine to identify the emotion of the message.

[0850] Step 14:

[0851] The server adjusts the tone and nuance of the translation as needed based on the emotions identified by the emotion engine.

[0852] Step 15:

[0853] The server saves the adjusted translated message in the chat log and sends the translation result to the other party's device.

[0854] Step 16:

[0855] The terminal displays the translated message from the other party to the user.

[0856] Step 17:

[0857] The device also stores the emotional data of messages in a chat log and manages changes in emotions as a history.

[0858] Step 18:

[0859] The user checks the other person's message and continues the conversation.

[0860] For example, if user A types "I want to know about Japanese New Year customs," the information is returned to the device and displayed. User A then finds a social partner and is connected with user B. If user A sends a message saying "How do you celebrate the New Year in your country?", the server analyzes the message and identifies the emotion "excited." The translation result is provided to user B in an appropriate tone, enabling natural conversation that takes into account each other's emotions.

[0861] Example 2

[0862] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0863] While conventional cultural exchange systems provide basic functions such as user authentication, cultural information provision, partner matching, and message translation and storage, they lack the ability to adjust emotional nuances during translation or the ability to store and manage emotional changes in chat logs. This makes it difficult for users with different cultural backgrounds to communicate smoothly and naturally. The objective of this invention is to resolve these shortcomings and promote deeper mutual understanding and smoother communication between users.

[0864] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0865] means for authenticating a user by matching authentication information entered by the user with information stored in a database;

[0866] a means of retrieving and providing information about multiple cultures from a database;

[0867] A means of recommending optimal interaction partners based on the user's profile information;

[0868] A means for translating a message sent by a user in real time, and storing and transmitting the translated message to the terminal of the other user;

[0869] A means to analyze the message to be translated using an emotion engine and adjust the tone and nuance of the translation,

[0870] A means to save the analysis results of the emotion engine in a chat log and manage changes in emotions as a history,

[0871] This allows users with different cultural backgrounds to communicate smoothly and naturally and manage emotional changes, enabling deeper mutual understanding.

[0872] "User authentication" is the process of verifying the identity of a user who accesses a system by comparing the authentication information entered by the user with the information stored in a database.

[0873] "Cultural information" is information about multiple cultures, and is a collection of knowledge and data that is provided by user selection.

[0874] An "exchange partner" is a user with whom you can engage in cultural exchange, optimally selected based on your profile information.

[0875] "Message translation" is the process of translating messages sent by users into other languages ​​in real time.

[0876] A "translated message" is a message after the original message has been translated by a real-time translation engine.

[0877] The "emotion engine" is an engine that analyzes messages entered by users and adjusts the tone and nuance based on emotions.

[0878] A "chat log" is a database for storing and managing the history of conversations between users.

[0879] "Emotion analysis results" are data related to emotions obtained by the emotion engine analyzing the user's message.

[0880] The embodiment of the present invention mainly includes a user authentication process, a cultural information providing process, an exchange partner matching process, a dialogue translation and storage process, and an emotion engine integration process.

[0881] User Authentication Process

[0882] When a user accesses the system, they enter their username and password on a login screen. This authentication information is collected by the terminal and sent to the server. The server compares the received authentication information with its database, and if it matches, it authenticates the user and allows them access to the system. This series of processes uses AWS's DynamoDB and the HTTP protocol.

[0883] Cultural information provision process

[0884] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device sends the user's selection as a request to the server. The server retrieves the cultural information from the database using MongoDB and returns it to the device. The returned information is displayed by the device for the user to review.

[0885] Exchange Partner Matching Process

[0886] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server uses Elasticsearch to search the database based on the user's profile information, creates a list of suitable partners, and returns it to the device. The user can then select the desired partner from the list of candidates.

[0887] Dialogue translation and storage process

[0888] A user starts a conversation session and inputs a message. The device sends the input message to the server. The server translates the message in real time using the Google Cloud Translate API, stores the translated message in DynamoDB, and sends it to the other user's device. The user can then view the translated message and continue the conversation.

[0889] Emotion Engine Integration Process

[0890] An emotion engine is built into messages entered by users. When a device sends a message to the server, it requests that the emotion engine be used to analyze the message's emotion. The server uses IBM Watson's NLP emotion analysis API to analyze the emotion of the message in real time and adjusts the tone and nuance of the translation based on the results. The emotion engine's analysis results are also saved in the chat log, and changes in emotion are managed as a history.

[0891] Specific examples

[0892] For example, User A types, "I want to know about Japanese New Year customs," and checks detailed information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" on his device. User A then finds a cultural exchange partner and is connected with User B. When User A sends a message saying, "How do you celebrate the New Year in your country?", the message is translated in real time by the server and converted into User B's language. At this time, User A's emotions are also analyzed, so User B can receive the translated message, "We will set off fireworks," in a natural tone.

[0893] Prompt Sentence Examples

[0894] An example of a prompt sentence to be input to the generative AI model is, "Please explain the process that occurs when a user enters their username and password on the login screen."

[0895] As described above, this system includes functions for user authentication, provision of cultural information, matching of communication partners, real-time translation and storage of messages, and even integration of an emotion engine, enabling users with different cultural backgrounds to communicate smoothly and promote deeper mutual understanding.

[0896] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0897] Step 1:

[0898] A user enters a username and password into a login screen to access the system.

[0899] Specific operation: The user enters a username and password in the input fields and presses the login button.

[0900] Input: Username and Password

[0901] Output: User credentials passed to the terminal

[0902] Step 2:

[0903] The terminal collects the entered username and password and sends them to the server.

[0904] Specific operation: The terminal sends the input data to the server as an HTTP POST request.

[0905] Input: User credentials

[0906] Output: HTTP request passed to the server

[0907] Step 3:

[0908] The server checks the received authentication information against a database, and if there is a match, it authenticates the user and allows them access to the system.

[0909] What happens: The server queries DynamoDB to find a matching record, and if authentication is successful, returns a success message in the HTTP response.

[0910] Input: User credentials

[0911] Output: Authentication result (success / failure)

[0912] Step 4:

[0913] Users navigate to the Cultural Exchange section and select a cultural topic of interest.

[0914] Specific operation: The user selects a cultural topic from the UI and presses the select button.

[0915] Input: Cultural topic selection

[0916] Output: Selection information passed to the terminal

[0917] Step 5:

[0918] The terminal collects the user's selection information and sends a request to the server.

[0919] Specific operation: The terminal sends the selected data to the server as an HTTP GET request.

[0920] Input: Cultural topic selection information

[0921] Output: HTTP request passed to the server

[0922] Step 6:

[0923] Upon receiving the request, the server retrieves the culture information from the database and returns it to the terminal.

[0924] Specific operation: The server searches MongoDB, obtains the corresponding culture information, and returns it in the HTTP response.

[0925] Input: Cultural topic selection information

[0926] Output: Culture information sent back to the terminal

[0927] Step 7:

[0928] The terminal displays the culture information received from the server so that the user can check it.

[0929] Specific operation: The device parses the received JSON data and displays it on the screen.

[0930] Input:Cultural information

[0931] Output: Culture information displayed on the screen

[0932] Step 8:

[0933] The user clicks on the "Find a Cultural Exchange Partner" button.

[0934] Specific Action: The user clicks a button in the UI.

[0935] Input: Button click event

[0936] Output: Click event passed to the terminal

[0937] Step 9:

[0938] The terminal sends a partner search request to the server.

[0939] Specific operation: The terminal sends an HTTP GET request including user information to the server.

[0940] Input: User information, click event

[0941] Output: HTTP request passed to the server

[0942] Step 10:

[0943] The server searches a database based on the user's profile information to find the most suitable partner and returns a list of candidate partners to the terminal.

[0944] What happens: The server uses Elasticsearch to search the database and return a list of matching users.

[0945] Input: User information

[0946] Output: A list of potential partners returned to the terminal

[0947] Step 11:

[0948] The terminal displays a list of candidate partners to the user, allowing the user to make a selection.

[0949] Specific operation: The device displays the received list on the screen and provides a UI that allows the user to select.

[0950] Input: List of potential partners

[0951] Output: Partner list displayed on screen

[0952] Step 12:

[0953] The user initiates an interactive session and enters a message.

[0954] Specific operation: The user enters a message in the text box of the UI and presses the send button.

[0955] Input: Message content

[0956] Output: Message content passed to the terminal

[0957] Step 13:

[0958] The terminal transmits the input message to the server.

[0959] Specific operation: The device sends an HTTP POST request containing a message to the server.

[0960] Input: Message content

[0961] Output: HTTP request passed to the server

[0962] Step 14:

[0963] The server translates the message in real time using the Google Cloud Translate API, stores the translated message in DynamoDB, and sends it to the other user's device.

[0964] Specific operation: The server calls the translation API, stores the translation results in DynamoDB, and sends them to the other device.

[0965] Input: Message content

[0966] Output: translated messages, saved chat logs

[0967] Step 15:

[0968] The other party's terminal displays the translated message received from the server to the user.

[0969] Specific operation: The device parses the received JSON data and displays it on the screen.

[0970] Input: Translated message

[0971] Output: Messages displayed on the screen

[0972] Step 16:

[0973] The device requests the server to analyze the emotions of the sent messages in real time.

[0974] Specific operation: The device adds an emotion analysis flag to an HTTP POST request containing a message and sends it to the server.

[0975] Input: Message content, sentiment analysis flag

[0976] Output: HTTP request passed to the server

[0977] Step 17:

[0978] The server uses IBM Watson's NLP sentiment analysis API to analyze messages in real time and set the tone and nuance of the translation based on the results.

[0979] Specific operation: The server calls the sentiment analysis API and passes the results to the Google Cloud Translate API for translation.

[0980] Input: Message content

[0981] Output: Sentiment analysis results, adjusted translation tone

[0982] Step 18:

[0983] The server stores the emotion analysis results in a chat log and manages changes in emotions as a history.

[0984] Specific operation: The server stores the sentiment analysis results in DynamoDB so that they can be retrieved later.

[0985] Input: Sentiment analysis results

[0986] Output: Saved sentiment analysis results

[0987] Through the above steps, communication between users is facilitated and a dialogue history including changes in emotions is managed.

[0988] (Application example 2)

[0989] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0990] Conventional cultural exchange systems have faced challenges in enabling smooth communication between users with different languages ​​and cultural backgrounds. In particular, they lacked the ability to find suitable exchange partners, perform real-time translation between multiple languages, and analyze emotions to maintain a natural tone in the dialogue. As a result, users not only failed to deepen their understanding of each other, but also lost interest and attention, preventing the effectiveness of cultural exchange from being maximized.

[0991] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for authenticating the user by comparing authentication information entered by the user with information stored in a database, means for retrieving and providing information on multiple cultures from the database, means for recommending optimal communication partners based on the user's profile information, means for translating messages sent by the user in real time and saving and sending the translated messages to the other user's device, means for analyzing the emotions of the messages using an emotion engine and adjusting the tone and nuance of the translation based on the results, and means for providing cultural information and related product information and promoting intercultural exchange. This enables users with different cultural backgrounds to communicate smoothly and foster deeper mutual understanding.

[0992] "User authentication" refers to authenticating a user by checking the authentication information entered by the user against the information stored in a database.

[0993] "Cultural information" is detailed information about different cultures that is retrieved from a database and provided to the user.

[0994] "Interaction Partners" refers to other users who share common interests and concerns and are recommended as suitable partners based on the user's profile information.

[0995] "Real-time translation" means instantly translating a message sent by a user into another language, and then saving and sending the translated message to the other user's terminal.

[0996] The "emotion engine" is a system that analyzes the emotions in messages sent by users and adjusts the tone and nuance of the translation based on the results.

[0997] "Cultural information provision" is a means of promoting intercultural exchange by providing information on cultures that interest users and related product information.

[0998] A "chat log" is a record used to store and manage the history of a conversation, including translated messages and the results of sentiment analysis.

[0999] The present invention is a system that includes functions for user authentication, provision of cultural information, matching of exchange partners, real-time translation and storage of messages, and integration of an emotion engine, and promotes smooth communication between users with different cultural backgrounds.

[1000] First, the server has a means for user authentication. Specifically, it checks the authentication information (user name and password) entered by the user against the information in the database, and if there is a match, it authenticates the user. This authentication process ensures secure access to the system.

[1001] The server then has a means for retrieving and providing information about multiple cultures from a database. When a user selects a cultural topic of interest, the terminal transmits the selection information to the server, which then retrieves the requested cultural information and returns it to the terminal. This allows the user to view detailed cultural information.

[1002] The server also has a means for recommending the most suitable exchange partner based on the user's profile information. When the user clicks the "Find a cultural exchange partner" button, the terminal sends a partner search request to the server, and the server searches the database for the most suitable exchange partner based on the user's profile information and returns the result to the terminal.

[1003] Furthermore, the server has a means for translating messages sent by users in real time, and storing and transmitting the translated messages to the terminals of the other users. When a user starts an interactive session and enters a message, the terminal sends the message to the server, which translates it into the target language using a real-time translation engine. The translated message is stored in a chat log and transmitted to the other users.

[1004] The emotion engine analyzes the emotion of the user's message and adjusts the tone and nuance of the translation based on the results. When the user enters and sends a message, the device requests the server to analyze the emotion of the message. The server uses the emotion engine to analyze the emotion in real time and reflects the appropriate tone and nuance in the translation. The emotion engine also saves the analysis results in the chat log and manages changes in emotion as a history.

[1005] A system for implementing the present invention includes a terminal such as a smartphone or a head-mounted display, a server using a RESTful API, and software such as Python or Flask / Django (server side), and React Native (client side).

[1006] As a concrete example, consider a situation in which a Japanese language enthusiast starts a conversation with a social partner by asking about how to celebrate the New Year. When the user enters a prompt such as, "Please explain about New Year's customs. I'd especially like to know more about hatsumode (first shrine visit of the year) and osechi ryori (New Year's food)," the system uses a generative AI model to provide detailed information. Furthermore, when the user sends a message such as, "How do you celebrate the New Year in your country?", the emotion engine analyzes the sentiment of the message and adjusts the translation tone to achieve natural communication.

[1007] As a result, users with different cultural backgrounds can communicate smoothly and foster deeper mutual understanding.

[1008] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1009] Step 1:

[1010] A user accesses a system using a terminal and enters authentication information (username and password). The terminal sends these credentials to a server, which checks them against information in a database. If there is a match, the server authenticates the user and allows access to the system. This ensures secure access.

[1011] Step 2:

[1012] The user selects a cultural topic of interest through the terminal. The terminal sends the selection information to the server, which then retrieves related cultural information from the database. The retrieved cultural information is then sent back to the terminal and displayed to the user, allowing the user to obtain detailed cultural information.

[1013] Step 3:

[1014] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server, which searches the database for the most suitable exchange partner based on the user's profile information. A list of suitable exchange partner candidates is returned to the device and displayed to the user. The user then selects the partner they would like to exchange with from the list of candidates. This completes the match.

[1015] Step 4:

[1016] A user inputs a message to start a conversation with a social partner. The device sends the message to the server, which translates the message into the target language using a real-time translation engine. The translated message is saved in the chat log and simultaneously sent to the other user's device. This enables real-time multilingual communication.

[1017] Step 5:

[1018] An emotion engine is built into messages entered by users. When a device sends a message to the server, it requests that the emotion of the message be analyzed. The server uses the emotion engine to analyze the emotion of the message in real time and adjusts the tone and nuance of the translation based on the results. For example, the message "I'm excited" is analyzed as "excited" and translated with an appropriate tone. The emotion results are saved in the chat log, and changes in emotion are managed as a history.

[1019] Step 6:

[1020] When a user requests specific cultural information or related product information, the device sends the request to the server. The server uses a generative AI model to generate and retrieve detailed information based on the requested information. The generated information is sent back to the device and displayed to the user. For example, detailed cultural information can be provided based on a prompt such as, "Please explain New Year's customs. I would particularly like to know more about hatsumode (first shrine visit of the year) and osechi ryori (New Year's food)." This allows the user to gain deeper knowledge about the culture they are interested in.

[1021] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1022] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1023] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1024] [Third embodiment]

[1025] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1026] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1028] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1029] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1030] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1032] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1033] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1035] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1036] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1037] The embodiment of the present invention first includes a user authentication process, a cultural information providing process, an exchange partner matching process, and a dialogue translation and storage process.

[1038] User Authentication

[1039] When a user accesses a system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the database, and if it matches, it authenticates the user and allows them access to the system.

[1040] Providing cultural information

[1041] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[1042] Matching exchange partners

[1043] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list of candidates.

[1044] Dialogue translation and storage

[1045] A user starts a conversation session and inputs a message. The device collects the input message and sends it to the server. The server translates the received message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[1046] Specific examples

[1047] For example, user A enters "I want to know about Japanese New Year customs," and as a result, he or she checks detailed information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" on his or her device. User A then finds a cultural exchange partner and is connected with user B. When user A sends a message saying, "How do you celebrate New Year in your country?", the message is translated by the server and converted into user B's language. User B replies, "We set off fireworks." The contents of the conversation are saved in a chat log and displayed on both devices.

[1048] As described above, this system includes functions for user authentication, provision of cultural information, matching of communication partners, and real-time translation and storage of messages, enabling users with different cultural backgrounds to communicate smoothly and promote mutual understanding.

[1049] The processing flow will be explained below.

[1050] Step 1:

[1051] The user accesses the login screen and enters their username and password.

[1052] Step 2:

[1053] The terminal collects the authentication information entered by the user and sends it to the server.

[1054] Step 3:

[1055] The server checks the received authentication information against the database, and if there is a match, authenticates the user and returns the authentication result to the terminal.

[1056] Step 4:

[1057] Users navigate to the Cultural Exchange section and select a cultural topic of interest.

[1058] Step 5:

[1059] The terminal transmits information on the cultural topic selected by the user to the server as a request.

[1060] Step 6:

[1061] Based on the request, the server retrieves the relevant cultural information from the database and returns the information to the terminal.

[1062] Step 7:

[1063] The terminal displays the received culture information to the user.

[1064] Step 8:

[1065] Users click the "Find a Cultural Exchange Partner" button.

[1066] Step 9:

[1067] The terminal sends a partner search request to the server.

[1068] Step 10:

[1069] The server searches a database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the terminal.

[1070] Step 11:

[1071] The user selects a partner from the candidate list and starts a dialogue session.

[1072] Step 12:

[1073] The user inputs a message, and the terminal transmits the input message to the server.

[1074] Step 13:

[1075] The server translates the received message in real time using a translation engine.

[1076] Step 14:

[1077] The server saves the translated message in the chat log and sends the translation result to the other party's device.

[1078] Step 15:

[1079] The terminal displays the translated message from the other party to the user.

[1080] Step 16:

[1081] The user checks the other person's message and continues the conversation.

[1082] Example 1

[1083] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1084] In modern society, smooth communication between people of different cultural backgrounds is essential to promoting mutual understanding. However, cultural exchange can be difficult due to language barriers, a lack of cultural information, and a lack of suitable exchange partners. There is a need to develop a system that can solve these issues.

[1085] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1086] In this invention, the server includes means for authenticating the user by comparing authentication information entered by the user with information stored in a database, means for retrieving and providing information on multiple cultures from the database, means for recommending optimal communication partners based on the user's profile information, and means for translating messages sent by the user in real time, saving the translated messages in a chat log, and sending them to the other user's device. This enables users with different cultural backgrounds to communicate smoothly and promote mutual understanding.

[1087] "User authentication information" refers to information such as a username and password that a user enters when accessing a system.

[1088] A "database" is a system for organizing and managing information, and stores user authentication information, cultural information, chat logs, etc.

[1089] "Cultural information" is information about the customs, practices, and events of different regions, countries, and societies.

[1090] "Exchange partners" are other users with different cultural backgrounds who are recommended to the user by the system for cultural exchange.

[1091] "Real-time translation" is the process of instantly translating a message sent by a user into another language.

[1092] A "chat log" is a database that records and stores the content of conversations between users.

[1093] "Server" means the central computer system that manages and executes the functions of the entire System.

[1094] "Authentication" refers to the process and technology used to authenticate a user by matching the user's credentials against a database.

[1095] "Providing means" refers to the process and technology for providing cultural information obtained from the database to the user's terminal.

[1096] "Recommendation tools" are processes and techniques for recommending optimal interaction partners based on a user's profile information.

[1097] "Translation means" refers to the process and technology for translating a message sent by a user in real time and sending it to the other user's terminal.

[1098] The "history database" is a database for managing the history of conversations including translated messages.

[1099] To implement this invention, a user authentication process, a cultural information provision process, an exchange partner matching process, and a dialogue translation and storage process are required.

[1100] User Authentication Process

[1101] When a user accesses the system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the database, and if it matches, it authenticates the user and allows them access to the system.

[1102] For example, when a user enters a "user name" and "password," the terminal sends this to the server, which then checks the database for authentication. If authentication is successful, the terminal displays "Authentication successful." An example of a prompt statement is "Please enter your user name and password."

[1103] Cultural information provision process

[1104] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[1105] As a concrete example, when a user selects the topic "Japanese New Year customs" and the device requests this from the server, the server retrieves information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" from the database and sends it to the device. The device then displays this information to the user. An example of a prompt sentence is "Please select a cultural topic that interests you."

[1106] Exchange Partner Matching Process

[1107] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list of candidates.

[1108] As a concrete example, when a user clicks the "Find a cultural exchange partner" button and the device sends a request to the server, the server searches for candidates such as "User B" and "User C" and sends them to the device. The device displays the candidate list to the user, and the user selects "User B." An example of a prompt sentence is "Click the button to find a cultural exchange partner."

[1109] Dialogue translation and storage process

[1110] A user starts a conversation session and inputs a message. The device collects the input message and sends it to the server. The server translates the message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[1111] For example, user A types "How do you celebrate New Year in your country?" and their device sends this message to the server, which translates the message and sends it in the language of "user B." User B replies, "We set off fireworks," and the reply is saved in the chat log and displayed on both devices.

[1112] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1113] User Authentication Process

[1114] Step 1:

[1115] The user enters their username and password into the login screen.

[1116] Input: Username, Password

[1117] Output: Credentials

[1118] Specific operation: The user enters a "user name" and "password."

[1119] Step 2:

[1120] The terminal collects this authentication information and sends it to the server.

[1121] Input: Credentials

[1122] Output: Request to server

[1123] Specific operation: The terminal sends the username and password to the server.

[1124] Step 3:

[1125] The server checks the received authentication information against a database.

[1126] Input: Credentials

[1127] Output: Matching result

[1128] What happens: The server checks the username and password in its database.

[1129] Step 4:

[1130] The server returns the result of the match to the terminal.

[1131] Input: Matching result

[1132] Output: Authentication result

[1133] Specific operation: The server sends the result of "authentication successful" or "authentication failed" to the terminal.

[1134] Step 5:

[1135] The terminal displays the authentication result to the user.

[1136] Input: Authentication result

[1137] Output: Display information

[1138] Specific operation: The device will display "Authentication successful" or "Authentication failed" on the screen.

[1139] Cultural information provision process

[1140] Step 1:

[1141] The user navigates to the Cultural Exchange section and selects a cultural topic of interest.

[1142] Input: Select a cultural topic

[1143] Output: Selection information

[1144] What happens: The user selects a topic, such as "Japanese New Year's Customs."

[1145] Step 2:

[1146] The terminal collects the user's selection information and sends a request to the server.

[1147] Input: Selection information

[1148] Output: Request to server

[1149] Specific operation: The terminal transmits the selection information to the server.

[1150] Step 3:

[1151] The server retrieves the requested cultural information from a database.

[1152] Input: Request information

[1153] Output:Cultural information

[1154] Specific operation: The server retrieves information such as "first shrine visit of the year" and "New Year's cuisine" from the database.

[1155] Step 4:

[1156] The server returns the acquired culture information to the terminal.

[1157] Input:Cultural information

[1158] Output:Return information

[1159] Specific operation: The server sends culture information to the terminal.

[1160] Step 5:

[1161] The terminal displays the returned information to the user.

[1162] Input:Return information

[1163] Output: Display information

[1164] Specific operation: The device displays the cultural information on the screen.

[1165] Exchange Partner Matching Process

[1166] Step 1:

[1167] The user clicks the "Find a Cultural Exchange Partner" button.

[1168] Input: Button click

[1169] Output: Partner search request

[1170] Specific action: The user clicks the "Find a cultural exchange partner" button.

[1171] Step 2:

[1172] The terminal sends a partner search request to the server.

[1173] Input: Search request

[1174] Output: Request to server

[1175] Specific operation: The device sends a search request to the server.

[1176] Step 3:

[1177] The server searches a database for the most suitable partner based on the user's profile information.

[1178] Input: Profile information

[1179] Output: List of potential partners

[1180] Specific operation: The server searches the database for candidates such as "User B" and "User C."

[1181] Step 4:

[1182] The server returns a list of potential partners to the terminal as a search result.

[1183] Input: List of potential partners

[1184] Output:Return information

[1185] Specific operation: The server sends the candidate list to the terminal.

[1186] Step 5:

[1187] The terminal displays the candidate list to the user.

[1188] Input:Return information

[1189] Output: Display information

[1190] Specific operation: The device displays the candidate list on the screen.

[1191] Step 6:

[1192] The user selects the partner with whom they wish to interact from a list of candidates.

[1193] Input: Partner selection

[1194] Output: Selection information

[1195] Specific operation: The user selects "User B."

[1196] Dialogue translation and storage process

[1197] Step 1:

[1198] The user initiates an interactive session and enters a message.

[1199] Input: Message

[1200] Output: The input message

[1201] What happens: User types, "How do you celebrate New Year's in your country?"

[1202] Step 2:

[1203] The terminal collects the input messages and sends them to the server.

[1204] Input: The message entered

[1205] Output: Request to server

[1206] Specific operation: The device sends a message to the server.

[1207] Step 3:

[1208] The server translates the message into the target language using a real-time translation engine.

[1209] Input: The message entered

[1210] Output: Translated message

[1211] Specific behavior: The server translates "How do you celebrate New Year in your country?" into the other person's language.

[1212] Step 4:

[1213] The server stores the translated messages in the chat log.

[1214] Input: Translated message

[1215] Output: Saved messages

[1216] What happens next: The server stores the translated message in a database.

[1217] Step 5:

[1218] The server sends the translated message to the other user's terminal.

[1219] Input: Translated message

[1220] Output: Send to the other terminal

[1221] Specific operation: The server sends the translated message to the other device.

[1222] Step 6:

[1223] The user can check the translated message on their device and continue the conversation.

[1224] Input: Translated message

[1225] Output: The displayed message

[1226] Specific action: The user sees the reply from the other party on their device: "We're going to set off fireworks."

[1227] (Application example 1)

[1228] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1229] For users with different cultural backgrounds to communicate smoothly while shopping online, real-time language translation and cultural and product information are necessary. Conventional systems do not integrate these elements and are unable to provide the diverse information and interaction opportunities users desire. Therefore, the challenge is to enable users from different cultural backgrounds to share cultural and product information while also enabling smooth communication with partners.

[1230] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1231] In this invention, the server includes means for authenticating the user by comparing authentication information entered by the user with information stored in a database, means for retrieving and providing information about multiple cultures from the database, means for recommending optimal communication partners based on the user's profile information, means for translating messages sent by the user in real time and storing and sending the translated messages to the other user's terminal, and means for users to retrieve information about cultures and products of different countries while shopping online and communicate with sales staff and other shopping users in the local language, thereby enabling users from different cultural backgrounds to share cultural and product information and communicate smoothly with partners.

[1232] "User authentication" refers to the process of verifying a user's identity by matching the authentication information entered by the user with information stored in a database.

[1233] "Cultural information provision" refers to the process of retrieving information about multiple cultures from a database and providing it appropriately to the user.

[1234] "Exchange partner recommendation" refers to the process of selecting and recommending the most suitable exchange partner to the user based on the user's profile information.

[1235] "Real-time translation" refers to the process of translating a message sent by a user into a different language in real time, and then storing and sending the translated message to the other user's device.

[1236] "Cultural information provision during online shopping" refers to the process of retrieving and providing information about cultures and products of different countries when users are shopping online.

[1237] "Communication methods" refers to the functions and processes for communicating in real time with salespeople and other shopping users in the local language.

[1238] As an embodiment of this invention, we will explain a system that includes user authentication, cultural information provision, exchange partner recommendation, real-time translation, cultural information provision during online shopping, and communication means. This system is configured using a program that uses the Flask framework and the Google Translate API.

[1239] User Authentication

[1240] When a user accesses the system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the information stored in the database, and if it matches, the user is authenticated and allowed to access the system.

[1241] Providing cultural information

[1242] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[1243] Exchange partner recommendation

[1244] When a user clicks the "Find a cultural exchange partner" button, the device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list.

[1245] Real-time translation

[1246] When a user starts a conversation session and inputs a message, the device collects the input message and sends it to the server. The server translates the received message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[1247] Providing cultural information while shopping online

[1248] When a user selects a specific item to retrieve information about the culture or products of different countries while shopping online, the terminal sends the request to the server, which retrieves the culture and product information from the database and returns it to the terminal, which uses the information to enrich the user's shopping experience.

[1249] Communication tools

[1250] A real-time translation function is used by users to communicate with other users and salespeople while online shopping. The server translates messages and sends the translated messages to each user's device.

[1251] Specific examples

[1252] For example, if User A is interested in French wine and requests cultural information about it, information about the history and brewing methods of wine will be displayed. If User A then communicates with a French salesperson and wants more detailed information, the message will be translated in real time and sent to the salesperson. The salesperson's reply will also be translated and displayed on User A's screen, enabling smooth communication.

[1253] Example prompt sentence:

[1254] 1. Log in:

[1255] Enter your username and password to log in

[1256] "username: user1, password: password1"

[1257] 2. Obtaining cultural information:

[1258] User requests information about "wine"

[1259] "topic: wine"

[1260] 3. Matching exchange partners:

[1261] Provide your profile information to find a partner

[1262] "profile: User Profile Data"

[1263] 4. Message translation and dialogue:

[1264] Translate and communicate messages

[1265] "message: Tell me about French wine, target_language: fr"

[1266] In this way, a system is realized that allows users to obtain information about different cultures while shopping online, and promote communication with their partners.

[1267] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1268] Step 1:

[1269] When a user accesses the system, they enter a username and password on a login screen.

[1270] Input: Username, Password

[1271] Operation: The device collects the authentication information entered by the user and sends it to the server.

[1272] Data processing: The server receives the authentication information and checks it against the information stored in the database.

[1273] Output: Authentication success / failure message

[1274] Step 2:

[1275] The user navigates to the Cultural Exchange section and selects a cultural topic of interest.

[1276] Input: Cultural topic selection information

[1277] Operation: The terminal collects the user's selection information and sends it to the server.

[1278] Data processing: The server retrieves the relevant cultural information from the database.

[1279] Output: Retrieved cultural information

[1280] Step 3:

[1281] The user clicks the "Find a Cultural Exchange Partner" button.

[1282] Input: Partner Search Request

[1283] Operation: The device sends a request to the server.

[1284] Data processing: The server searches the database for the most suitable partner based on the user's profile information.

[1285] Output: List of potential partners

[1286] Step 4:

[1287] The user selects the partner they want to interact with from a list of candidates.

[1288] Input: Partner selection information

[1289] Operation: The device sends the selection information to the server, which then sends a notification to the selected partner.

[1290] Output: Connection status with partner

[1291] Step 5:

[1292] The user initiates an interactive session and enters a message.

[1293] Input: Dialogue message

[1294] Operation: The terminal sends the inputted interaction message to the server.

[1295] Data processing: The server translates the message using a real-time translation engine.

[1296] Output: Save the translated message and send it to the other user's device

[1297] Step 6:

[1298] Users select specific items to retrieve specific cultural or product information while shopping online.

[1299] Input: Request for cultural or product information

[1300] Operation: The device sends a request to the server.

[1301] Data processing: The server retrieves the relevant information from the database and returns it to the terminal.

[1302] Output: Returned culture and product information

[1303] Step 7:

[1304] While a user is shopping online, the user communicates with sales staff and other users in real time.

[1305] Input: Dialogue message

[1306] Operation: The terminal sends the input message to the server, which translates the message and sends it to the other terminal.

[1307] Data processing: Message translation and saving to chat log

[1308] Output: Sends translated message and updates chat log

[1309] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1310] The embodiment of the present invention includes a user authentication process, a cultural information providing process, an exchange partner matching process, a dialogue translation and storage process, and an emotion engine integration process.

[1311] User Authentication

[1312] When a user accesses a system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the database, and if it matches, it authenticates the user and allows them access to the system.

[1313] Providing cultural information

[1314] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[1315] Matching exchange partners

[1316] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list of candidates.

[1317] Dialogue translation and storage

[1318] A user starts a conversation session and inputs a message. The device sends the input message to the server. The server translates the received message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[1319] Emotion engine integration

[1320] An emotion engine is built into messages entered by users. When a device sends a message to the server, it requests that the emotion of the message be analyzed. The server uses the emotion engine to analyze the emotion of the message in real time and adjusts the tone and nuance of the translation based on the results. For example, if user A enters "I'm excited," the emotion engine analyzes the emotion as "excited" and translates it with an appropriate tone. The emotion engine also saves the results in the chat log and manages changes in emotion as a history.

[1321] Specific examples

[1322] For example, User A types, "I want to know about Japanese New Year customs," and checks detailed information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" on his device. User A then finds a cultural exchange partner and is connected with User B. When User A sends a message saying, "How do you celebrate the New Year in your country?", the message is translated in real time by the server and converted into User B's language. At this time, User A's emotions are also analyzed, so User B can receive the translated message, "We will set off fireworks," in a natural tone.

[1323] As described above, this system includes functions for user authentication, provision of cultural information, matching of communication partners, real-time translation and storage of messages, and integration of an emotion engine, enabling users with different cultural backgrounds to communicate smoothly and promote deeper mutual understanding.

[1324] The processing flow will be explained below.

[1325] Step 1:

[1326] The user accesses the login screen and enters their username and password.

[1327] Step 2:

[1328] The terminal collects the authentication information entered by the user and sends it to the server.

[1329] Step 3:

[1330] The server checks the received authentication information against the database, and if there is a match, authenticates the user and returns the authentication result to the terminal.

[1331] Step 4:

[1332] Users navigate to the Cultural Exchange section and select a cultural topic of interest.

[1333] Step 5:

[1334] The terminal transmits information on the cultural topic selected by the user to the server as a request.

[1335] Step 6:

[1336] Based on the request, the server retrieves the relevant cultural information from a database and returns the information to the terminal.

[1337] Step 7:

[1338] The terminal displays the received culture information to the user.

[1339] Step 8:

[1340] Users click the "Find a Cultural Exchange Partner" button.

[1341] Step 9:

[1342] The terminal sends a partner search request to the server.

[1343] Step 10:

[1344] The server searches a database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the terminal.

[1345] Step 11:

[1346] The user selects a partner from the candidate list and starts a dialogue session.

[1347] Step 12:

[1348] The user inputs a message, and the terminal requests emotion analysis by the emotion engine when transmitting the input message to the server.

[1349] Step 13:

[1350] The server analyzes the received message in real time using an emotion engine to identify the emotion of the message.

[1351] Step 14:

[1352] The server adjusts the tone and nuance of the translation as needed based on the emotions identified by the emotion engine.

[1353] Step 15:

[1354] The server saves the adjusted translated message in the chat log and sends the translation result to the other party's device.

[1355] Step 16:

[1356] The terminal displays the translated message from the other party to the user.

[1357] Step 17:

[1358] The device also stores the emotional data of messages in a chat log and manages changes in emotions as a history.

[1359] Step 18:

[1360] The user checks the other person's message and continues the conversation.

[1361] For example, if user A types "I want to know about Japanese New Year customs," the information is returned to the device and displayed. User A then finds a social partner and is connected with user B. If user A sends a message saying "How do you celebrate the New Year in your country?", the server analyzes the message and identifies the emotion "excited." The translation result is provided to user B in an appropriate tone, enabling natural conversation that takes into account each other's emotions.

[1362] Example 2

[1363] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1364] While conventional cultural exchange systems provide basic functions such as user authentication, cultural information provision, partner matching, and message translation and storage, they lack the ability to adjust emotional nuances during translation or the ability to store and manage emotional changes in chat logs. This makes it difficult for users with different cultural backgrounds to communicate smoothly and naturally. The objective of this invention is to resolve these shortcomings and promote deeper mutual understanding and smoother communication between users.

[1365] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1366] means for authenticating a user by matching authentication information entered by the user with information stored in a database;

[1367] a means of retrieving and providing information about multiple cultures from a database;

[1368] A means of recommending optimal interaction partners based on the user's profile information;

[1369] A means for translating a message sent by a user in real time, and storing and transmitting the translated message to the terminal of the other user;

[1370] A means to analyze the message to be translated using an emotion engine and adjust the tone and nuance of the translation,

[1371] A means to save the analysis results of the emotion engine in a chat log and manage changes in emotions as a history,

[1372] This allows users with different cultural backgrounds to communicate smoothly and naturally and manage emotional changes, enabling deeper mutual understanding.

[1373] "User authentication" is the process of verifying the identity of a user who accesses a system by comparing the authentication information entered by the user with the information stored in a database.

[1374] "Cultural information" is information about multiple cultures, and is a collection of knowledge and data that is provided by user selection.

[1375] An "exchange partner" is a user with whom you can engage in cultural exchange, optimally selected based on your profile information.

[1376] "Message translation" is the process of translating messages sent by users into other languages ​​in real time.

[1377] A "translated message" is a message after the original message has been translated by a real-time translation engine.

[1378] The "emotion engine" is an engine that analyzes messages entered by users and adjusts the tone and nuance based on emotions.

[1379] A "chat log" is a database for storing and managing the history of conversations between users.

[1380] "Emotion analysis results" are data related to emotions obtained by the emotion engine analyzing the user's message.

[1381] The embodiment of the present invention mainly includes a user authentication process, a cultural information providing process, an exchange partner matching process, a dialogue translation and storage process, and an emotion engine integration process.

[1382] User Authentication Process

[1383] When a user accesses the system, they enter their username and password on a login screen. This authentication information is collected by the terminal and sent to the server. The server compares the received authentication information with its database, and if it matches, it authenticates the user and allows them access to the system. This series of processes uses AWS's DynamoDB and the HTTP protocol.

[1384] Cultural information provision process

[1385] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device sends the user's selection as a request to the server. The server retrieves the cultural information from the database using MongoDB and returns it to the device. The returned information is displayed by the device for the user to review.

[1386] Exchange Partner Matching Process

[1387] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server uses Elasticsearch to search the database based on the user's profile information, creates a list of suitable partners, and returns it to the device. The user can then select the desired partner from the list of candidates.

[1388] Dialogue translation and storage process

[1389] A user starts a conversation session and inputs a message. The device sends the input message to the server. The server translates the message in real time using the Google Cloud Translate API, stores the translated message in DynamoDB, and sends it to the other user's device. The user can then view the translated message and continue the conversation.

[1390] Emotion Engine Integration Process

[1391] An emotion engine is built into messages entered by users. When a device sends a message to the server, it requests that the emotion engine be used to analyze the message's emotion. The server uses IBM Watson's NLP emotion analysis API to analyze the emotion of the message in real time and adjusts the tone and nuance of the translation based on the results. The emotion engine's analysis results are also saved in the chat log, and changes in emotion are managed as a history.

[1392] Specific examples

[1393] For example, User A types, "I want to know about Japanese New Year customs," and checks detailed information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" on his device. User A then finds a cultural exchange partner and is connected with User B. When User A sends a message saying, "How do you celebrate the New Year in your country?", the message is translated in real time by the server and converted into User B's language. At this time, User A's emotions are also analyzed, so User B can receive the translated message, "We will set off fireworks," in a natural tone.

[1394] Prompt Sentence Examples

[1395] An example of a prompt sentence to be input to the generative AI model is, "Please explain the process that occurs when a user enters their username and password on the login screen."

[1396] As described above, this system includes functions for user authentication, provision of cultural information, matching of communication partners, real-time translation and storage of messages, and even integration of an emotion engine, enabling users with different cultural backgrounds to communicate smoothly and promote deeper mutual understanding.

[1397] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1398] Step 1:

[1399] A user enters a username and password into a login screen to access the system.

[1400] Specific operation: The user enters a username and password in the input fields and presses the login button.

[1401] Input: Username and Password

[1402] Output: User credentials passed to the terminal

[1403] Step 2:

[1404] The terminal collects the entered username and password and sends them to the server.

[1405] Specific operation: The terminal sends the input data to the server as an HTTP POST request.

[1406] Input: User credentials

[1407] Output: HTTP request passed to the server

[1408] Step 3:

[1409] The server checks the received authentication information against a database, and if there is a match, it authenticates the user and allows them access to the system.

[1410] What happens: The server queries DynamoDB to find a matching record, and if authentication is successful, returns a success message in the HTTP response.

[1411] Input: User credentials

[1412] Output: Authentication result (success / failure)

[1413] Step 4:

[1414] Users navigate to the Cultural Exchange section and select a cultural topic of interest.

[1415] Specific operation: The user selects a cultural topic from the UI and presses the select button.

[1416] Input: Cultural topic selection

[1417] Output: Selection information passed to the terminal

[1418] Step 5:

[1419] The terminal collects the user's selection information and sends a request to the server.

[1420] Specific operation: The terminal sends the selected data to the server as an HTTP GET request.

[1421] Input: Cultural topic selection information

[1422] Output: HTTP request passed to the server

[1423] Step 6:

[1424] Upon receiving the request, the server retrieves the culture information from the database and returns it to the terminal.

[1425] Specific operation: The server searches MongoDB, obtains the corresponding culture information, and returns it in the HTTP response.

[1426] Input: Cultural topic selection information

[1427] Output: Culture information sent back to the terminal

[1428] Step 7:

[1429] The terminal displays the culture information received from the server so that the user can check it.

[1430] Specific operation: The device parses the received JSON data and displays it on the screen.

[1431] Input:Cultural information

[1432] Output: Culture information displayed on the screen

[1433] Step 8:

[1434] The user clicks on the "Find a Cultural Exchange Partner" button.

[1435] Specific Action: The user clicks a button in the UI.

[1436] Input: Button click event

[1437] Output: Click event passed to the terminal

[1438] Step 9:

[1439] The terminal sends a partner search request to the server.

[1440] Specific operation: The terminal sends an HTTP GET request including user information to the server.

[1441] Input: User information, click event

[1442] Output: HTTP request passed to the server

[1443] Step 10:

[1444] The server searches a database based on the user's profile information to find the most suitable partner and returns a list of candidate partners to the terminal.

[1445] What happens: The server uses Elasticsearch to search the database and return a list of matching users.

[1446] Input: User information

[1447] Output: A list of potential partners returned to the terminal

[1448] Step 11:

[1449] The terminal displays a list of candidate partners to the user, allowing the user to make a selection.

[1450] Specific operation: The device displays the received list on the screen and provides a UI that allows the user to select.

[1451] Input: List of potential partners

[1452] Output: Partner list displayed on screen

[1453] Step 12:

[1454] The user initiates an interactive session and enters a message.

[1455] Specific operation: The user enters a message in the text box of the UI and presses the send button.

[1456] Input: Message content

[1457] Output: Message content passed to the terminal

[1458] Step 13:

[1459] The terminal transmits the input message to the server.

[1460] Specific operation: The device sends an HTTP POST request containing a message to the server.

[1461] Input: Message content

[1462] Output: HTTP request passed to the server

[1463] Step 14:

[1464] The server translates the message in real time using the Google Cloud Translate API, stores the translated message in DynamoDB, and sends it to the other user's device.

[1465] Specific operation: The server calls the translation API, stores the translation results in DynamoDB, and sends them to the other device.

[1466] Input: Message content

[1467] Output: translated messages, saved chat logs

[1468] Step 15:

[1469] The other party's terminal displays the translated message received from the server to the user.

[1470] Specific operation: The device parses the received JSON data and displays it on the screen.

[1471] Input: Translated message

[1472] Output: Messages displayed on the screen

[1473] Step 16:

[1474] The device requests the server to analyze the emotions of the sent messages in real time.

[1475] Specific operation: The device adds an emotion analysis flag to an HTTP POST request containing a message and sends it to the server.

[1476] Input: Message content, sentiment analysis flag

[1477] Output: HTTP request passed to the server

[1478] Step 17:

[1479] The server uses IBM Watson's NLP sentiment analysis API to analyze messages in real time and set the tone and nuance of the translation based on the results.

[1480] Specific operation: The server calls the sentiment analysis API and passes the results to the Google Cloud Translate API for translation.

[1481] Input: Message content

[1482] Output: Sentiment analysis results, adjusted translation tone

[1483] Step 18:

[1484] The server stores the emotion analysis results in a chat log and manages changes in emotions as a history.

[1485] Specific operation: The server stores the sentiment analysis results in DynamoDB so that they can be retrieved later.

[1486] Input: Sentiment analysis results

[1487] Output: Saved sentiment analysis results

[1488] Through the above steps, communication between users is facilitated and a dialogue history including changes in emotions is managed.

[1489] (Application example 2)

[1490] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1491] Conventional cultural exchange systems have faced challenges in enabling smooth communication between users with different languages ​​and cultural backgrounds. In particular, they lacked the ability to find suitable exchange partners, perform real-time translation between multiple languages, and analyze emotions to maintain a natural tone in the dialogue. As a result, users not only failed to deepen their understanding of each other, but also lost interest and attention, preventing the effectiveness of cultural exchange from being maximized.

[1492] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for authenticating the user by comparing authentication information entered by the user with information stored in a database, means for retrieving and providing information on multiple cultures from the database, means for recommending optimal communication partners based on the user's profile information, means for translating messages sent by the user in real time and saving and sending the translated messages to the other user's device, means for analyzing the emotions of the messages using an emotion engine and adjusting the tone and nuance of the translation based on the results, and means for providing cultural information and related product information and promoting intercultural exchange. This enables users with different cultural backgrounds to communicate smoothly and foster deeper mutual understanding.

[1493] "User authentication" refers to authenticating a user by checking the authentication information entered by the user against the information stored in a database.

[1494] "Cultural information" is detailed information about different cultures that is retrieved from a database and provided to the user.

[1495] "Interaction Partners" refers to other users who share common interests and concerns and are recommended as suitable partners based on the user's profile information.

[1496] "Real-time translation" means instantly translating a message sent by a user into another language, and then saving and sending the translated message to the other user's terminal.

[1497] The "emotion engine" is a system that analyzes the emotions in messages sent by users and adjusts the tone and nuance of the translation based on the results.

[1498] "Cultural information provision" is a means of promoting intercultural exchange by providing information on cultures that interest users and related product information.

[1499] A "chat log" is a record used to store and manage the history of a conversation, including translated messages and the results of sentiment analysis.

[1500] The present invention is a system that includes functions for user authentication, provision of cultural information, matching of exchange partners, real-time translation and storage of messages, and integration of an emotion engine, and promotes smooth communication between users with different cultural backgrounds.

[1501] First, the server has a means for user authentication. Specifically, it checks the authentication information (user name and password) entered by the user against the information in the database, and if there is a match, it authenticates the user. This authentication process ensures secure access to the system.

[1502] The server then has a means for retrieving and providing information about multiple cultures from a database. When a user selects a cultural topic of interest, the terminal transmits the selection information to the server, which then retrieves the requested cultural information and returns it to the terminal. This allows the user to view detailed cultural information.

[1503] The server also has a means for recommending the most suitable exchange partner based on the user's profile information. When the user clicks the "Find a cultural exchange partner" button, the terminal sends a partner search request to the server, and the server searches the database for the most suitable exchange partner based on the user's profile information and returns the result to the terminal.

[1504] Furthermore, the server has a means for translating messages sent by users in real time, and storing and transmitting the translated messages to the terminals of the other users. When a user starts an interactive session and enters a message, the terminal sends the message to the server, which translates it into the target language using a real-time translation engine. The translated message is stored in a chat log and transmitted to the other users.

[1505] The emotion engine analyzes the emotion of the user's message and adjusts the tone and nuance of the translation based on the results. When the user enters and sends a message, the device requests the server to analyze the emotion of the message. The server uses the emotion engine to analyze the emotion in real time and reflects the appropriate tone and nuance in the translation. The emotion engine also saves the analysis results in the chat log and manages changes in emotion as a history.

[1506] A system for implementing the present invention includes a terminal such as a smartphone or a head-mounted display, a server using a RESTful API, and software such as Python or Flask / Django (server side), and React Native (client side).

[1507] As a concrete example, consider a situation in which a Japanese language enthusiast starts a conversation with a social partner by asking about how to celebrate the New Year. When the user enters a prompt such as, "Please explain about New Year's customs. I'd especially like to know more about hatsumode (first shrine visit of the year) and osechi ryori (New Year's food)," the system uses a generative AI model to provide detailed information. Furthermore, when the user sends a message such as, "How do you celebrate the New Year in your country?", the emotion engine analyzes the sentiment of the message and adjusts the translation tone to achieve natural communication.

[1508] As a result, users with different cultural backgrounds can communicate smoothly and foster deeper mutual understanding.

[1509] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1510] Step 1:

[1511] A user accesses a system using a terminal and enters authentication information (username and password). The terminal sends these credentials to a server, which checks them against information in a database. If there is a match, the server authenticates the user and allows access to the system. This ensures secure access.

[1512] Step 2:

[1513] The user selects a cultural topic of interest through the terminal. The terminal sends the selection information to the server, which then retrieves related cultural information from the database. The retrieved cultural information is then sent back to the terminal and displayed to the user, allowing the user to obtain detailed cultural information.

[1514] Step 3:

[1515] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server, which searches the database for the most suitable exchange partner based on the user's profile information. A list of suitable exchange partner candidates is returned to the device and displayed to the user. The user then selects the partner they would like to exchange with from the list of candidates. This completes the match.

[1516] Step 4:

[1517] A user inputs a message to start a conversation with a social partner. The device sends the message to the server, which translates the message into the target language using a real-time translation engine. The translated message is saved in the chat log and simultaneously sent to the other user's device. This enables real-time multilingual communication.

[1518] Step 5:

[1519] An emotion engine is built into messages entered by users. When a device sends a message to the server, it requests that the emotion of the message be analyzed. The server uses the emotion engine to analyze the emotion of the message in real time and adjusts the tone and nuance of the translation based on the results. For example, the message "I'm excited" is analyzed as "excited" and translated with an appropriate tone. The emotion results are saved in the chat log, and changes in emotion are managed as a history.

[1520] Step 6:

[1521] When a user requests specific cultural information or related product information, the device sends the request to the server. The server uses a generative AI model to generate and retrieve detailed information based on the requested information. The generated information is sent back to the device and displayed to the user. For example, detailed cultural information can be provided based on a prompt such as, "Please explain New Year's customs. I would particularly like to know more about hatsumode (first shrine visit of the year) and osechi ryori (New Year's food)." This allows the user to gain deeper knowledge about the culture they are interested in.

[1522] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1523] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1524] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1525] [Fourth embodiment]

[1526] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1527] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1528] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1529] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1530] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1531] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1532] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1533] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1534] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1535] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1536] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1537] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1538] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1539] The embodiment of the present invention first includes a user authentication process, a cultural information providing process, an exchange partner matching process, and a dialogue translation and storage process.

[1540] User Authentication

[1541] When a user accesses a system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the database, and if it matches, it authenticates the user and allows them access to the system.

[1542] Providing cultural information

[1543] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[1544] Matching exchange partners

[1545] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list of candidates.

[1546] Dialogue translation and storage

[1547] A user starts a conversation session and inputs a message. The device collects the input message and sends it to the server. The server translates the received message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[1548] Specific examples

[1549] For example, user A enters "I want to know about Japanese New Year customs," and as a result, he or she checks detailed information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" on his or her device. User A then finds a cultural exchange partner and is connected with user B. When user A sends a message saying, "How do you celebrate New Year in your country?", the message is translated by the server and converted into user B's language. User B replies, "We set off fireworks." The contents of the conversation are saved in a chat log and displayed on both devices.

[1550] As described above, this system includes functions for user authentication, provision of cultural information, matching of communication partners, and real-time translation and storage of messages, enabling users with different cultural backgrounds to communicate smoothly and promote mutual understanding.

[1551] The processing flow will be explained below.

[1552] Step 1:

[1553] The user accesses the login screen and enters their username and password.

[1554] Step 2:

[1555] The terminal collects the authentication information entered by the user and sends it to the server.

[1556] Step 3:

[1557] The server checks the received authentication information against the database, and if there is a match, authenticates the user and returns the authentication result to the terminal.

[1558] Step 4:

[1559] Users navigate to the Cultural Exchange section and select a cultural topic of interest.

[1560] Step 5:

[1561] The terminal transmits information on the cultural topic selected by the user to the server as a request.

[1562] Step 6:

[1563] Based on the request, the server retrieves the relevant cultural information from the database and returns the information to the terminal.

[1564] Step 7:

[1565] The terminal displays the received culture information to the user.

[1566] Step 8:

[1567] Users click the "Find a Cultural Exchange Partner" button.

[1568] Step 9:

[1569] The terminal sends a partner search request to the server.

[1570] Step 10:

[1571] The server searches a database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the terminal.

[1572] Step 11:

[1573] The user selects a partner from the candidate list and starts a dialogue session.

[1574] Step 12:

[1575] The user inputs a message, and the terminal transmits the input message to the server.

[1576] Step 13:

[1577] The server translates the received message in real time using a translation engine.

[1578] Step 14:

[1579] The server saves the translated message in the chat log and sends the translation result to the other party's device.

[1580] Step 15:

[1581] The terminal displays the translated message from the other party to the user.

[1582] Step 16:

[1583] The user checks the other person's message and continues the conversation.

[1584] Example 1

[1585] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1586] In modern society, smooth communication between people of different cultural backgrounds is essential to promoting mutual understanding. However, cultural exchange can be difficult due to language barriers, a lack of cultural information, and a lack of suitable exchange partners. There is a need to develop a system that can solve these issues.

[1587] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1588] In this invention, the server includes means for authenticating the user by comparing authentication information entered by the user with information stored in a database, means for retrieving and providing information on multiple cultures from the database, means for recommending optimal communication partners based on the user's profile information, and means for translating messages sent by the user in real time, saving the translated messages in a chat log, and sending them to the other user's device. This enables users with different cultural backgrounds to communicate smoothly and promote mutual understanding.

[1589] "User authentication information" refers to information such as a username and password that a user enters when accessing a system.

[1590] A "database" is a system for organizing and managing information, and stores user authentication information, cultural information, chat logs, etc.

[1591] "Cultural information" is information about the customs, practices, and events of different regions, countries, and societies.

[1592] "Exchange partners" are other users with different cultural backgrounds who are recommended to the user by the system for cultural exchange.

[1593] "Real-time translation" is the process of instantly translating a message sent by a user into another language.

[1594] A "chat log" is a database that records and stores the content of conversations between users.

[1595] "Server" means the central computer system that manages and executes the functions of the entire System.

[1596] "Authentication" refers to the process and technology used to authenticate a user by matching the user's credentials against a database.

[1597] "Providing means" refers to the process and technology for providing cultural information obtained from the database to the user's terminal.

[1598] "Recommendation tools" are processes and techniques for recommending optimal interaction partners based on a user's profile information.

[1599] "Translation means" refers to the process and technology for translating a message sent by a user in real time and sending it to the other user's terminal.

[1600] The "history database" is a database for managing the history of conversations including translated messages.

[1601] To implement this invention, a user authentication process, a cultural information provision process, an exchange partner matching process, and a dialogue translation and storage process are required.

[1602] User Authentication Process

[1603] When a user accesses the system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the database, and if it matches, it authenticates the user and allows them access to the system.

[1604] For example, when a user enters a "user name" and "password," the terminal sends this to the server, which then checks the database for authentication. If authentication is successful, the terminal displays "Authentication successful." An example of a prompt statement is "Please enter your user name and password."

[1605] Cultural information provision process

[1606] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[1607] As a concrete example, when a user selects the topic "Japanese New Year customs" and the device requests this from the server, the server retrieves information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" from the database and sends it to the device. The device then displays this information to the user. An example of a prompt sentence is "Please select a cultural topic that interests you."

[1608] Exchange Partner Matching Process

[1609] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list of candidates.

[1610] As a concrete example, when a user clicks the "Find a cultural exchange partner" button and the device sends a request to the server, the server searches for candidates such as "User B" and "User C" and sends them to the device. The device displays the candidate list to the user, and the user selects "User B." An example of a prompt sentence is "Click the button to find a cultural exchange partner."

[1611] Dialogue translation and storage process

[1612] A user starts a conversation session and inputs a message. The device collects the input message and sends it to the server. The server translates the message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[1613] For example, user A types "How do you celebrate New Year in your country?" and their device sends this message to the server, which translates the message and sends it in the language of "user B." User B replies, "We set off fireworks," and the reply is saved in the chat log and displayed on both devices.

[1614] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1615] User Authentication Process

[1616] Step 1:

[1617] The user enters their username and password into the login screen.

[1618] Input: Username, Password

[1619] Output: Credentials

[1620] Specific operation: The user enters a "user name" and "password."

[1621] Step 2:

[1622] The terminal collects this authentication information and sends it to the server.

[1623] Input: Credentials

[1624] Output: Request to server

[1625] Specific operation: The terminal sends the username and password to the server.

[1626] Step 3:

[1627] The server checks the received authentication information against a database.

[1628] Input: Credentials

[1629] Output: Matching result

[1630] What happens: The server checks the username and password in its database.

[1631] Step 4:

[1632] The server returns the result of the match to the terminal.

[1633] Input: Matching result

[1634] Output: Authentication result

[1635] Specific operation: The server sends the result of "authentication successful" or "authentication failed" to the terminal.

[1636] Step 5:

[1637] The terminal displays the authentication result to the user.

[1638] Input: Authentication result

[1639] Output: Display information

[1640] Specific operation: The device will display "Authentication successful" or "Authentication failed" on the screen.

[1641] Cultural information provision process

[1642] Step 1:

[1643] The user navigates to the Cultural Exchange section and selects a cultural topic of interest.

[1644] Input: Select a cultural topic

[1645] Output: Selection information

[1646] What happens: The user selects a topic, such as "Japanese New Year's Customs."

[1647] Step 2:

[1648] The terminal collects the user's selection information and sends a request to the server.

[1649] Input: Selection information

[1650] Output: Request to server

[1651] Specific operation: The terminal transmits the selection information to the server.

[1652] Step 3:

[1653] The server retrieves the requested cultural information from a database.

[1654] Input: Request information

[1655] Output:Cultural information

[1656] Specific operation: The server retrieves information such as "first shrine visit of the year" and "New Year's cuisine" from the database.

[1657] Step 4:

[1658] The server returns the acquired culture information to the terminal.

[1659] Input:Cultural information

[1660] Output:Return information

[1661] Specific operation: The server sends culture information to the terminal.

[1662] Step 5:

[1663] The terminal displays the returned information to the user.

[1664] Input:Return information

[1665] Output: Display information

[1666] Specific operation: The device displays the cultural information on the screen.

[1667] Exchange Partner Matching Process

[1668] Step 1:

[1669] The user clicks the "Find a Cultural Exchange Partner" button.

[1670] Input: Button click

[1671] Output: Partner search request

[1672] Specific action: The user clicks the "Find a cultural exchange partner" button.

[1673] Step 2:

[1674] The terminal sends a partner search request to the server.

[1675] Input: Search request

[1676] Output: Request to server

[1677] Specific operation: The device sends a search request to the server.

[1678] Step 3:

[1679] The server searches a database for the most suitable partner based on the user's profile information.

[1680] Input: Profile information

[1681] Output: List of potential partners

[1682] Specific operation: The server searches the database for candidates such as "User B" and "User C."

[1683] Step 4:

[1684] The server returns a list of potential partners to the terminal as a search result.

[1685] Input: List of potential partners

[1686] Output:Return information

[1687] Specific operation: The server sends the candidate list to the terminal.

[1688] Step 5:

[1689] The terminal displays the candidate list to the user.

[1690] Input:Return information

[1691] Output: Display information

[1692] Specific operation: The device displays the candidate list on the screen.

[1693] Step 6:

[1694] The user selects the partner with whom they wish to interact from a list of candidates.

[1695] Input: Partner selection

[1696] Output: Selection information

[1697] Specific operation: The user selects "User B."

[1698] Dialogue translation and storage process

[1699] Step 1:

[1700] The user initiates an interactive session and enters a message.

[1701] Input: Message

[1702] Output: The input message

[1703] What happens: User types, "How do you celebrate New Year's in your country?"

[1704] Step 2:

[1705] The terminal collects the input messages and sends them to the server.

[1706] Input: The message entered

[1707] Output: Request to server

[1708] Specific operation: The device sends a message to the server.

[1709] Step 3:

[1710] The server translates the message into the target language using a real-time translation engine.

[1711] Input: The message entered

[1712] Output: Translated message

[1713] Specific behavior: The server translates "How do you celebrate New Year in your country?" into the other person's language.

[1714] Step 4:

[1715] The server stores the translated messages in the chat log.

[1716] Input: Translated message

[1717] Output: Saved messages

[1718] What happens next: The server stores the translated message in a database.

[1719] Step 5:

[1720] The server sends the translated message to the other user's terminal.

[1721] Input: Translated message

[1722] Output: Send to the other terminal

[1723] Specific operation: The server sends the translated message to the other device.

[1724] Step 6:

[1725] The user can check the translated message on their device and continue the conversation.

[1726] Input: Translated message

[1727] Output: The displayed message

[1728] Specific action: The user sees the reply from the other party on their device: "We're going to set off fireworks."

[1729] (Application example 1)

[1730] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1731] For users with different cultural backgrounds to communicate smoothly while shopping online, real-time language translation and cultural and product information are necessary. Conventional systems do not integrate these elements and are unable to provide the diverse information and interaction opportunities users desire. Therefore, the challenge is to enable users from different cultural backgrounds to share cultural and product information while also enabling smooth communication with partners.

[1732] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1733] In this invention, the server includes means for authenticating the user by comparing authentication information entered by the user with information stored in a database, means for retrieving and providing information about multiple cultures from the database, means for recommending optimal communication partners based on the user's profile information, means for translating messages sent by the user in real time and storing and sending the translated messages to the other user's terminal, and means for users to retrieve information about cultures and products of different countries while shopping online and communicate with sales staff and other shopping users in the local language, thereby enabling users from different cultural backgrounds to share cultural and product information and communicate smoothly with partners.

[1734] "User authentication" refers to the process of verifying a user's identity by matching the authentication information entered by the user with information stored in a database.

[1735] "Cultural information provision" refers to the process of retrieving information about multiple cultures from a database and providing it appropriately to the user.

[1736] "Exchange partner recommendation" refers to the process of selecting and recommending the most suitable exchange partner to the user based on the user's profile information.

[1737] "Real-time translation" refers to the process of translating a message sent by a user into a different language in real time, and then storing and sending the translated message to the other user's device.

[1738] "Cultural information provision during online shopping" refers to the process of retrieving and providing information about cultures and products of different countries when users are shopping online.

[1739] "Communication methods" refers to the functions and processes for communicating in real time with salespeople and other shopping users in the local language.

[1740] As an embodiment of this invention, we will explain a system that includes user authentication, cultural information provision, exchange partner recommendation, real-time translation, cultural information provision during online shopping, and communication means. This system is configured using a program that uses the Flask framework and the Google Translate API.

[1741] User Authentication

[1742] When a user accesses the system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the information stored in the database, and if it matches, the user is authenticated and allowed to access the system.

[1743] Providing cultural information

[1744] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[1745] Exchange partner recommendation

[1746] When a user clicks the "Find a cultural exchange partner" button, the device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list.

[1747] Real-time translation

[1748] When a user starts a conversation session and inputs a message, the device collects the input message and sends it to the server. The server translates the received message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[1749] Providing cultural information while shopping online

[1750] When a user selects a specific item to retrieve information about the culture or products of different countries while shopping online, the terminal sends the request to the server, which retrieves the culture and product information from the database and returns it to the terminal, which uses the information to enrich the user's shopping experience.

[1751] Communication tools

[1752] A real-time translation function is used by users to communicate with other users and salespeople while online shopping. The server translates messages and sends the translated messages to each user's device.

[1753] Specific examples

[1754] For example, if User A is interested in French wine and requests cultural information about it, information about the history and brewing methods of wine will be displayed. If User A then communicates with a French salesperson and wants more detailed information, the message will be translated in real time and sent to the salesperson. The salesperson's reply will also be translated and displayed on User A's screen, enabling smooth communication.

[1755] Example prompt sentence:

[1756] 1. Log in:

[1757] Enter your username and password to log in

[1758] "username: user1, password: password1"

[1759] 2. Obtaining cultural information:

[1760] User requests information about "wine"

[1761] "topic: wine"

[1762] 3. Matching exchange partners:

[1763] Provide your profile information to find a partner

[1764] "profile: User Profile Data"

[1765] 4. Message translation and dialogue:

[1766] Translate and communicate messages

[1767] "message: Tell me about French wine, target_language: fr"

[1768] In this way, a system is realized that allows users to obtain information about different cultures while shopping online, and promote communication with their partners.

[1769] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1770] Step 1:

[1771] When a user accesses the system, they enter a username and password on a login screen.

[1772] Input: Username, Password

[1773] Operation: The device collects the authentication information entered by the user and sends it to the server.

[1774] Data processing: The server receives the authentication information and checks it against the information stored in the database.

[1775] Output: Authentication success / failure message

[1776] Step 2:

[1777] The user navigates to the Cultural Exchange section and selects a cultural topic of interest.

[1778] Input: Cultural topic selection information

[1779] Operation: The terminal collects the user's selection information and sends it to the server.

[1780] Data processing: The server retrieves the relevant cultural information from the database.

[1781] Output: Retrieved cultural information

[1782] Step 3:

[1783] The user clicks the "Find a Cultural Exchange Partner" button.

[1784] Input: Partner Search Request

[1785] Operation: The device sends a request to the server.

[1786] Data processing: The server searches the database for the most suitable partner based on the user's profile information.

[1787] Output: List of potential partners

[1788] Step 4:

[1789] The user selects the partner they want to interact with from a list of candidates.

[1790] Input: Partner selection information

[1791] Operation: The device sends the selection information to the server, which then sends a notification to the selected partner.

[1792] Output: Connection status with partner

[1793] Step 5:

[1794] The user initiates an interactive session and enters a message.

[1795] Input: Dialogue message

[1796] Operation: The terminal sends the inputted interaction message to the server.

[1797] Data processing: The server translates the message using a real-time translation engine.

[1798] Output: Save the translated message and send it to the other user's device

[1799] Step 6:

[1800] Users select specific items to retrieve specific cultural or product information while shopping online.

[1801] Input: Request for cultural or product information

[1802] Operation: The device sends a request to the server.

[1803] Data processing: The server retrieves the relevant information from the database and returns it to the terminal.

[1804] Output: Returned culture and product information

[1805] Step 7:

[1806] While a user is shopping online, the user communicates with sales staff and other users in real time.

[1807] Input: Dialogue message

[1808] Operation: The terminal sends the input message to the server, which translates the message and sends it to the other terminal.

[1809] Data processing: Message translation and saving to chat log

[1810] Output: Sends translated message and updates chat log

[1811] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1812] The embodiment of the present invention includes a user authentication process, a cultural information providing process, an exchange partner matching process, a dialogue translation and storage process, and an emotion engine integration process.

[1813] User Authentication

[1814] When a user accesses a system, they enter their username and password on the login screen. The terminal collects this authentication information and sends it to the server. The server compares the received authentication information with the database, and if it matches, it authenticates the user and allows them access to the system.

[1815] Providing cultural information

[1816] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device collects the user's selection and sends a request to the server. The server retrieves the requested cultural information from a database and returns it to the device. The returned information is displayed on the device for the user to review.

[1817] Matching exchange partners

[1818] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server searches the database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the device. The user can then select the partner they want to interact with from the list of candidates.

[1819] Dialogue translation and storage

[1820] A user starts a conversation session and inputs a message. The device sends the input message to the server. The server translates the received message into the target language using a real-time translation engine, saves the translated message in the chat log, and sends it to the other user's device. The user can then view the translated message on their device and continue the conversation.

[1821] Emotion engine integration

[1822] An emotion engine is built into messages entered by users. When a device sends a message to the server, it requests that the emotion of the message be analyzed. The server uses the emotion engine to analyze the emotion of the message in real time and adjusts the tone and nuance of the translation based on the results. For example, if user A enters "I'm excited," the emotion engine analyzes the emotion as "excited" and translates it with an appropriate tone. The emotion engine also saves the results in the chat log and manages changes in emotion as a history.

[1823] Specific examples

[1824] For example, User A types, "I want to know about Japanese New Year customs," and checks detailed information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" on his device. User A then finds a cultural exchange partner and is connected with User B. When User A sends a message saying, "How do you celebrate the New Year in your country?", the message is translated in real time by the server and converted into User B's language. At this time, User A's emotions are also analyzed, so User B can receive the translated message, "We will set off fireworks," in a natural tone.

[1825] As described above, this system includes functions for user authentication, provision of cultural information, matching of communication partners, real-time translation and storage of messages, and integration of an emotion engine, enabling users with different cultural backgrounds to communicate smoothly and promote deeper mutual understanding.

[1826] The processing flow will be explained below.

[1827] Step 1:

[1828] The user accesses the login screen and enters their username and password.

[1829] Step 2:

[1830] The terminal collects the authentication information entered by the user and sends it to the server.

[1831] Step 3:

[1832] The server checks the received authentication information against the database, and if there is a match, authenticates the user and returns the authentication result to the terminal.

[1833] Step 4:

[1834] Users navigate to the Cultural Exchange section and select a cultural topic of interest.

[1835] Step 5:

[1836] The terminal transmits information on the cultural topic selected by the user to the server as a request.

[1837] Step 6:

[1838] Based on the request, the server retrieves the relevant cultural information from a database and returns the information to the terminal.

[1839] Step 7:

[1840] The terminal displays the received culture information to the user.

[1841] Step 8:

[1842] Users click the "Find a Cultural Exchange Partner" button.

[1843] Step 9:

[1844] The terminal sends a partner search request to the server.

[1845] Step 10:

[1846] The server searches a database for the most suitable partner based on the user's profile information and returns a list of candidate partners to the terminal.

[1847] Step 11:

[1848] The user selects a partner from the candidate list and starts a dialogue session.

[1849] Step 12:

[1850] The user inputs a message, and the terminal requests emotion analysis by the emotion engine when transmitting the input message to the server.

[1851] Step 13:

[1852] The server analyzes the received message in real time using an emotion engine to identify the emotion of the message.

[1853] Step 14:

[1854] The server adjusts the tone and nuance of the translation as needed based on the emotions identified by the emotion engine.

[1855] Step 15:

[1856] The server saves the adjusted translated message in the chat log and sends the translation result to the other party's device.

[1857] Step 16:

[1858] The terminal displays the translated message from the other party to the user.

[1859] Step 17:

[1860] The device also stores the emotional data of messages in a chat log and manages changes in emotions as a history.

[1861] Step 18:

[1862] The user checks the other person's message and continues the conversation.

[1863] For example, if user A types "I want to know about Japanese New Year customs," the information is returned to the device and displayed. User A then finds a social partner and is connected with user B. If user A sends a message saying "How do you celebrate the New Year in your country?", the server analyzes the message and identifies the emotion "excited." The translation result is provided to user B in an appropriate tone, enabling natural conversation that takes into account each other's emotions.

[1864] Example 2

[1865] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1866] While conventional cultural exchange systems provide basic functions such as user authentication, cultural information provision, partner matching, and message translation and storage, they lack the ability to adjust emotional nuances during translation or the ability to store and manage emotional changes in chat logs. This makes it difficult for users with different cultural backgrounds to communicate smoothly and naturally. The objective of this invention is to resolve these shortcomings and promote deeper mutual understanding and smoother communication between users.

[1867] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1868] means for authenticating a user by matching authentication information entered by the user with information stored in a database;

[1869] a means of retrieving and providing information about multiple cultures from a database;

[1870] A means of recommending optimal interaction partners based on the user's profile information;

[1871] A means for translating a message sent by a user in real time, and storing and transmitting the translated message to the terminal of the other user;

[1872] A means to analyze the message to be translated using an emotion engine and adjust the tone and nuance of the translation,

[1873] A means to save the analysis results of the emotion engine in a chat log and manage changes in emotions as a history,

[1874] This allows users with different cultural backgrounds to communicate smoothly and naturally and manage emotional changes, enabling deeper mutual understanding.

[1875] "User authentication" is the process of verifying the identity of a user who accesses a system by comparing the authentication information entered by the user with the information stored in a database.

[1876] "Cultural information" is information about multiple cultures, and is a collection of knowledge and data that is provided by user selection.

[1877] An "exchange partner" is a user with whom you can engage in cultural exchange, optimally selected based on your profile information.

[1878] "Message translation" is the process of translating messages sent by users into other languages ​​in real time.

[1879] A "translated message" is a message after the original message has been translated by a real-time translation engine.

[1880] The "emotion engine" is an engine that analyzes messages entered by users and adjusts the tone and nuance based on emotions.

[1881] A "chat log" is a database for storing and managing the history of conversations between users.

[1882] "Emotion analysis results" are data related to emotions obtained by the emotion engine analyzing the user's message.

[1883] The embodiment of the present invention mainly includes a user authentication process, a cultural information providing process, an exchange partner matching process, a dialogue translation and storage process, and an emotion engine integration process.

[1884] User Authentication Process

[1885] When a user accesses the system, they enter their username and password on a login screen. This authentication information is collected by the terminal and sent to the server. The server compares the received authentication information with its database, and if it matches, it authenticates the user and allows them access to the system. This series of processes uses AWS's DynamoDB and the HTTP protocol.

[1886] Cultural information provision process

[1887] The user navigates to the cultural exchange section and selects a cultural topic of interest. The device sends the user's selection as a request to the server. The server retrieves the cultural information from the database using MongoDB and returns it to the device. The returned information is displayed by the device for the user to review.

[1888] Exchange Partner Matching Process

[1889] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server. The server uses Elasticsearch to search the database based on the user's profile information, creates a list of suitable partners, and returns it to the device. The user can then select the desired partner from the list of candidates.

[1890] Dialogue translation and storage process

[1891] A user starts a conversation session and inputs a message. The device sends the input message to the server. The server translates the message in real time using the Google Cloud Translate API, stores the translated message in DynamoDB, and sends it to the other user's device. The user can then view the translated message and continue the conversation.

[1892] Emotion Engine Integration Process

[1893] An emotion engine is built into messages entered by users. When a device sends a message to the server, it requests that the emotion engine be used to analyze the message's emotion. The server uses IBM Watson's NLP emotion analysis API to analyze the emotion of the message in real time and adjusts the tone and nuance of the translation based on the results. The emotion engine's analysis results are also saved in the chat log, and changes in emotion are managed as a history.

[1894] Specific examples

[1895] For example, User A types, "I want to know about Japanese New Year customs," and checks detailed information such as "first shrine visit of the year," "Osechi cuisine," and "New Year's cards" on his device. User A then finds a cultural exchange partner and is connected with User B. When User A sends a message saying, "How do you celebrate the New Year in your country?", the message is translated in real time by the server and converted into User B's language. At this time, User A's emotions are also analyzed, so User B can receive the translated message, "We will set off fireworks," in a natural tone.

[1896] Prompt Sentence Examples

[1897] An example of a prompt sentence to be input to the generative AI model is, "Please explain the process that occurs when a user enters their username and password on the login screen."

[1898] As described above, this system includes functions for user authentication, provision of cultural information, matching of communication partners, real-time translation and storage of messages, and even integration of an emotion engine, enabling users with different cultural backgrounds to communicate smoothly and promote deeper mutual understanding.

[1899] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1900] Step 1:

[1901] A user enters a username and password into a login screen to access the system.

[1902] Specific operation: The user enters a username and password in the input fields and presses the login button.

[1903] Input: Username and Password

[1904] Output: User credentials passed to the terminal

[1905] Step 2:

[1906] The terminal collects the entered username and password and sends them to the server.

[1907] Specific operation: The terminal sends the input data to the server as an HTTP POST request.

[1908] Input: User credentials

[1909] Output: HTTP request passed to the server

[1910] Step 3:

[1911] The server checks the received authentication information against a database, and if there is a match, it authenticates the user and allows them access to the system.

[1912] What happens: The server queries DynamoDB to find a matching record, and if authentication is successful, returns a success message in the HTTP response.

[1913] Input: User credentials

[1914] Output: Authentication result (success / failure)

[1915] Step 4:

[1916] Users navigate to the Cultural Exchange section and select a cultural topic of interest.

[1917] Specific operation: The user selects a cultural topic from the UI and presses the select button.

[1918] Input: Cultural topic selection

[1919] Output: Selection information passed to the terminal

[1920] Step 5:

[1921] The terminal collects the user's selection information and sends a request to the server.

[1922] Specific operation: The terminal sends the selected data to the server as an HTTP GET request.

[1923] Input: Cultural topic selection information

[1924] Output: HTTP request passed to the server

[1925] Step 6:

[1926] Upon receiving the request, the server retrieves the culture information from the database and returns it to the terminal.

[1927] Specific operation: The server searches MongoDB, obtains the corresponding culture information, and returns it in the HTTP response.

[1928] Input: Cultural topic selection information

[1929] Output: Culture information sent back to the terminal

[1930] Step 7:

[1931] The terminal displays the culture information received from the server so that the user can check it.

[1932] Specific operation: The device parses the received JSON data and displays it on the screen.

[1933] Input:Cultural information

[1934] Output: Culture information displayed on the screen

[1935] Step 8:

[1936] The user clicks on the "Find a Cultural Exchange Partner" button.

[1937] Specific Action: The user clicks a button in the UI.

[1938] Input: Button click event

[1939] Output: Click event passed to the terminal

[1940] Step 9:

[1941] The terminal sends a partner search request to the server.

[1942] Specific operation: The terminal sends an HTTP GET request including user information to the server.

[1943] Input: User information, click event

[1944] Output: HTTP request passed to the server

[1945] Step 10:

[1946] The server searches a database based on the user's profile information to find the most suitable partner and returns a list of candidate partners to the terminal.

[1947] What happens: The server uses Elasticsearch to search the database and return a list of matching users.

[1948] Input: User information

[1949] Output: A list of potential partners returned to the terminal

[1950] Step 11:

[1951] The terminal displays a list of candidate partners to the user, allowing the user to make a selection.

[1952] Specific operation: The device displays the received list on the screen and provides a UI that allows the user to select.

[1953] Input: List of potential partners

[1954] Output: Partner list displayed on screen

[1955] Step 12:

[1956] The user initiates an interactive session and enters a message.

[1957] Specific operation: The user enters a message in the text box of the UI and presses the send button.

[1958] Input: Message content

[1959] Output: Message content passed to the terminal

[1960] Step 13:

[1961] The terminal transmits the input message to the server.

[1962] Specific operation: The device sends an HTTP POST request containing a message to the server.

[1963] Input: Message content

[1964] Output: HTTP request passed to the server

[1965] Step 14:

[1966] The server translates the message in real time using the Google Cloud Translate API, stores the translated message in DynamoDB, and sends it to the other user's device.

[1967] Specific operation: The server calls the translation API, stores the translation results in DynamoDB, and sends them to the other device.

[1968] Input: Message content

[1969] Output: translated messages, saved chat logs

[1970] Step 15:

[1971] The other party's terminal displays the translated message received from the server to the user.

[1972] Specific operation: The device parses the received JSON data and displays it on the screen.

[1973] Input: Translated message

[1974] Output: Messages displayed on the screen

[1975] Step 16:

[1976] The device requests the server to analyze the emotions of the sent messages in real time.

[1977] Specific operation: The device adds an emotion analysis flag to an HTTP POST request containing a message and sends it to the server.

[1978] Input: Message content, sentiment analysis flag

[1979] Output: HTTP request passed to the server

[1980] Step 17:

[1981] The server uses IBM Watson's NLP sentiment analysis API to analyze messages in real time and set the tone and nuance of the translation based on the results.

[1982] Specific operation: The server calls the sentiment analysis API and passes the results to the Google Cloud Translate API for translation.

[1983] Input: Message content

[1984] Output: Sentiment analysis results, adjusted translation tone

[1985] Step 18:

[1986] The server stores the emotion analysis results in a chat log and manages changes in emotions as a history.

[1987] Specific operation: The server stores the sentiment analysis results in DynamoDB so that they can be retrieved later.

[1988] Input: Sentiment analysis results

[1989] Output: Saved sentiment analysis results

[1990] Through the above steps, communication between users is facilitated and a dialogue history including changes in emotions is managed.

[1991] (Application example 2)

[1992] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1993] Conventional cultural exchange systems have faced challenges in enabling smooth communication between users with different languages ​​and cultural backgrounds. In particular, they lacked the ability to find suitable exchange partners, perform real-time translation between multiple languages, and analyze emotions to maintain a natural tone in the dialogue. As a result, users not only failed to deepen their understanding of each other, but also lost interest and attention, preventing the effectiveness of cultural exchange from being maximized.

[1994] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for authenticating the user by comparing authentication information entered by the user with information stored in a database, means for retrieving and providing information on multiple cultures from the database, means for recommending optimal communication partners based on the user's profile information, means for translating messages sent by the user in real time and saving and sending the translated messages to the other user's device, means for analyzing the emotions of the messages using an emotion engine and adjusting the tone and nuance of the translation based on the results, and means for providing cultural information and related product information and promoting intercultural exchange. This enables users with different cultural backgrounds to communicate smoothly and foster deeper mutual understanding.

[1995] "User authentication" refers to authenticating a user by checking the authentication information entered by the user against the information stored in a database.

[1996] "Cultural information" is detailed information about different cultures that is retrieved from a database and provided to the user.

[1997] "Interaction Partners" refers to other users who share common interests and concerns and are recommended as suitable partners based on the user's profile information.

[1998] "Real-time translation" means instantly translating a message sent by a user into another language, and then saving and sending the translated message to the other user's terminal.

[1999] The "emotion engine" is a system that analyzes the emotions in messages sent by users and adjusts the tone and nuance of the translation based on the results.

[2000] "Cultural information provision" is a means of promoting intercultural exchange by providing information on cultures that interest users and related product information.

[2001] A "chat log" is a record used to store and manage the history of a conversation, including translated messages and the results of sentiment analysis.

[2002] The present invention is a system that includes functions for user authentication, provision of cultural information, matching of exchange partners, real-time translation and storage of messages, and integration of an emotion engine, and promotes smooth communication between users with different cultural backgrounds.

[2003] First, the server has a means for user authentication. Specifically, it checks the authentication information (user name and password) entered by the user against the information in the database, and if there is a match, it authenticates the user. This authentication process ensures secure access to the system.

[2004] The server then has a means for retrieving and providing information about multiple cultures from a database. When a user selects a cultural topic of interest, the terminal transmits the selection information to the server, which then retrieves the requested cultural information and returns it to the terminal. This allows the user to view detailed cultural information.

[2005] The server also has a means for recommending the most suitable exchange partner based on the user's profile information. When the user clicks the "Find a cultural exchange partner" button, the terminal sends a partner search request to the server, and the server searches the database for the most suitable exchange partner based on the user's profile information and returns the result to the terminal.

[2006] Furthermore, the server has a means for translating messages sent by users in real time, and storing and transmitting the translated messages to the terminals of the other users. When a user starts an interactive session and enters a message, the terminal sends the message to the server, which translates it into the target language using a real-time translation engine. The translated message is stored in a chat log and transmitted to the other users.

[2007] The emotion engine analyzes the emotion of the user's message and adjusts the tone and nuance of the translation based on the results. When the user enters and sends a message, the device requests the server to analyze the emotion of the message. The server uses the emotion engine to analyze the emotion in real time and reflects the appropriate tone and nuance in the translation. The emotion engine also saves the analysis results in the chat log and manages changes in emotion as a history.

[2008] A system for implementing the present invention includes a terminal such as a smartphone or a head-mounted display, a server using a RESTful API, and software such as Python or Flask / Django (server side), and React Native (client side).

[2009] As a concrete example, consider a situation in which a Japanese language enthusiast starts a conversation with a social partner by asking about how to celebrate the New Year. When the user enters a prompt such as, "Please explain about New Year's customs. I'd especially like to know more about hatsumode (first shrine visit of the year) and osechi ryori (New Year's food)," the system uses a generative AI model to provide detailed information. Furthermore, when the user sends a message such as, "How do you celebrate the New Year in your country?", the emotion engine analyzes the sentiment of the message and adjusts the translation tone to achieve natural communication.

[2010] As a result, users with different cultural backgrounds can communicate smoothly and foster deeper mutual understanding.

[2011] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2012] Step 1:

[2013] A user accesses a system using a terminal and enters authentication information (username and password). The terminal sends these credentials to a server, which checks them against information in a database. If there is a match, the server authenticates the user and allows access to the system. This ensures secure access.

[2014] Step 2:

[2015] The user selects a cultural topic of interest through the terminal. The terminal sends the selection information to the server, which then retrieves related cultural information from the database. The retrieved cultural information is then sent back to the terminal and displayed to the user, allowing the user to obtain detailed cultural information.

[2016] Step 3:

[2017] The user clicks the "Find a cultural exchange partner" button. The device sends a partner search request to the server, which searches the database for the most suitable exchange partner based on the user's profile information. A list of suitable exchange partner candidates is returned to the device and displayed to the user. The user then selects the partner they would like to exchange with from the list of candidates. This completes the match.

[2018] Step 4:

[2019] A user inputs a message to start a conversation with a social partner. The device sends the message to the server, which translates the message into the target language using a real-time translation engine. The translated message is saved in the chat log and simultaneously sent to the other user's device. This enables real-time multilingual communication.

[2020] Step 5:

[2021] An emotion engine is built into messages entered by users. When a device sends a message to the server, it requests that the emotion of the message be analyzed. The server uses the emotion engine to analyze the emotion of the message in real time and adjusts the tone and nuance of the translation based on the results. For example, the message "I'm excited" is analyzed as "excited" and translated with an appropriate tone. The emotion results are saved in the chat log, and changes in emotion are managed as a history.

[2022] Step 6:

[2023] When a user requests specific cultural information or related product information, the device sends the request to the server. The server uses a generative AI model to generate and retrieve detailed information based on the requested information. The generated information is sent back to the device and displayed to the user. For example, detailed cultural information can be provided based on a prompt such as, "Please explain New Year's customs. I would particularly like to know more about hatsumode (first shrine visit of the year) and osechi ryori (New Year's food)." This allows the user to gain deeper knowledge about the culture they are interested in.

[2024] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2025] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2026] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2027] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2028] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2029] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2030] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2031] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2032] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2033] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2034] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2035] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2036] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2037] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2038] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2039] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2040] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2041] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2042] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2043] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2044] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2045] The following is further disclosed regarding the above embodiment.

[2046] (Claim 1)

[2047] means for authenticating a user by matching authentication information entered by the user with information stored in a database;

[2048] a means of retrieving and providing information about multiple cultures from a database;

[2049] A means of recommending optimal interaction partners based on the user's profile information;

[2050] A means for translating a message sent by a user in real time, and storing and transmitting the translated message to the terminal of the other user;

[2051] A system including:

[2052] (Claim 2)

[2053] means for executing an algorithm for matching cultural exchange partners based on user input information;

[2054] 10. The system of claim 1, comprising:

[2055] (Claim 3)

[2056] A means to store translated messages in a chat log and manage conversation history;

[2057] 10. The system of claim 1, comprising:

[2058] "Example 1"

[2059] (Claim 1)

[2060] means for authenticating a user by matching authentication information entered by the user with information stored in a database;

[2061] a means of retrieving and providing information about multiple cultures from a database;

[2062] A means of recommending optimal interaction partners based on the user's profile information;

[2063] A means for translating messages sent by a user in real time, saving the translated messages in a chat log, and sending them to the other user's device;

[2064] A system including:

[2065] (Claim 2)

[2066] means for executing an algorithm for matching cultural exchange partners based on user input information;

[2067] 10. The system of claim 1, comprising:

[2068] (Claim 3)

[2069] a means for storing translated messages in a history database and managing the history of the dialogue;

[2070] 10. The system of claim 1, comprising:

[2071] "Application Example 1"

[2072] (Claim 1)

[2073] means for authenticating a user by matching authentication information entered by the user with information stored in a database;

[2074] a means of retrieving and providing information about multiple cultures from a database;

[2075] A means of recommending optimal interaction partners based on the user's profile information;

[2076] A means for translating a message sent by a user in real time, and storing and transmitting the translated message to the terminal of the other user;

[2077] A means for users to access information about cultures and products in different countries while shopping online, and to communicate with salespeople and other shopping users in their local languages;

[2078] A system including:

[2079] (Claim 2)

[2080] means for executing an algorithm for matching cultural exchange partners based on user input information;

[2081] means for executing algorithms to provide cultural and product information desired by a user while shopping;

[2082] 10. The system of claim 1, comprising:

[2083] (Claim 3)

[2084] A means to store translated messages in a chat log and manage conversation history;

[2085] A means for storing a user's history of cultural and product information inquiries to facilitate future inquiries;

[2086] 10. The system of claim 1, comprising:

[2087] "Example 2: Combining Emotion Engines"

[2088] (Claim 1)

[2089] means for authenticating a user by matching authentication information entered by the user with information stored in a database;

[2090] a means of retrieving and providing information about multiple cultures from a database;

[2091] A means of recommending optimal interaction partners based on the user's profile information;

[2092] A means for translating a message sent by a user in real time, and storing and transmitting the translated message to the terminal of the other user;

[2093] A means to analyze the message to be translated using an emotion engine and adjust the tone and nuance of the translation,

[2094] A means to save the analysis results of the emotion engine in a chat log and manage changes in emotions as a history,

[2095] A system including:

[2096] (Claim 2)

[2097] means for executing an algorithm for matching cultural exchange partners based on user input information;

[2098] 10. The system of claim 1, comprising:

[2099] (Claim 3)

[2100] A means to store translated messages in a chat log and manage conversation history;

[2101] 10. The system of claim 1, comprising:

[2102] "Application example 2 when combining emotion engines"

[2103] (Claim 1)

[2104] means for authenticating a user by matching authentication information entered by the user with information stored in a database;

[2105] a means of retrieving and providing information about multiple cultures from a database;

[2106] A means of recommending optimal interaction partners based on the user's profile information;

[2107] A means for translating a message sent by a user in real time, and storing and transmitting the translated message to the terminal of the other user;

[2108] A means to analyze the sentiment of a message using an emotion engine and adjust the tone and nuance of the translation based on the results;

[2109] A means to provide cultural information and related product information and promote intercultural exchange;

[2110] A system including:

[2111] (Claim 2)

[2112] means for executing an algorithm for matching cultural exchange partners based on user input information;

[2113] 10. The system of claim 1, comprising:

[2114] (Claim 3)

[2115] A means to store translated messages in a chat log and manage conversation history;

[2116] a means for storing and managing emotion changes in the dialogue history using an emotion engine;

[2117] 10. The system of claim 1, comprising: [Explanation of symbols]

[2118] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for authenticating a user by matching authentication information entered by the user with information stored in a database; a means of retrieving and providing information about multiple cultures from a database; A means of recommending optimal interaction partners based on the user's profile information; A means for translating a message sent by a user in real time, and storing and transmitting the translated message to the terminal of the other user; A system including:

2. means for executing an algorithm for matching cultural exchange partners based on user input information; The system of claim 1 , comprising:

3. A means to store translated messages in a chat log and manage conversation history; The system of claim 1 , comprising:

Images