system

Abstract

The system according to the embodiment aims to translate conversations with people who speak different languages ​​in real time, thereby realizing smooth communication. [Solution] A system according to an embodiment includes a receiving unit, an analyzing unit, a translating unit, and a playing unit. The receiving unit receives speech from the other party as audio. The analyzing unit analyzes the audio data received by the receiving unit. The translating unit translates the text data analyzed by the analyzing unit. The playing unit converts the text data translated by the translating unit into audio data and plays it back.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology has made it difficult to translate in real time when speaking with people who speak different languages, making communication difficult.

[0005] The system according to the embodiment aims to translate conversations with people who speak different languages ​​in real time, thereby realizing smooth communication. [Means for solving the problem]

[0006] The system according to the embodiment includes a receiving unit, an analyzing unit, a translating unit, and a playing unit. The receiving unit receives speech from the other party as audio. The analyzing unit analyzes the audio data received by the receiving unit. The translating unit translates the text data analyzed by the analyzing unit. The playing unit converts the text data translated by the translating unit into audio data and plays it back. [Effects of the Invention]

[0007] The system according to the embodiment can translate conversations with people who speak different languages ​​in real time, enabling smooth communication. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A wireless earphone system according to an embodiment of the present invention automatically translates and plays back speech when speaking with a person who speaks a different language. This wireless earphone system receives speech from the other person as audio, and a generation AI analyzes, translates, and converts the speech into the user's language for playback. For example, the earphones receive the speech from the other person. The earphones are equipped with a high-performance microphone that accurately captures the speech while filtering out ambient noise. For example, the earphones can clearly pick up the other person's voice even in noisy places like cafes and train stations. The generation AI then analyzes the received speech data. The generation AI converts the speech data into text data using speech recognition technology. For example, if the other person says "hello," the speech data is converted into the text data "hello." The generation AI then translates the text data. The generation AI then uses a multilingual translation model to convert the text data into the user's language. For example, the Japanese text data "hello" is translated into the English text data "Hello." Finally, the translated text data is converted into audio data, which is played by the earphones. The generation AI then converts the text data into audio data using speech synthesis technology. For example, text data such as "Hello" is converted into voice data, and the voice "Hello" is played from the earphones. This mechanism allows users to communicate smoothly with people who speak different languages. For example, this is extremely convenient when conversing with local people while traveling, or when discussing with participants who speak different languages ​​at an international conference. In this way, the wireless earphone system can enable smooth communication between people who speak different languages.

[0029] A wireless earphone system according to an embodiment includes a receiving unit, an analyzing unit, a translating unit, and a playing unit. The receiving unit receives speech from the other party as audio. The receiving unit, for example, has a built-in high-performance microphone and accurately captures the speech from the other party while filtering out ambient noise. For example, the receiving unit can clearly pick up the other party's voice even in noisy places such as cafes and train stations. The analyzing unit analyzes the audio data received by the receiving unit. The analyzing unit converts the audio data into text data using, for example, voice recognition technology. For example, if the other party says "hello," the analyzing unit converts the audio data into text data saying "hello." The translating unit translates the text data analyzed by the analyzing unit. The translating unit converts the text data into a user's language using, for example, a multilingual translation model. For example, the translating unit translates Japanese text data saying "hello" into English text data saying "Hello." The playing unit converts the text data translated by the translating unit into audio data and plays it back. For example, the playing unit converts text data into audio data using speech synthesis technology. For example, the playback unit converts text data such as "Hello" into audio data and plays back the audio "Hello" from the earphones. This allows the wireless earphone system according to the embodiment to enable smooth communication with people who speak different languages.

[0030] The wireless earphone system includes a noise canceling unit with a noise canceling function. The noise canceling unit removes ambient noise and provides clear audio. The noise canceling unit, for example, uses active noise canceling technology to analyze ambient noise in real time and generate anti-phase sound waves to cancel the noise. The noise canceling unit can also use passive noise canceling technology to block noise through the physical structure of the earphone. For example, the noise canceling unit can block external sounds and provide clear audio by devising the shape and materials of the earphone. In this way, the noise canceling unit can remove ambient noise and provide clear audio.

[0031] The wireless earphone system includes a setting unit for changing a user's language setting. The setting unit allows the user to easily change the language setting. The setting unit allows the user to select a desired language, for example, through a button operation on the earphones or a smartphone app. For example, the setting unit allows the user to select a language such as English, Japanese, or French using the smartphone app. The setting unit can also recognize a user's voice command and change the language setting. For example, if the user says, "Change to English," the setting unit automatically changes the language setting to English. In this way, the setting unit allows the user to easily change the language setting.

[0032] The wireless earphone system includes an adjustment unit that fine-tunes the translation result. The adjustment unit can fine-tune the translation result to suit the user's preferences. The adjustment unit improves the accuracy of the translation, for example, by having the user provide feedback on the translation result. For example, if the user provides feedback such as "this translation is inaccurate," the adjustment unit updates the translation model and improves the translation result from the next time onwards. The adjustment unit can also allow the user to manually correct the translation result. For example, if the user edits the translation result and inputs a correct translation, the adjustment unit learns the correction and reflects it in the translation result from the next time onwards. This allows the adjustment unit to fine-tune the translation result to suit the user's preferences.

[0033] The receiving unit has a built-in microphone and can accurately capture what the other person is saying while filtering out ambient noise. The receiving unit has, for example, a built-in high-performance microphone and can accurately capture what the other person is saying while filtering out ambient noise. For example, the receiving unit can clearly pick up the other person's voice even in noisy places such as cafes and train stations. The receiving unit can also use noise canceling technology to analyze ambient noise in real time and generate anti-phase sound waves to cancel out the noise. For example, the receiving unit can use active noise canceling technology to analyze ambient noise in real time and generate anti-phase sound waves to cancel out the noise. This allows the receiving unit to clearly pick up the other person's voice even in noisy places.

[0034] The analysis unit can convert voice data into text data using voice recognition technology. The analysis unit, for example, converts voice data into text data using voice recognition technology. For example, if the other person says "hello," the analysis unit converts the voice data into text data saying "hello." The analysis unit can achieve highly accurate voice recognition using deep learning-based voice recognition technology. For example, the analysis unit uses a deep neural network to analyze voice data and convert it into text data. The analysis unit can also convert voice data into text data using HMM (hidden Markov model)-based voice recognition technology. For example, the analysis unit uses an HMM to model temporal changes in the voice data and convert it into text data. This allows the analysis unit to accurately convert voice data into text data.

[0035] The translation unit can convert text data into the user's language using a multilingual translation model. The translation unit, for example, converts text data into the user's language using a multilingual translation model. For example, the translation unit translates Japanese text data such as "Hello" into English text data such as "Hello." The translation unit can achieve highly accurate translation using a neural machine translation model. For example, the translation unit analyzes text data using a neural network and converts it into the user's language. The translation unit can also convert text data into the user's language using a statistical machine translation model. For example, the translation unit translates text data using a statistical method. This enables the translation unit to perform multilingual translation.

[0036] The playback unit can convert text data into voice data using voice synthesis technology and play it back through the earphones. For example, the playback unit converts text data into voice data using voice synthesis technology. For example, the playback unit converts text data such as "Hello" into voice data and plays back the voice "Hello" through the earphones. The playback unit can achieve high-quality voice synthesis using text-to-speech synthesis technology. For example, the playback unit generates natural voice using text-to-speech synthesis technology. The playback unit can also generate voice data using concatenative voice synthesis technology. For example, the playback unit generates voice data using concatenative voice synthesis technology and plays it back through the earphones. In this way, the playback unit can play back the translated voice data to the user.

[0037] The receiving unit can be set to preferentially receive a specific frequency band. For example, the receiving unit may preferentially receive a frequency band of human voices to suppress ambient noise. For example, the receiving unit may preferentially receive a frequency band from 300 Hz to 3 kHz to suppress ambient noise. The receiving unit may also preferentially receive high-frequency sounds to provide clear audio. For example, the receiving unit may preferentially receive high-frequency sounds above 3 kHz to provide clear audio. The receiving unit may also preferentially receive low-frequency sounds to provide deep audio. For example, the receiving unit may preferentially receive low-frequency sounds below 300 Hz to provide deep audio. In this way, the receiving unit can provide clear audio by preferentially receiving a specific frequency band. Some or all of the above-described processing in the receiving unit may be performed using, or without, AI. For example, the receiving unit may input audio data to a generation AI and cause the generation AI to preferentially receive a specific frequency band.

[0038] The receiving unit can have the function of simultaneously receiving audio from multiple sound sources and analyzing it individually. For example, the receiving unit may simultaneously receive the voices of multiple speakers during a meeting and analyze them individually. For example, the receiving unit may use multiple microphones to simultaneously receive the voices of multiple speakers and analyze them individually. The receiving unit may also simultaneously receive multiple conversations at a cafe and analyze only the necessary conversations. For example, the receiving unit may use multiple microphones to simultaneously receive multiple conversations at a cafe and analyze only the necessary conversations. The receiving unit may also simultaneously receive a station announcement and a friend's conversation and analyze them individually. For example, the receiving unit may use multiple microphones to simultaneously receive a station announcement and a friend's conversation and analyze them individually. This allows the receiving unit to efficiently obtain necessary information by simultaneously receiving audio from multiple sound sources and analyzing them individually. Some or all of the above-described processing in the receiving unit may be performed using, for example, AI, or may be performed without AI. For example, the receiving unit may input multiple pieces of audio data into a generation AI and have the generation AI perform individual analyses.

[0039] The receiving unit can set optimal voice reception settings based on the user's location information. For example, if the user is in a cafe, the receiving unit receives conversations while suppressing ambient noise. For example, if the receiving unit analyzes the user's location information and determines that the user is in a cafe, the receiving unit receives conversations while suppressing ambient noise. The receiving unit can also prioritize receiving announcements if the user is at a train station. For example, if the receiving unit analyzes the user's location information and determines that the user is at a train station, the receiving unit prioritizes receiving announcements. The receiving unit can also set normal reception settings if the user is at home. For example, if the receiving unit analyzes the user's location information and determines that the user is at home, the receiving unit sets normal reception settings. This allows the receiving unit to provide optimal voice reception settings based on the user's location information. Some or all of the above-described processing in the receiving unit may be performed using AI, for example, or may be performed without using AI. For example, the receiving unit can input the user's location information to a generation AI and cause the generation AI to execute optimal voice reception settings.

[0040] The receiving unit can refer to the user's past conversation history and preferentially receive the voice of a specific speaker. The receiving unit, for example, preferentially receives the voice of a person with whom the user has frequently conversed in the past. For example, the receiving unit analyzes the user's past conversation history and preferentially receives the voice of a person with whom the user has frequently conversed. The receiving unit can also preferentially receive the voice of a person with whom the user spoke in an important meeting. For example, the receiving unit analyzes the user's past conversation history and preferentially receives the voice of a person with whom the user spoke in an important meeting. The receiving unit can also preferentially receive the conversations the user has with family members. For example, the receiving unit analyzes the user's past conversation history and preferentially receives the conversations with family members. In this way, the receiving unit can preferentially receive the voice of an important speaker based on the user's past conversation history. Some or all of the above-described processing in the receiving unit may be performed using, for example, AI, or may be performed without using AI. For example, the receiving unit can input the user's past conversation history to the generation AI and cause the generation AI to preferentially receive the voice of a specific speaker.

[0041] The analysis unit may have a function to automatically identify and remove background sounds from audio data. The analysis unit may, for example, automatically identify and remove noise during a meeting. For example, the analysis unit may analyze audio data and automatically identify and remove noise during a meeting. The analysis unit may also automatically identify and remove background sounds in a cafe. For example, the analysis unit may analyze audio data and automatically identify and remove background sounds in a cafe. The analysis unit may also automatically identify and remove noise other than station announcements. For example, the analysis unit may analyze audio data and automatically identify and remove noise other than station announcements. In this way, the analysis unit can provide clear audio by automatically removing background sounds from audio data. Some or all of the above-described processing in the analysis unit may be performed using, or without, AI. For example, the analysis unit may input audio data to a generation AI and cause the generation AI to identify and remove background sounds.

[0042] The analysis unit can identify speakers of the audio data and perform different analyses for each speaker. The analysis unit, for example, can identify multiple speakers in a meeting and analyze them individually. For example, the analysis unit can analyze audio data to identify multiple speakers in a meeting and analyze them individually. The analysis unit can also identify multiple conversations at a cafe and analyze only the necessary conversations. For example, the analysis unit can analyze audio data to identify multiple conversations at a cafe and analyze only the necessary conversations. The analysis unit can also identify station announcements and conversations with friends and analyze them individually. For example, the analysis unit can analyze audio data to identify station announcements and conversations with friends and analyze them individually. This allows the analysis unit to identify speakers of the audio data and provide optimal analysis for each speaker. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without AI. For example, the analysis unit can input audio data to a generation AI and have the generation AI perform speaker identification and individual analysis.

[0043] The analysis unit can prioritize analyzing specific keywords by referring to the content of the user's past conversations. The analysis unit, for example, prioritizes analyzing keywords that the user frequently used in the past. For example, the analysis unit analyzes the content of the user's past conversations and prioritizes analyzing frequently used keywords. The analysis unit can also prioritize analyzing keywords that the user used in important meetings. For example, the analysis unit analyzes the content of the user's past conversations and prioritizes analyzing keywords that were used in important meetings. The analysis unit can also prioritize analyzing keywords that the user used in conversations with family members. For example, the analysis unit analyzes the content of the user's past conversations and prioritizes analyzing keywords that were used in conversations with family members. This allows the analysis unit to prioritize analyzing important keywords based on the content of the user's past conversations. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the content of the user's past conversations to the generation AI and cause the generation AI to perform prioritized analysis of specific keywords.

[0044] The analysis unit can analyze the user's current environmental sounds and perform optimal voice recognition settings. For example, when the user is in a cafe, the analysis unit analyzes the environmental sounds and performs optimal voice recognition settings. For example, when the analysis unit analyzes the user's environmental sounds and determines that the user is in a cafe, the analysis unit analyzes the environmental sounds and performs optimal voice recognition settings. The analysis unit can also analyze the environmental sounds and perform optimal voice recognition settings when the user is in a train station. For example, when the analysis unit analyzes the user's environmental sounds and determines that the user is in a train station, the analysis unit analyzes the environmental sounds and performs optimal voice recognition settings. The analysis unit can also analyze the environmental sounds and perform optimal voice recognition settings when the user is at home. For example, when the analysis unit analyzes the user's environmental sounds and determines that the user is at home, the analysis unit analyzes the environmental sounds and performs optimal voice recognition settings. This allows the analysis unit to provide optimal voice recognition settings based on the user's current environmental sounds. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the user's environmental sounds into the generation AI and have the generation AI execute optimal voice recognition settings.

[0045] The translation unit may have the function of automatically recognizing specific technical terms and slang and appropriately translating them. The translation unit, for example, automatically recognizes medical terms and appropriately translates them. For example, the translation unit may analyze voice data, automatically recognize medical terms, and appropriately translate them. The translation unit may also automatically recognize business terms and appropriately translate them. For example, the translation unit may analyze voice data, automatically recognize business terms, and appropriately translate them. The translation unit may also automatically recognize youth slang and appropriately translate them. For example, the translation unit may analyze voice data, automatically recognize youth slang, and appropriately translate them. This allows the translation unit to appropriately translate specific technical terms and slang, thereby providing accurate translations. Some or all of the above-described processing in the translation unit may be performed using, or without, AI. For example, the translation unit may input voice data into a generation AI and have the generation AI recognize and translate technical terms and slang.

[0046] The translation unit can present multiple translation candidates based on the context, allowing the user to select one. The translation unit, for example, presents multiple translation candidates based on the context of a conversation, allowing the user to select one. For example, the translation unit analyzes audio data and presents multiple translation candidates based on the context of the conversation, allowing the user to select one. The translation unit can also present multiple translation candidates based on the context of an email, allowing the user to select one. For example, the translation unit analyzes audio data and presents multiple translation candidates based on the context of the email, allowing the user to select one. The translation unit can also present multiple translation candidates based on the context of a presentation, allowing the user to select one. For example, the translation unit analyzes audio data and presents multiple translation candidates based on the context of the presentation, allowing the user to select one. In this way, the translation unit provides multiple translation candidates based on the context, allowing the user to select the optimal translation. Some or all of the above-described processing in the translation unit may be performed using, for example, AI, or may be performed without AI. For example, the translation unit can input audio data to a generation AI and cause the generation AI to present multiple translation candidates based on the context.

[0047] The translation unit can provide an optimal translation by referring to the user's past translation history. The translation unit can provide an optimal translation based on, for example, translations used by the user in the past. For example, the translation unit can analyze the user's past translation history and provide an optimal translation based on the translations used in the past. The translation unit can also prioritize translation of phrases that the user has used frequently in the past. For example, the translation unit can analyze the user's past translation history and prioritize translation of frequently used phrases. The translation unit can also analyze the user's past translation history and provide the most appropriate translation. For example, the translation unit can analyze the user's past translation history and provide the most appropriate translation. This allows the translation unit to provide an optimal translation based on the user's past translation history. Some or all of the above-described processing in the translation unit can be performed using, for example, AI, or can be performed without using AI. For example, the translation unit can input the user's past translation history into a generation AI and cause the generation AI to perform an optimal translation.

[0048] The translation unit can analyze the content of a user's current conversation in real time and provide an appropriate translation. For example, the translation unit can analyze in real time while a user is talking and provide an appropriate translation. For example, the translation unit can analyze voice data in real time and provide an appropriate translation during the conversation. The translation unit can also analyze in real time while a user is giving a presentation and provide an appropriate translation. For example, the translation unit can analyze voice data in real time and provide an appropriate translation during the presentation. The translation unit can also analyze in real time while a user is on the phone and provide an appropriate translation. For example, the translation unit can analyze voice data in real time and provide an appropriate translation during the phone call. This allows the translation unit to analyze the content of the conversation in real time and provide an appropriate translation. Some or all of the above-mentioned processing in the translation unit may be performed using AI, for example, or may be performed without using AI. For example, the translation unit can input voice data to a generation AI and have the generation AI perform real-time analysis and translation.

[0049] The playback unit may have a function to automatically adjust a specific voice tone or pitch. For example, if a user prefers a high-pitched voice, the playback unit adjusts the voice tone higher. For example, if the playback unit analyzes the user's voice data and determines that the user prefers a high-pitched voice, it adjusts the voice tone higher. Furthermore, if the user prefers a low-pitched voice, the playback unit can adjust the voice tone lower. For example, if the playback unit analyzes the user's voice data and determines that the user prefers a low-pitched voice, it adjusts the voice tone lower. Furthermore, if the user prefers a specific pitch, the playback unit can adjust the pitch to that pitch. For example, if the playback unit analyzes the user's voice data and determines that the user prefers a specific pitch, it adjusts the pitch to that pitch. In this way, the playback unit can provide the user with an optimal voice by adjusting the specific voice tone or pitch. Some or all of the above-described processing in the playback unit may be performed using, for example, AI, or may be performed without AI. For example, the playback unit may input the user's voice data to a generation AI and have the generation AI adjust the voice tone or pitch.

[0050] The playback unit can simultaneously play multiple pieces of audio data, allowing the user to select from them. For example, the playback unit can simultaneously play the audio of multiple speakers during a conference, allowing the user to select from them. For example, the playback unit can simultaneously play multiple pieces of audio data, allowing the user to select from them. The playback unit can also simultaneously play multiple conversations at a cafe, allowing the user to select from them. For example, the playback unit can simultaneously play multiple pieces of audio data, allowing the user to select from them. The playback unit can also simultaneously play a station announcement and a conversation between friends, allowing the user to select from them. For example, the playback unit can simultaneously play multiple pieces of audio data, allowing the user to select from them. In this way, the playback unit can simultaneously play multiple pieces of audio data, allowing the user to select from the audio they need. Some or all of the above-described processing in the playback unit may be performed using, for example, AI, or may be performed without using AI. For example, the playback unit can input multiple pieces of audio data to a generation AI, causing the generation AI to perform simultaneous playback and selection.

[0051] The playback unit can set optimal audio playback settings based on the user's location information. For example, if the user is in a cafe, the playback unit plays audio while suppressing ambient noise. For example, if the playback unit analyzes the user's location information and determines that the user is in a cafe, the playback unit plays audio while suppressing ambient noise. The playback unit can also prioritize playing announcements when the user is at a train station. For example, if the playback unit analyzes the user's location information and determines that the user is at a train station, the playback unit prioritizes playing announcements. The playback unit can also set normal playback settings when the user is at home. For example, if the playback unit analyzes the user's location information and determines that the user is at home, the playback unit sets normal playback settings. This allows the playback unit to provide optimal audio playback settings based on the user's location information. Some or all of the above-described processing in the playback unit may be performed using, for example, AI, or may be performed without using AI. For example, the playback unit can input the user's location information to a generation AI and cause the generation AI to execute optimal audio playback settings.

[0052] The playback unit can prioritize playback of specific audio by referring to the user's past playback history. The playback unit, for example, prioritizes playback of audio that the user has frequently played in the past. For example, the playback unit can analyze the user's past playback history and prioritize playback of audio that was frequently played. The playback unit can also prioritize playback of audio that the user played in important meetings. For example, the playback unit can analyze the user's past playback history and prioritize playback of audio that was played in important meetings. The playback unit can also prioritize playback of conversations the user had with family members. For example, the playback unit can analyze the user's past playback history and prioritize playback of conversations with family members. This allows the playback unit to prioritize playback of important audio based on the user's past playback history. Some or all of the above-described processing in the playback unit may be performed using, for example, AI, or may be performed without using AI. For example, the playback unit can input the user's past playback history to a generation AI and cause the generation AI to prioritize playback of specific audio.

[0053] The noise canceling unit can have a function of preferentially removing noise in a specific frequency band. For example, the noise canceling unit preferentially removes noise in a low frequency band to provide a deeper sound. For example, the noise canceling unit preferentially removes noise in a low frequency band of 300 Hz or less to provide a deeper sound. The noise canceling unit can also preferentially remove noise in a high frequency band to provide a clearer sound. For example, the noise canceling unit preferentially removes noise in a high frequency band of 3 kHz or more to provide a clearer sound. The noise canceling unit can also preferentially remove noise in a mid-frequency band to provide a balanced sound. For example, the noise canceling unit preferentially removes noise in a mid-frequency band of 300 Hz to 3 kHz to provide a balanced sound. In this way, the noise canceling unit can provide a clearer sound by removing noise in a specific frequency band. Some or all of the above-described processing in the noise canceling unit may be performed using AI, or may be performed without using AI. For example, the noise canceling unit may input audio data to a generation AI and cause the generation AI to remove noise in a specific frequency band.

[0054] The noise canceling unit can perform optimal noise canceling settings based on the user's location information. For example, when the user is in a cafe, the noise canceling unit suppresses ambient noise and performs noise canceling. For example, the noise canceling unit analyzes the user's location information and, if it determines that the user is in a cafe, suppresses ambient noise and performs noise canceling. The noise canceling unit can also prioritize noise canceling of announcements when the user is at a train station. For example, the noise canceling unit analyzes the user's location information and, if it determines that the user is at a train station, prioritizes noise canceling of announcements. The noise canceling unit can also perform normal noise canceling settings when the user is at home. For example, the noise canceling unit analyzes the user's location information and, if it determines that the user is at home, performs normal noise canceling settings. This allows the noise canceling unit to provide optimal noise canceling settings based on the user's location information. Some or all of the above-described processing in the noise canceling unit may be performed using, for example, AI, or may be performed without using AI. For example, the noise canceling unit can input the user's location information into the generation AI and have the generation AI execute the optimal noise canceling settings.

[0055] The setting unit can suggest optimal language settings by referring to the user's past usage history. The setting unit, for example, automatically suggests language settings that the user has frequently used in the past. For example, the setting unit analyzes the user's past usage history and automatically suggests frequently used language settings. The setting unit can also predict and suggest language settings that the user will use during a specific time period. For example, the setting unit analyzes the user's past usage history and predicts and suggests language settings that the user will use during a specific time period. The setting unit can also analyze the user's past usage history and suggest optimal language settings. For example, the setting unit analyzes the user's past usage history and suggests optimal language settings. This allows the setting unit to provide optimal language settings based on the user's past usage history. Some or all of the above-described processing in the setting unit may be performed using, for example, AI, or may be performed without using AI. For example, the setting unit can input the user's past usage history into a generation AI and cause the generation AI to suggest optimal language settings.

[0056] The setting unit can set an optimal language setting based on the user's location information. For example, if the user is in a cafe, the setting unit provides a language setting that matches the surrounding environment. For example, if the setting unit analyzes the user's location information and determines that the user is in a cafe, the setting unit provides a language setting that matches the surrounding environment. The setting unit can also provide a language setting that matches announcements if the user is in a train station. For example, if the setting unit analyzes the user's location information and determines that the user is in a train station, the setting unit provides a language setting that matches announcements. The setting unit can also provide a normal language setting if the user is at home. For example, if the setting unit analyzes the user's location information and determines that the user is at home, the setting unit provides a normal language setting. This allows the setting unit to provide an optimal language setting based on the user's location information. Some or all of the above-described processing in the setting unit may be performed using AI, for example, or may be performed without using AI. For example, the setting unit can input the user's location information to a generation AI and cause the generation AI to execute the optimal language setting.

[0057] The adjustment unit can suggest an optimal translation result by referring to the user's past translation history. The adjustment unit can suggest an optimal translation result based on, for example, translation results used by the user in the past. For example, the adjustment unit can analyze the user's past translation history and suggest an optimal translation result based on the translation results used in the past. The adjustment unit can also prioritize translating phrases that the user has frequently used in the past. For example, the adjustment unit can analyze the user's past translation history and prioritize translating frequently used phrases. The adjustment unit can also analyze the user's past translation history and suggest the most appropriate translation result. For example, the adjustment unit can analyze the user's past translation history and suggest the most appropriate translation result. This allows the adjustment unit to provide an optimal translation result based on the user's past translation history. Some or all of the above-mentioned processing in the adjustment unit can be performed using, for example, AI, or can be performed without using AI. For example, the adjustment unit can input the user's past translation history into the generation AI and cause the generation AI to suggest an optimal translation result.

[0058] The adjustment unit can provide an optimal translation result based on the user's location information. For example, if the user is in a cafe, the adjustment unit provides a translation result tailored to the surrounding environment. For example, if the adjustment unit analyzes the user's location information and determines that the user is in a cafe, it provides a translation result tailored to the surrounding environment. The adjustment unit can also provide a translation result tailored to an announcement if the user is at a train station. For example, if the adjustment unit analyzes the user's location information and determines that the user is at a train station, it provides a translation result tailored to the announcement. The adjustment unit can also provide a normal translation result if the user is at home. For example, if the adjustment unit analyzes the user's location information and determines that the user is at home, it provides a normal translation result. This allows the adjustment unit to provide an optimal translation result based on the user's location information. Some or all of the above-described processing in the adjustment unit may be performed using, for example, AI, or may be performed without using AI. For example, the adjustment unit can input the user's location information to a generation AI and cause the generation AI to execute an optimal translation result.

[0059] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0060] The wireless earphone system may further include a health monitoring unit that monitors the user's health condition. The health monitoring unit, for example, measures the user's heart rate and blood pressure and issues a warning if an abnormality is detected. For example, if the user's heart rate suddenly increases while exercising, the health monitoring unit may issue a warning sound to notify the user. The health monitoring unit may also measure the user's stress level and play music to help relax the user if stress is high. For example, if the health monitoring unit determines that the user is feeling stressed, it may play music with a relaxing effect. This allows the wireless earphone system to monitor the user's health condition and take appropriate measures.

[0061] The wireless earphone system may further include an activity tracking unit that tracks the user's activity level. The activity tracking unit, for example, measures the user's number of steps and calories burned and records daily activity data. For example, it may record the number of steps the user takes per day and calculate the calories burned. The activity tracking unit may also set the user's exercise goal and notify the user of progress toward that goal. For example, if the user sets a goal of walking 10,000 steps per day, the activity tracking unit may notify the user of progress toward that goal. This allows the wireless earphone system to track the user's activity level and support health management.

[0062] The wireless earphone system may further include a location-based music recommendation unit that recommends optimal music based on the user's location information. For example, if the user is in a park, the location-based music recommendation unit may recommend music containing natural sounds. For example, if the user's location information is analyzed and it is determined that the user is in a park, music containing natural sounds may be played. The location-based music recommendation unit may also recommend fast-tempo music suitable for exercise if the user is in a gym. For example, if the user's location information is analyzed and it is determined that the user is in a gym, fast-tempo music may be played. This allows the wireless earphone system to recommend optimal music based on the user's location information.

[0063] The wireless earphone system may further include a historical music recommendation unit that refers to the user's past music playback history and recommends optimal music. The historical music recommendation unit, for example, prioritizes and recommends music that the user has frequently played in the past. For example, it may analyze the user's past music playback history and recommend music that has been frequently played. The historical music recommendation unit may also predict and recommend music that the user played during a specific time period. For example, it may analyze the user's past music playback history and recommend music that was played during a specific time period. This allows the wireless earphone system to recommend optimal music based on the user's past music playback history.

[0064] The wireless earphone system can further include a location information guide unit that provides optimal audio guidance based on the user's location information. For example, if the user is in a tourist spot, the location information guide unit provides information about the location. For example, if the user's location information is analyzed and it is determined that the user is in a tourist spot, the location information guide unit provides information about the history and tourist attractions of the location. The location information guide unit can also provide information about stores when the user is in a shopping mall. For example, if the user's location information is analyzed and it is determined that the user is in a shopping mall, the location information guide unit provides information about stores. This allows the wireless earphone system to provide optimal audio guidance based on the user's location information.

[0065] The processing flow of the first embodiment will be briefly explained below.

[0066] Step 1: The receiver receives the other person's speech as audio. The receiver, for example, may have a built-in high-performance microphone that accurately captures the other person's speech while filtering out surrounding noise. For example, the receiver can clearly pick up the other person's voice even in noisy places such as cafes or train stations. Step 2: The analysis unit analyzes the voice data received by the receiving unit. The analysis unit converts the voice data into text data, for example, using voice recognition technology. For example, if the other person says "Hello," the analysis unit converts the voice data into text data saying "Hello." Step 3: The translation unit translates the text data analyzed by the analysis unit. The translation unit converts the text data into the user's language using, for example, a multilingual translation model. For example, the translation unit translates the Japanese text data "Hello" into the English text data "Hello." Step 4: The playback unit converts the text data translated by the translation unit into audio data and plays it back. For example, the playback unit converts the text data into audio data using speech synthesis technology. For example, the playback unit converts the text data "Hello" into audio data and plays back the audio "Hello" from the earphones.

[0067] (Example 2) A wireless earphone system according to an embodiment of the present invention automatically translates and plays back speech when speaking with a person who speaks a different language. This wireless earphone system receives speech from the other person as audio, and a generation AI analyzes, translates, and converts the speech into the user's language for playback. For example, the earphones receive the speech from the other person. The earphones are equipped with a high-performance microphone that accurately captures the speech while filtering out ambient noise. For example, the earphones can clearly pick up the other person's voice even in noisy places like cafes and train stations. The generation AI then analyzes the received speech data. The generation AI converts the speech data into text data using speech recognition technology. For example, if the other person says "hello," the speech data is converted into the text data "hello." The generation AI then translates the text data. The generation AI then uses a multilingual translation model to convert the text data into the user's language. For example, the Japanese text data "hello" is translated into the English text data "Hello." Finally, the translated text data is converted into audio data, which is played by the earphones. The generation AI then converts the text data into audio data using speech synthesis technology. For example, text data such as "Hello" is converted into voice data, and the voice "Hello" is played from the earphones. This mechanism allows users to communicate smoothly with people who speak different languages. For example, this is extremely convenient when conversing with local people while traveling, or when discussing with participants who speak different languages ​​at an international conference. In this way, the wireless earphone system can enable smooth communication between people who speak different languages.

[0068] A wireless earphone system according to an embodiment includes a receiving unit, an analyzing unit, a translating unit, and a playing unit. The receiving unit receives speech from the other party as audio. The receiving unit, for example, has a built-in high-performance microphone and accurately captures the speech from the other party while filtering out ambient noise. For example, the receiving unit can clearly pick up the other party's voice even in noisy places such as cafes and train stations. The analyzing unit analyzes the audio data received by the receiving unit. The analyzing unit converts the audio data into text data using, for example, voice recognition technology. For example, if the other party says "hello," the analyzing unit converts the audio data into text data saying "hello." The translating unit translates the text data analyzed by the analyzing unit. The translating unit converts the text data into a user's language using, for example, a multilingual translation model. For example, the translating unit translates Japanese text data saying "hello" into English text data saying "Hello." The playing unit converts the text data translated by the translating unit into audio data and plays it back. For example, the playing unit converts text data into audio data using speech synthesis technology. For example, the playback unit converts text data such as "Hello" into audio data and plays back the audio "Hello" from the earphones. This allows the wireless earphone system according to the embodiment to enable smooth communication with people who speak different languages.

[0069] The wireless earphone system includes a noise canceling unit with a noise canceling function. The noise canceling unit removes ambient noise and provides clear audio. The noise canceling unit, for example, uses active noise canceling technology to analyze ambient noise in real time and generate anti-phase sound waves to cancel the noise. The noise canceling unit can also use passive noise canceling technology to block noise through the physical structure of the earphone. For example, the noise canceling unit can block external sounds and provide clear audio by devising the shape and materials of the earphone. In this way, the noise canceling unit can remove ambient noise and provide clear audio.

[0070] The wireless earphone system includes a setting unit for changing a user's language setting. The setting unit allows the user to easily change the language setting. The setting unit allows the user to select a desired language, for example, through a button operation on the earphones or a smartphone app. For example, the setting unit allows the user to select a language such as English, Japanese, or French using the smartphone app. The setting unit can also recognize a user's voice command and change the language setting. For example, if the user says, "Change to English," the setting unit automatically changes the language setting to English. In this way, the setting unit allows the user to easily change the language setting.

[0071] The wireless earphone system includes an adjustment unit that fine-tunes the translation result. The adjustment unit can fine-tune the translation result to suit the user's preferences. The adjustment unit improves the accuracy of the translation, for example, by having the user provide feedback on the translation result. For example, if the user provides feedback such as "this translation is inaccurate," the adjustment unit updates the translation model and improves the translation result from the next time onwards. The adjustment unit can also allow the user to manually correct the translation result. For example, if the user edits the translation result and inputs a correct translation, the adjustment unit learns the correction and reflects it in the translation result from the next time onwards. This allows the adjustment unit to fine-tune the translation result to suit the user's preferences.

[0072] The receiving unit has a built-in microphone and can accurately capture what the other person is saying while filtering out ambient noise. The receiving unit has, for example, a built-in high-performance microphone and can accurately capture what the other person is saying while filtering out ambient noise. For example, the receiving unit can clearly pick up the other person's voice even in noisy places such as cafes and train stations. The receiving unit can also use noise canceling technology to analyze ambient noise in real time and generate anti-phase sound waves to cancel out the noise. For example, the receiving unit can use active noise canceling technology to analyze ambient noise in real time and generate anti-phase sound waves to cancel out the noise. This allows the receiving unit to clearly pick up the other person's voice even in noisy places.

[0073] The analysis unit can convert voice data into text data using voice recognition technology. The analysis unit, for example, converts voice data into text data using voice recognition technology. For example, if the other person says "hello," the analysis unit converts the voice data into text data saying "hello." The analysis unit can achieve highly accurate voice recognition using deep learning-based voice recognition technology. For example, the analysis unit uses a deep neural network to analyze voice data and convert it into text data. The analysis unit can also convert voice data into text data using HMM (hidden Markov model)-based voice recognition technology. For example, the analysis unit uses an HMM to model temporal changes in the voice data and convert it into text data. This allows the analysis unit to accurately convert voice data into text data.

[0074] The translation unit can convert text data into the user's language using a multilingual translation model. The translation unit, for example, converts text data into the user's language using a multilingual translation model. For example, the translation unit translates Japanese text data such as "Hello" into English text data such as "Hello." The translation unit can achieve highly accurate translation using a neural machine translation model. For example, the translation unit analyzes text data using a neural network and converts it into the user's language. The translation unit can also convert text data into the user's language using a statistical machine translation model. For example, the translation unit translates text data using a statistical method. This enables the translation unit to perform multilingual translation.

[0075] The playback unit can convert text data into voice data using voice synthesis technology and play it back through the earphones. For example, the playback unit converts text data into voice data using voice synthesis technology. For example, the playback unit converts text data such as "Hello" into voice data and plays back the voice "Hello" through the earphones. The playback unit can achieve high-quality voice synthesis using text-to-speech synthesis technology. For example, the playback unit generates natural voice using text-to-speech synthesis technology. The playback unit can also generate voice data using concatenative voice synthesis technology. For example, the playback unit generates voice data using concatenative voice synthesis technology and plays it back through the earphones. In this way, the playback unit can play back the translated voice data to the user.

[0076] The receiving unit can estimate the user's emotions and adjust the audio reception sensitivity based on the estimated user emotions. For example, if the user is nervous, the receiving unit increases the reception sensitivity to receive the other party's voice more clearly. For example, the receiving unit analyzes the user's voice data and increases the reception sensitivity if it determines that the user is nervous. The receiving unit can also set the reception sensitivity to normal if the user is relaxed, allowing the user to enjoy a natural conversation. For example, the receiving unit analyzes the user's voice data and sets the reception sensitivity to normal if it determines that the user is relaxed. The receiving unit can also automatically adjust the reception sensitivity if the user is tired, providing an easy-to-listen voice. For example, the receiving unit analyzes the user's voice data and automatically adjusts the reception sensitivity if it determines that the user is tired. This allows the receiving unit to provide optimal reception sensitivity according to the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-described processing in the receiving unit may be performed using, for example, AI, or may be performed without AI. For example, the receiving unit can input the user's voice data into the generation AI and have the generation AI estimate emotions.

[0077] The receiving unit can be set to preferentially receive a specific frequency band. For example, the receiving unit may preferentially receive a frequency band of human voices to suppress ambient noise. For example, the receiving unit may preferentially receive a frequency band from 300 Hz to 3 kHz to suppress ambient noise. The receiving unit may also preferentially receive high-frequency sounds to provide clear audio. For example, the receiving unit may preferentially receive high-frequency sounds above 3 kHz to provide clear audio. The receiving unit may also preferentially receive low-frequency sounds to provide deep audio. For example, the receiving unit may preferentially receive low-frequency sounds below 300 Hz to provide deep audio. In this way, the receiving unit can provide clear audio by preferentially receiving a specific frequency band. Some or all of the above-described processing in the receiving unit may be performed using, or without, AI. For example, the receiving unit may input audio data to a generation AI and cause the generation AI to preferentially receive a specific frequency band.

[0078] The receiving unit can have the function of simultaneously receiving audio from multiple sound sources and analyzing it individually. For example, the receiving unit may simultaneously receive the voices of multiple speakers during a meeting and analyze them individually. For example, the receiving unit may use multiple microphones to simultaneously receive the voices of multiple speakers and analyze them individually. The receiving unit may also simultaneously receive multiple conversations at a cafe and analyze only the necessary conversations. For example, the receiving unit may use multiple microphones to simultaneously receive multiple conversations at a cafe and analyze only the necessary conversations. The receiving unit may also simultaneously receive a station announcement and a friend's conversation and analyze them individually. For example, the receiving unit may use multiple microphones to simultaneously receive a station announcement and a friend's conversation and analyze them individually. This allows the receiving unit to efficiently obtain necessary information by simultaneously receiving audio from multiple sound sources and analyzing them individually. Some or all of the above-described processing in the receiving unit may be performed using, for example, AI, or may be performed without AI. For example, the receiving unit may input multiple pieces of audio data into a generation AI and have the generation AI perform individual analyses.

[0079] The receiving unit can estimate the user's emotions and determine the priority of audio to be received based on the estimated user's emotions. For example, if the user is nervous, the receiving unit prioritizes receiving important conversations. For example, if the receiving unit analyzes the user's voice data and determines that the user is nervous, it prioritizes receiving important conversations. The receiving unit can also receive all conversations equally if the user is relaxed. For example, if the receiving unit analyzes the user's voice data and determines that the user is relaxed, it receives all conversations equally. The receiving unit can also prioritize receiving important information if the user is in a hurry. For example, if the receiving unit analyzes the user's voice data and determines that the user is in a hurry, it prioritizes receiving important information. This allows the receiving unit to prioritize receiving important audio according to the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-mentioned processing in the receiving unit may be performed using, for example, AI, or may be performed without AI. For example, the receiving unit can input the user's voice data to a generation AI and cause the generation AI to estimate the user's emotions.

[0080] The receiving unit can set optimal voice reception settings based on the user's location information. For example, if the user is in a cafe, the receiving unit receives conversations while suppressing ambient noise. For example, if the receiving unit analyzes the user's location information and determines that the user is in a cafe, the receiving unit receives conversations while suppressing ambient noise. The receiving unit can also prioritize receiving announcements if the user is at a train station. For example, if the receiving unit analyzes the user's location information and determines that the user is at a train station, the receiving unit prioritizes receiving announcements. The receiving unit can also set normal reception settings if the user is at home. For example, if the receiving unit analyzes the user's location information and determines that the user is at home, the receiving unit sets normal reception settings. This allows the receiving unit to provide optimal voice reception settings based on the user's location information. Some or all of the above-described processing in the receiving unit may be performed using AI, for example, or may be performed without using AI. For example, the receiving unit can input the user's location information to a generation AI and cause the generation AI to execute optimal voice reception settings.

[0081] The receiving unit can refer to the user's past conversation history and preferentially receive the voice of a specific speaker. The receiving unit, for example, preferentially receives the voice of a person with whom the user has frequently conversed in the past. For example, the receiving unit analyzes the user's past conversation history and preferentially receives the voice of a person with whom the user has frequently conversed. The receiving unit can also preferentially receive the voice of a person with whom the user spoke in an important meeting. For example, the receiving unit analyzes the user's past conversation history and preferentially receives the voice of a person with whom the user spoke in an important meeting. The receiving unit can also preferentially receive the conversations the user has with family members. For example, the receiving unit analyzes the user's past conversation history and preferentially receives the conversations with family members. In this way, the receiving unit can preferentially receive the voice of an important speaker based on the user's past conversation history. Some or all of the above-described processing in the receiving unit may be performed using, for example, AI, or may be performed without using AI. For example, the receiving unit can input the user's past conversation history to the generation AI and cause the generation AI to preferentially receive the voice of a specific speaker.

[0082] The analysis unit can estimate the user's emotions and adjust the analysis algorithm based on the estimated user emotions. For example, if the user is nervous, the analysis unit speeds up the analysis algorithm and provides results quickly. For example, the analysis unit analyzes the user's voice data and determines that the user is nervous, then speeds up the analysis algorithm. The analysis unit can also use a normal analysis algorithm if the user is relaxed. For example, the analysis unit analyzes the user's voice data and determines that the user is relaxed, then uses a normal analysis algorithm. The analysis unit can also optimize the analysis algorithm if the user is tired, and provide accurate results. For example, the analysis unit analyzes the user's voice data and determines that the user is tired, then optimizes the analysis algorithm. This allows the analysis unit to provide an optimal analysis algorithm depending on the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without AI. For example, the analysis unit can input the user's voice data to a generation AI and have the generation AI perform emotion estimation.

[0083] The analysis unit may have a function to automatically identify and remove background sounds from audio data. The analysis unit may, for example, automatically identify and remove noise during a meeting. For example, the analysis unit may analyze audio data and automatically identify and remove noise during a meeting. The analysis unit may also automatically identify and remove background sounds in a cafe. For example, the analysis unit may analyze audio data and automatically identify and remove background sounds in a cafe. The analysis unit may also automatically identify and remove noise other than station announcements. For example, the analysis unit may analyze audio data and automatically identify and remove noise other than station announcements. In this way, the analysis unit can provide clear audio by automatically removing background sounds from audio data. Some or all of the above-described processing in the analysis unit may be performed using, or without, AI. For example, the analysis unit may input audio data to a generation AI and cause the generation AI to identify and remove background sounds.

[0084] The analysis unit can identify speakers of the audio data and perform different analyses for each speaker. The analysis unit, for example, can identify multiple speakers in a meeting and analyze them individually. For example, the analysis unit can analyze audio data to identify multiple speakers in a meeting and analyze them individually. The analysis unit can also identify multiple conversations at a cafe and analyze only the necessary conversations. For example, the analysis unit can analyze audio data to identify multiple conversations at a cafe and analyze only the necessary conversations. The analysis unit can also identify station announcements and conversations with friends and analyze them individually. For example, the analysis unit can analyze audio data to identify station announcements and conversations with friends and analyze them individually. This allows the analysis unit to identify speakers of the audio data and provide optimal analysis for each speaker. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without AI. For example, the analysis unit can input audio data to a generation AI and have the generation AI perform speaker identification and individual analysis.

[0085] The analysis unit can estimate the user's emotions and adjust the display method of the analysis results based on the estimated user emotions. For example, if the user is nervous, the analysis unit provides a simple, highly visible display method. For example, if the analysis unit analyzes the user's voice data and determines that the user is nervous, it provides a simple, highly visible display method. The analysis unit can also provide a display method including detailed information if the user is relaxed. For example, if the analysis unit analyzes the user's voice data and determines that the user is relaxed, it provides a display method including detailed information. The analysis unit can also provide a display method that focuses on the main points if the user is in a hurry. For example, if the analysis unit analyzes the user's voice data and determines that the user is in a hurry, it provides a display method that focuses on the main points. This allows the analysis unit to provide an optimal display method of the analysis results depending on the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the user's voice data into the generation AI and have the generation AI estimate emotions.

[0086] The analysis unit can prioritize analyzing specific keywords by referring to the content of the user's past conversations. The analysis unit, for example, prioritizes analyzing keywords that the user frequently used in the past. For example, the analysis unit analyzes the content of the user's past conversations and prioritizes analyzing frequently used keywords. The analysis unit can also prioritize analyzing keywords that the user used in important meetings. For example, the analysis unit analyzes the content of the user's past conversations and prioritizes analyzing keywords that were used in important meetings. The analysis unit can also prioritize analyzing keywords that the user used in conversations with family members. For example, the analysis unit analyzes the content of the user's past conversations and prioritizes analyzing keywords that were used in conversations with family members. This allows the analysis unit to prioritize analyzing important keywords based on the content of the user's past conversations. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the content of the user's past conversations to the generation AI and cause the generation AI to perform prioritized analysis of specific keywords.

[0087] The analysis unit can analyze the user's current environmental sounds and perform optimal voice recognition settings. For example, when the user is in a cafe, the analysis unit analyzes the environmental sounds and performs optimal voice recognition settings. For example, when the analysis unit analyzes the user's environmental sounds and determines that the user is in a cafe, the analysis unit analyzes the environmental sounds and performs optimal voice recognition settings. The analysis unit can also analyze the environmental sounds and perform optimal voice recognition settings when the user is in a train station. For example, when the analysis unit analyzes the user's environmental sounds and determines that the user is in a train station, the analysis unit analyzes the environmental sounds and performs optimal voice recognition settings. The analysis unit can also analyze the environmental sounds and perform optimal voice recognition settings when the user is at home. For example, when the analysis unit analyzes the user's environmental sounds and determines that the user is at home, the analysis unit analyzes the environmental sounds and performs optimal voice recognition settings. This allows the analysis unit to provide optimal voice recognition settings based on the user's current environmental sounds. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the user's environmental sounds into the generation AI and have the generation AI execute optimal voice recognition settings.

[0088] The translation unit can estimate the user's emotions and adjust the translation expression method based on the estimated user's emotions. For example, if the user is nervous, the translation unit provides a simple and easy-to-understand expression. For example, if the translation unit analyzes the user's voice data and determines that the user is nervous, it provides a simple and easy-to-understand expression. The translation unit can also provide a detailed expression if the user is relaxed. For example, if the translation unit analyzes the user's voice data and determines that the user is relaxed, it provides a detailed expression. The translation unit can also provide a quick and concise expression if the user is in a hurry. For example, if the translation unit analyzes the user's voice data and determines that the user is in a hurry, it provides a quick and concise expression. This allows the translation unit to provide the optimal translation expression method according to the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-mentioned processing in the translation unit may be performed using, for example, AI, or may be performed without AI. For example, the translation unit can input the user's voice data into a generation AI and have the generation AI perform emotion estimation.

[0089] The translation unit may have the function of automatically recognizing specific technical terms and slang and appropriately translating them. The translation unit, for example, automatically recognizes medical terms and appropriately translates them. For example, the translation unit may analyze voice data, automatically recognize medical terms, and appropriately translate them. The translation unit may also automatically recognize business terms and appropriately translate them. For example, the translation unit may analyze voice data, automatically recognize business terms, and appropriately translate them. The translation unit may also automatically recognize youth slang and appropriately translate them. For example, the translation unit may analyze voice data, automatically recognize youth slang, and appropriately translate them. This allows the translation unit to appropriately translate specific technical terms and slang, thereby providing accurate translations. Some or all of the above-described processing in the translation unit may be performed using, or without, AI. For example, the translation unit may input voice data into a generation AI and have the generation AI recognize and translate technical terms and slang.

[0090] The translation unit can present multiple translation candidates based on the context, allowing the user to select one. The translation unit, for example, presents multiple translation candidates based on the context of a conversation, allowing the user to select one. For example, the translation unit analyzes audio data and presents multiple translation candidates based on the context of the conversation, allowing the user to select one. The translation unit can also present multiple translation candidates based on the context of an email, allowing the user to select one. For example, the translation unit analyzes audio data and presents multiple translation candidates based on the context of the email, allowing the user to select one. The translation unit can also present multiple translation candidates based on the context of a presentation, allowing the user to select one. For example, the translation unit analyzes audio data and presents multiple translation candidates based on the context of the presentation, allowing the user to select one. In this way, the translation unit provides multiple translation candidates based on the context, allowing the user to select the optimal translation. Some or all of the above-described processing in the translation unit may be performed using, for example, AI, or may be performed without AI. For example, the translation unit can input audio data to a generation AI and cause the generation AI to present multiple translation candidates based on the context.

[0091] The translation unit can estimate the user's emotions and adjust the level of detail in the translation based on the estimated user's emotions. For example, if the user is nervous, the translation unit provides a concise and to-the-point translation. For example, if the translation unit analyzes the user's voice data and determines that the user is nervous, it provides a concise and to-the-point translation. The translation unit can also provide a detailed translation if the user is relaxed. For example, if the translation unit analyzes the user's voice data and determines that the user is relaxed, it provides a detailed translation. The translation unit can also provide a quick and concise translation if the user is in a hurry. For example, if the translation unit analyzes the user's voice data and determines that the user is in a hurry, it provides a quick and concise translation. This allows the translation unit to provide an optimal level of detail in the translation depending on the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-mentioned processing in the translation unit may be performed using, for example, AI, or without AI. For example, the translation unit can input the user's voice data into a generation AI and have the generation AI perform emotion estimation.

[0092] The translation unit can provide an optimal translation by referring to the user's past translation history. The translation unit can provide an optimal translation based on, for example, translations used by the user in the past. For example, the translation unit can analyze the user's past translation history and provide an optimal translation based on the translations used in the past. The translation unit can also prioritize translation of phrases that the user has used frequently in the past. For example, the translation unit can analyze the user's past translation history and prioritize translation of frequently used phrases. The translation unit can also analyze the user's past translation history and provide the most appropriate translation. For example, the translation unit can analyze the user's past translation history and provide the most appropriate translation. This allows the translation unit to provide an optimal translation based on the user's past translation history. Some or all of the above-described processing in the translation unit can be performed using, for example, AI, or can be performed without using AI. For example, the translation unit can input the user's past translation history into a generation AI and cause the generation AI to perform an optimal translation.

[0093] The translation unit can analyze the content of a user's current conversation in real time and provide an appropriate translation. For example, the translation unit can analyze in real time while a user is talking and provide an appropriate translation. For example, the translation unit can analyze voice data in real time and provide an appropriate translation during the conversation. The translation unit can also analyze in real time while a user is giving a presentation and provide an appropriate translation. For example, the translation unit can analyze voice data in real time and provide an appropriate translation during the presentation. The translation unit can also analyze in real time while a user is on the phone and provide an appropriate translation. For example, the translation unit can analyze voice data in real time and provide an appropriate translation during the phone call. This allows the translation unit to analyze the content of the conversation in real time and provide an appropriate translation. Some or all of the above-mentioned processing in the translation unit may be performed using AI, for example, or may be performed without using AI. For example, the translation unit can input voice data to a generation AI and have the generation AI perform real-time analysis and translation.

[0094] The playback unit can estimate the user's emotions and adjust the playback speed of the audio based on the estimated user emotions. For example, if the user is nervous, the playback unit slows down the playback speed to make the audio easier to understand. For example, the playback unit analyzes the user's voice data and slows down the playback speed if it determines that the user is nervous. The playback unit can also play audio at a normal playback speed if the user is relaxed. For example, the playback unit analyzes the user's voice data and plays audio at a normal playback speed if it determines that the user is relaxed. The playback unit can also speed up the playback speed if the user is in a hurry to provide information quickly. For example, the playback unit analyzes the user's voice data and speeds up the playback speed if it determines that the user is in a hurry. This allows the playback unit to provide an optimal playback speed depending on the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-mentioned processing in the playback unit may be performed using, for example, AI, or without AI. For example, the playback unit can input the user's voice data to a generation AI and have the generation AI perform emotion estimation.

[0095] The playback unit may have a function to automatically adjust a specific voice tone or pitch. For example, if a user prefers a high-pitched voice, the playback unit adjusts the voice tone higher. For example, if the playback unit analyzes the user's voice data and determines that the user prefers a high-pitched voice, it adjusts the voice tone higher. Furthermore, if the user prefers a low-pitched voice, the playback unit can adjust the voice tone lower. For example, if the playback unit analyzes the user's voice data and determines that the user prefers a low-pitched voice, it adjusts the voice tone lower. Furthermore, if the user prefers a specific pitch, the playback unit can adjust the pitch to that pitch. For example, if the playback unit analyzes the user's voice data and determines that the user prefers a specific pitch, it adjusts the pitch to that pitch. In this way, the playback unit can provide the user with an optimal voice by adjusting the specific voice tone or pitch. Some or all of the above-described processing in the playback unit may be performed using, for example, AI, or may be performed without AI. For example, the playback unit may input the user's voice data to a generation AI and have the generation AI adjust the voice tone or pitch.

[0096] The playback unit can simultaneously play multiple pieces of audio data, allowing the user to select from them. For example, the playback unit can simultaneously play the audio of multiple speakers during a conference, allowing the user to select from them. For example, the playback unit can simultaneously play multiple pieces of audio data, allowing the user to select from them. The playback unit can also simultaneously play multiple conversations at a cafe, allowing the user to select from them. For example, the playback unit can simultaneously play multiple pieces of audio data, allowing the user to select from them. The playback unit can also simultaneously play a station announcement and a conversation between friends, allowing the user to select from them. For example, the playback unit can simultaneously play multiple pieces of audio data, allowing the user to select from them. In this way, the playback unit can simultaneously play multiple pieces of audio data, allowing the user to select from the audio they need. Some or all of the above-described processing in the playback unit may be performed using, for example, AI, or may be performed without using AI. For example, the playback unit can input multiple pieces of audio data to a generation AI, causing the generation AI to perform simultaneous playback and selection.

[0097] The playback unit can estimate the user's emotions and determine the priority of audio to be played back based on the estimated user's emotions. For example, if the user is nervous, the playback unit prioritizes playing back important audio. For example, if the playback unit analyzes the user's audio data and determines that the user is nervous, it prioritizes playing back important audio. The playback unit can also play back all audio evenly if the user is relaxed. For example, if the playback unit analyzes the user's audio data and determines that the user is relaxed, it prioritizes playing back all audio evenly. The playback unit can also prioritize playing back important information if the user is in a hurry. For example, if the playback unit analyzes the user's audio data and determines that the user is in a hurry, it prioritizes playing back important information. This allows the playback unit to prioritize playing back important audio according to the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-mentioned processing in the playback unit may be performed using, for example, AI, or may be performed without AI. For example, the playback unit can input the user's audio data to a generation AI and cause the generation AI to estimate emotions.

[0098] The playback unit can set optimal audio playback settings based on the user's location information. For example, if the user is in a cafe, the playback unit plays audio while suppressing ambient noise. For example, if the playback unit analyzes the user's location information and determines that the user is in a cafe, the playback unit plays audio while suppressing ambient noise. The playback unit can also prioritize playing announcements when the user is at a train station. For example, if the playback unit analyzes the user's location information and determines that the user is at a train station, the playback unit prioritizes playing announcements. The playback unit can also set normal playback settings when the user is at home. For example, if the playback unit analyzes the user's location information and determines that the user is at home, the playback unit sets normal playback settings. This allows the playback unit to provide optimal audio playback settings based on the user's location information. Some or all of the above-described processing in the playback unit may be performed using, for example, AI, or may be performed without using AI. For example, the playback unit can input the user's location information to a generation AI and cause the generation AI to execute optimal audio playback settings.

[0099] The playback unit can prioritize playback of specific audio by referring to the user's past playback history. The playback unit, for example, prioritizes playback of audio that the user has frequently played in the past. For example, the playback unit can analyze the user's past playback history and prioritize playback of audio that was frequently played. The playback unit can also prioritize playback of audio that the user played in important meetings. For example, the playback unit can analyze the user's past playback history and prioritize playback of audio that was played in important meetings. The playback unit can also prioritize playback of conversations the user had with family members. For example, the playback unit can analyze the user's past playback history and prioritize playback of conversations with family members. This allows the playback unit to prioritize playback of important audio based on the user's past playback history. Some or all of the above-described processing in the playback unit may be performed using, for example, AI, or may be performed without using AI. For example, the playback unit can input the user's past playback history to a generation AI and cause the generation AI to prioritize playback of specific audio.

[0100] The noise canceling unit can estimate the user's emotions and adjust the noise canceling intensity based on the estimated user's emotions. For example, if the user is nervous, the noise canceling unit increases the noise canceling intensity to make it easier for the user to concentrate. For example, the noise canceling unit analyzes the user's voice data and increases the noise canceling intensity if it determines that the user is nervous. The noise canceling unit can also set the noise canceling intensity to normal if the user is relaxed. For example, the noise canceling unit analyzes the user's voice data and sets the noise canceling intensity to normal if it determines that the user is relaxed. The noise canceling unit can also automatically adjust the noise canceling intensity if the user is tired, providing a comfortable audio environment. For example, the noise canceling unit analyzes the user's voice data and automatically adjusts the noise canceling intensity if it determines that the user is tired. This allows the noise canceling unit to provide an optimal noise canceling intensity according to the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-described processing in the noise canceling unit may be performed using, for example, AI, or may be performed without using AI. For example, the noise canceling unit may input user voice data to a generation AI and have the generation AI estimate emotions.

[0101] The noise canceling unit can have a function of preferentially removing noise in a specific frequency band. For example, the noise canceling unit preferentially removes noise in a low frequency band to provide a deeper sound. For example, the noise canceling unit preferentially removes noise in a low frequency band of 300 Hz or less to provide a deeper sound. The noise canceling unit can also preferentially remove noise in a high frequency band to provide a clearer sound. For example, the noise canceling unit preferentially removes noise in a high frequency band of 3 kHz or more to provide a clearer sound. The noise canceling unit can also preferentially remove noise in a mid-frequency band to provide a balanced sound. For example, the noise canceling unit preferentially removes noise in a mid-frequency band of 300 Hz to 3 kHz to provide a balanced sound. In this way, the noise canceling unit can provide a clearer sound by removing noise in a specific frequency band. Some or all of the above-described processing in the noise canceling unit may be performed using AI, or may be performed without using AI. For example, the noise canceling unit may input audio data to a generation AI and cause the generation AI to remove noise in a specific frequency band.

[0102] The noise canceling unit can estimate the user's emotions and determine the priority of noise canceling based on the estimated user's emotions. For example, if the user is nervous, the noise canceling unit prioritizes noise canceling of important sounds. For example, if the noise canceling unit analyzes the user's voice data and determines that the user is nervous, it prioritizes noise canceling of important sounds. The noise canceling unit can also uniformly noise cancel all sounds if the user is relaxed. For example, if the noise canceling unit analyzes the user's voice data and determines that the user is relaxed, it uniformly noise cancels all sounds. The noise canceling unit can also prioritize noise canceling of important information if the user is in a hurry. For example, if the noise canceling unit analyzes the user's voice data and determines that the user is in a hurry, it prioritizes noise canceling of important information. This allows the noise canceling unit to prioritize noise canceling of important sounds according to the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-described processing in the noise canceling unit may be performed using, for example, AI, or may be performed without using AI. For example, the noise canceling unit may input user voice data to a generation AI and have the generation AI estimate emotions.

[0103] The noise canceling unit can perform optimal noise canceling settings based on the user's location information. For example, when the user is in a cafe, the noise canceling unit suppresses ambient noise and performs noise canceling. For example, the noise canceling unit analyzes the user's location information and, if it determines that the user is in a cafe, suppresses ambient noise and performs noise canceling. The noise canceling unit can also prioritize noise canceling of announcements when the user is at a train station. For example, the noise canceling unit analyzes the user's location information and, if it determines that the user is at a train station, prioritizes noise canceling of announcements. The noise canceling unit can also perform normal noise canceling settings when the user is at home. For example, the noise canceling unit analyzes the user's location information and, if it determines that the user is at home, performs normal noise canceling settings. This allows the noise canceling unit to provide optimal noise canceling settings based on the user's location information. Some or all of the above-described processing in the noise canceling unit may be performed using, for example, AI, or may be performed without using AI. For example, the noise canceling unit can input the user's location information into the generation AI and have the generation AI execute the optimal noise canceling settings.

[0104] The setting unit can estimate the user's emotions and automatically change the language setting based on the estimated user's emotions. For example, if the user is nervous, the setting unit provides a simple language setting. For example, if the setting unit analyzes the user's voice data and determines that the user is nervous, the setting unit provides a simple language setting. The setting unit can also provide a detailed language setting if the user is relaxed. For example, if the setting unit analyzes the user's voice data and determines that the user is relaxed, the setting unit provides a detailed language setting. The setting unit can also quickly change the language setting if the user is in a hurry. For example, if the setting unit analyzes the user's voice data and determines that the user is in a hurry, the setting unit quickly changes the language setting. This allows the setting unit to provide an optimal language setting according to the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-mentioned processing in the setting unit may be performed using, for example, AI, or may be performed without AI. For example, the setting unit can input the user's voice data to a generation AI and cause the generation AI to estimate the user's emotions.

[0105] The setting unit can suggest optimal language settings by referring to the user's past usage history. The setting unit, for example, automatically suggests language settings that the user has frequently used in the past. For example, the setting unit analyzes the user's past usage history and automatically suggests frequently used language settings. The setting unit can also predict and suggest language settings that the user will use during a specific time period. For example, the setting unit analyzes the user's past usage history and predicts and suggests language settings that the user will use during a specific time period. The setting unit can also analyze the user's past usage history and suggest optimal language settings. For example, the setting unit analyzes the user's past usage history and suggests optimal language settings. This allows the setting unit to provide optimal language settings based on the user's past usage history. Some or all of the above-described processing in the setting unit may be performed using, for example, AI, or may be performed without using AI. For example, the setting unit can input the user's past usage history into a generation AI and cause the generation AI to suggest optimal language settings.

[0106] The setting unit can estimate the user's emotions and determine the priority of language settings based on the estimated user emotions. For example, if the user is nervous, the setting unit can prioritize providing an important language setting. For example, if the setting unit analyzes the user's voice data and determines that the user is nervous, the setting unit can prioritize providing an important language setting. The setting unit can also provide all language settings equally if the user is relaxed. For example, if the setting unit analyzes the user's voice data and determines that the user is relaxed, the setting unit can provide all language settings equally. The setting unit can also prioritize providing an important language setting if the user is in a hurry. For example, if the setting unit analyzes the user's voice data and determines that the user is in a hurry, the setting unit can prioritize providing an important language setting. This allows the setting unit to prioritize providing an important language setting according to the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-mentioned processing in the setting unit may be performed using, for example, AI, or may be performed without AI. For example, the setting unit can input the user's voice data to a generation AI and cause the generation AI to estimate emotions.

[0107] The setting unit can set an optimal language setting based on the user's location information. For example, if the user is in a cafe, the setting unit provides a language setting that matches the surrounding environment. For example, if the setting unit analyzes the user's location information and determines that the user is in a cafe, the setting unit provides a language setting that matches the surrounding environment. The setting unit can also provide a language setting that matches announcements if the user is in a train station. For example, if the setting unit analyzes the user's location information and determines that the user is in a train station, the setting unit provides a language setting that matches announcements. The setting unit can also provide a normal language setting if the user is at home. For example, if the setting unit analyzes the user's location information and determines that the user is at home, the setting unit provides a normal language setting. This allows the setting unit to provide an optimal language setting based on the user's location information. Some or all of the above-described processing in the setting unit may be performed using AI, for example, or may be performed without using AI. For example, the setting unit can input the user's location information to a generation AI and cause the generation AI to execute the optimal language setting.

[0108] The adjustment unit can estimate the user's emotions and fine-tune the translation result based on the estimated user's emotions. For example, if the user is nervous, the adjustment unit provides a simple and easy-to-understand translation result. For example, if the adjustment unit analyzes the user's voice data and determines that the user is nervous, it provides a simple and easy-to-understand translation result. The adjustment unit can also provide a detailed translation result if the user is relaxed. For example, if the adjustment unit analyzes the user's voice data and determines that the user is relaxed, it provides a detailed translation result. The adjustment unit can also provide a quick and concise translation result if the user is in a hurry. For example, if the adjustment unit analyzes the user's voice data and determines that the user is in a hurry, it provides a quick and concise translation result. This allows the adjustment unit to provide an optimal translation result according to the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-mentioned processing in the adjustment unit may be performed using, for example, AI, or may be performed without AI. For example, the adjustment unit can input the user's voice data to a generation AI and have the generation AI perform emotion estimation.

[0109] The adjustment unit can suggest an optimal translation result by referring to the user's past translation history. The adjustment unit can suggest an optimal translation result based on, for example, translation results used by the user in the past. For example, the adjustment unit can analyze the user's past translation history and suggest an optimal translation result based on the translation results used in the past. The adjustment unit can also prioritize translating phrases that the user has frequently used in the past. For example, the adjustment unit can analyze the user's past translation history and prioritize translating frequently used phrases. The adjustment unit can also analyze the user's past translation history and suggest the most appropriate translation result. For example, the adjustment unit can analyze the user's past translation history and suggest the most appropriate translation result. This allows the adjustment unit to provide an optimal translation result based on the user's past translation history. Some or all of the above-mentioned processing in the adjustment unit can be performed using, for example, AI, or can be performed without using AI. For example, the adjustment unit can input the user's past translation history into the generation AI and cause the generation AI to suggest an optimal translation result.

[0110] The adjustment unit can estimate the user's emotions and determine the priority of translation results based on the estimated user's emotions. For example, if the user is nervous, the adjustment unit can prioritize providing important translation results. For example, if the adjustment unit analyzes the user's voice data and determines that the user is nervous, it can prioritize providing important translation results. The adjustment unit can also provide all translation results equally if the user is relaxed. For example, if the adjustment unit analyzes the user's voice data and determines that the user is relaxed, it can prioritize providing all translation results equally. The adjustment unit can also prioritize providing important translation results if the user is in a hurry. For example, if the adjustment unit analyzes the user's voice data and determines that the user is in a hurry, it can prioritize providing important translation results. This allows the adjustment unit to prioritize providing important translation results according to the user's emotions. Emotion estimation is performed using technologies such as voice analysis and facial expression recognition. Some or all of the above-mentioned processing in the adjustment unit may be performed using, for example, AI, or may be performed without AI. For example, the adjustment unit can input the user's voice data to a generation AI and cause the generation AI to estimate emotions.

[0111] The adjustment unit can provide an optimal translation result based on the user's location information. For example, if the user is in a cafe, the adjustment unit provides a translation result tailored to the surrounding environment. For example, if the adjustment unit analyzes the user's location information and determines that the user is in a cafe, it provides a translation result tailored to the surrounding environment. The adjustment unit can also provide a translation result tailored to an announcement if the user is at a train station. For example, if the adjustment unit analyzes the user's location information and determines that the user is at a train station, it provides a translation result tailored to the announcement. The adjustment unit can also provide a normal translation result if the user is at home. For example, if the adjustment unit analyzes the user's location information and determines that the user is at home, it provides a normal translation result. This allows the adjustment unit to provide an optimal translation result based on the user's location information. Some or all of the above-described processing in the adjustment unit may be performed using, for example, AI, or may be performed without using AI. For example, the adjustment unit can input the user's location information to a generation AI and cause the generation AI to execute an optimal translation result. === Hard Collateral 1-1 === Each of the multiple elements, including the receiving unit, analyzing unit, translating unit, reproducing unit, noise canceling unit, setting unit, adjusting unit, and emotion estimating unit, is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the receiving unit receives the other person's speech as audio using the smart device 14's high-performance microphone. The analyzing unit converts the audio data into text data using the specific processing unit 290 of the data processing device 12. The translating unit translates the text data into the user's language using the specific processing unit 290 of the data processing device 12. The reproducing unit plays back the translated audio data using the control unit 46A of the smart device 14. The noise canceling unit removes ambient noise using the smart device 14's active noise canceling technology. The setting unit changes the language setting via button operations on the smart device 14 or a smartphone app. The adjusting unit fine-tunes the translation result using the specific processing unit 290 of the data processing device 12. The emotion estimating unit estimates the user's emotion using the specific processing unit 290 of the data processing device 12 and adjusts the receiving sensitivity. === Hard Collateral 1-2 === Each of the multiple elements, including the above-mentioned receiving unit, analyzing unit, translating unit, reproducing unit, noise canceling unit, setting unit, adjusting unit, and emotion estimation unit, is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the receiving unit receives the other person's speech as audio using a high-performance microphone of the smart glasses 214. The analyzing unit converts the audio data into text data using the specific processing unit 290 of the data processing device 12. The translating unit translates the text data into the user's language using the specific processing unit 290 of the data processing device 12. The reproducing unit reproduces the translated audio data using the control unit 46A of the smart glasses 214. The noise canceling unit removes ambient noise using the active noise canceling technology of the smart glasses 214. The setting unit changes the language setting by operating the buttons on the smart glasses 214 or through a smartphone app. The adjusting unit fine-tunes the translation result using the specific processing unit 290 of the data processing device 12. The emotion estimation unit estimates the user's emotion using the specific processing unit 290 of the data processing device 12 and adjusts the receiving sensitivity. === Hard Collateral 1-3 === Each of the multiple elements including the above-mentioned receiving unit, analyzing unit, translating unit, reproducing unit, noise canceling unit, setting unit, adjusting unit, and emotion estimation unit is realized, for example, by at least one of the headset-type terminal 314 and the data processing device 12. For example, the receiving unit receives the other person's speech as audio using a high-performance microphone of the headset-type terminal 314. The analyzing unit converts the audio data into text data using the specific processing unit 290 of the data processing device 12. The translating unit translates the text data into the user's language using the specific processing unit 290 of the data processing device 12. The reproducing unit reproduces the translated audio data using the control unit 46A of the headset-type terminal 314. The noise canceling unit removes ambient noise using the active noise canceling technology of the headset-type terminal 314. The setting unit changes the language setting by operating the buttons on the headset-type terminal 314 or through a smartphone app. The adjusting unit fine-tunes the translation result using the specific processing unit 290 of the data processing device 12. The emotion estimation unit estimates the user's emotion using the specific processing unit 290 of the data processing device 12, and adjusts the reception sensitivity. === Hard Collateral 1-4 === Each of the multiple elements, including the receiving unit, analyzing unit, translating unit, reproducing unit, noise canceling unit, setting unit, adjusting unit, and emotion estimating unit, is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the receiving unit receives the other person's speech as audio using the robot 414's high-performance microphone. The analyzing unit converts the audio data into text data using the specific processing unit 290 of the data processing device 12. The translating unit translates the text data into the user's language using the specific processing unit 290 of the data processing device 12. The reproducing unit plays back the translated audio data using the control unit 46A of the robot 414. The noise canceling unit removes ambient noise using the robot 414's active noise canceling technology. The setting unit changes the language setting by operating the robot 414's buttons or via a smartphone app. The adjusting unit fine-tunes the translation result using the specific processing unit 290 of the data processing device 12. The emotion estimating unit estimates the user's emotion using the specific processing unit 290 of the data processing device 12 and adjusts the receiving sensitivity.

[0112] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0113] The wireless earphone system may further include a health monitoring unit that monitors the user's health condition. The health monitoring unit, for example, measures the user's heart rate and blood pressure and issues a warning if an abnormality is detected. For example, if the user's heart rate suddenly increases while exercising, the health monitoring unit may issue a warning sound to notify the user. The health monitoring unit may also measure the user's stress level and play music to help relax the user if stress is high. For example, if the health monitoring unit determines that the user is feeling stressed, it may play music with a relaxing effect. This allows the wireless earphone system to monitor the user's health condition and take appropriate measures.

[0114] The wireless earphone system may further include an activity tracking unit that tracks the user's activity level. The activity tracking unit, for example, measures the user's number of steps and calories burned and records daily activity data. For example, it may record the number of steps the user takes per day and calculate the calories burned. The activity tracking unit may also set the user's exercise goal and notify the user of progress toward that goal. For example, if the user sets a goal of walking 10,000 steps per day, the activity tracking unit may notify the user of progress toward that goal. This allows the wireless earphone system to track the user's activity level and support health management.

[0115] The wireless earphone system may further include a music recommendation unit that estimates the user's emotions and automatically generates a music playlist based on the estimated emotions. For example, if the user feels sad, the music recommendation unit may recommend uplifting music. For example, if the user's voice data is analyzed and it is determined that the user is sad, uplifting music may be played. The music recommendation unit may also recommend music with a relaxing effect if the user is relaxed. For example, if the user's voice data is analyzed and it is determined that the user is relaxed, relaxing music may be played. This allows the wireless earphone system to automatically recommend music according to the user's emotions.

[0116] The wireless earphone system may further include a location-based music recommendation unit that recommends optimal music based on the user's location information. For example, if the user is in a park, the location-based music recommendation unit may recommend music containing natural sounds. For example, if the user's location information is analyzed and it is determined that the user is in a park, music containing natural sounds may be played. The location-based music recommendation unit may also recommend fast-tempo music suitable for exercise if the user is in a gym. For example, if the user's location information is analyzed and it is determined that the user is in a gym, fast-tempo music may be played. This allows the wireless earphone system to recommend optimal music based on the user's location information.

[0117] The wireless earphone system may further include a historical music recommendation unit that refers to the user's past music playback history and recommends optimal music. The historical music recommendation unit, for example, prioritizes and recommends music that the user has frequently played in the past. For example, it may analyze the user's past music playback history and recommend music that has been frequently played. The historical music recommendation unit may also predict and recommend music that the user played during a specific time period. For example, it may analyze the user's past music playback history and recommend music that was played during a specific time period. This allows the wireless earphone system to recommend optimal music based on the user's past music playback history.

[0118] The wireless earphone system may further include a voice assistant unit that estimates the user's emotions and adjusts the voice assistant's response based on the estimated emotions. For example, if the user is nervous, the voice assistant unit responds in a calm tone. For example, if the voice assistant unit analyzes the user's voice data and determines that the user is nervous, it responds in a calm tone. The voice assistant unit may also respond in a friendly tone if the user is relaxed. For example, if the voice assistant analyzes the user's voice data and determines that the user is relaxed, it responds in a friendly tone. This allows the wireless earphone system to provide a voice assistant response that corresponds to the user's emotions.

[0119] The wireless earphone system may further include a notification management unit that estimates the user's emotions and determines the priority of notifications based on the estimated emotions. For example, if the user is nervous, the notification management unit may prioritize and display only important notifications. For example, if the user's voice data is analyzed and it is determined that the user is nervous, the notification management unit may display only important notifications. The notification management unit may also display all notifications equally if the user is relaxed. For example, if the user's voice data is analyzed and it is determined that the user is relaxed, the notification management unit may display all notifications equally. This allows the wireless earphone system to prioritize notifications according to the user's emotions.

[0120] The wireless earphone system may further include a feedback unit that estimates the user's emotions and provides audio feedback based on the estimated emotions. For example, the feedback unit may provide encouraging words if the user is tired. For example, the feedback unit may analyze the user's voice data and provide encouraging words if it determines that the user is tired. The feedback unit may also provide words that have a relaxing effect if the user is relaxed. For example, the feedback unit may analyze the user's voice data and provide words that have a relaxing effect if it determines that the user is relaxed. This allows the wireless earphone system to provide audio feedback that corresponds to the user's emotions.

[0121] The wireless earphone system may further include a guide unit that estimates the user's emotions and adjusts the content of the audio guidance based on the estimated emotions. For example, if the user is nervous, the guide unit provides simple and easy-to-understand guidance. For example, if the user's voice data is analyzed and it is determined that the user is nervous, the guide unit provides simple and easy-to-understand guidance. The guide unit may also provide detailed guidance if the user is relaxed. For example, if the user's voice data is analyzed and it is determined that the user is relaxed, the guide unit provides detailed guidance. This allows the wireless earphone system to provide audio guidance that corresponds to the user's emotions.

[0122] The wireless earphone system can further include a location information guide unit that provides optimal audio guidance based on the user's location information. For example, if the user is in a tourist spot, the location information guide unit provides information about the location. For example, if the user's location information is analyzed and it is determined that the user is in a tourist spot, the location information guide unit provides information about the history and tourist attractions of the location. The location information guide unit can also provide information about stores when the user is in a shopping mall. For example, if the user's location information is analyzed and it is determined that the user is in a shopping mall, the location information guide unit provides information about stores. This allows the wireless earphone system to provide optimal audio guidance based on the user's location information.

[0123] The processing flow of the second embodiment will be briefly explained below.

[0124] Step 1: The receiver receives the other person's speech as audio. The receiver, for example, may have a built-in high-performance microphone that accurately captures the other person's speech while filtering out surrounding noise. For example, the receiver can clearly pick up the other person's voice even in noisy places such as cafes or train stations. Step 2: The analysis unit analyzes the voice data received by the receiving unit. The analysis unit converts the voice data into text data, for example, using voice recognition technology. For example, if the other person says "Hello," the analysis unit converts the voice data into text data saying "Hello." Step 3: The translation unit translates the text data analyzed by the analysis unit. The translation unit converts the text data into the user's language using, for example, a multilingual translation model. For example, the translation unit translates the Japanese text data "Hello" into the English text data "Hello." Step 4: The playback unit converts the text data translated by the translation unit into audio data and plays it back. For example, the playback unit converts the text data into audio data using speech synthesis technology. For example, the playback unit converts the text data "Hello" into audio data and plays back the audio "Hello" from the earphones.

[0125] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0126] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of the generative AI include a neural network (NN) and a neural network (NN). The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, image data, etc. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and may perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-mentioned parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. The processing performed by an AI including the generative AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI including the generative AI.

[0127] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0128] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0129] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0130] 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0131] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0132] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0133] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0134] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0135] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0136] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0137] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0138] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0139] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0140] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0141] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0142] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0143] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0144] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0145] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0146] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0147] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0148] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0149] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0150] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0151] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0152] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0153] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0154] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0155] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0156] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0157] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0158] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0159] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0160] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0161] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0162] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0163] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0164] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0165] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0166] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0167] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0168] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0169] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0170] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0171] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0172] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0173] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0174] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0175] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0176] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0177] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0178] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0179] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0180] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0181] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0182] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0183] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0184] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0185] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0186] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0187] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0188] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0189] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0190] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0191] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0192] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0193] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0194] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0195] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0196] [Explanation of symbols]

[0197] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a receiving unit that receives the speech of the other party as audio; an analysis unit that analyzes the voice data received by the receiving unit; a translation unit that translates the text data analyzed by the analysis unit; a playback unit that converts the text data translated by the translation unit into audio data and plays it back; Equipped with A system characterized by:

2. Equipped with a noise canceling unit with noise canceling function The system of claim 1 .

3. A setting section for changing the user's language settings is provided. The system of claim 1 .

4. Equipped with an adjustment section to fine-tune the translation results The system of claim 1 .

5. The receiving unit The built-in microphone accurately captures what the other person is saying while eliminating surrounding noise. The system of claim 1 .

6. The analysis unit Converting voice data into text data using voice recognition technology The system of claim 1 .

7. The translation unit Convert text data into the user's language using a multilingual translation model The system of claim 1 .

8. The playback unit Text data is converted into audio data using speech synthesis technology and played through earphones. The system of claim 1 .

Images