System

Abstract

An object of a system according to an embodiment is to convert a pronunciation of a foreign language into a vernacular style to make the pronunciation easy to hear.SOLUTION: A system includes a collection unit, an analysis unit, a conversion unit, and a provision unit. The collection unit collects pronunciation. The analysis unit analyzes the pronunciation collected by the collection unit. The conversion unit converts the pronunciation analyzed by the analysis unit into a vernacular style. The providing unit provides the pronunciation converted by the conversion unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, it is difficult to understand the pronunciation of other languages ​​in real time, which can be an obstacle to communication.

[0005] The system according to the embodiment aims to convert the pronunciation of a foreign language into one that sounds like the native language, making it easier to understand. [Means for solving the problem]

[0006] The system according to the embodiment includes a collection unit, an analysis unit, a conversion unit, and a provision unit. The collection unit collects pronunciations. The analysis unit analyzes the pronunciations collected by the collection unit. The conversion unit converts the pronunciations analyzed by the analysis unit into pronunciations in the native language. The provision unit provides the pronunciations converted by the conversion unit. [Effects of the Invention]

[0007] The system according to the embodiment can convert pronunciation in a foreign language into a pronunciation similar to one's own language, making it easier to hear. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A pronunciation conversion system according to an embodiment of the present invention converts the pronunciation of a foreign language into a native-like pronunciation. The pronunciation conversion system learns the pronunciation of the language of the conversation partner's country of origin and the user's own country of origin, and converts the conversation partner's foreign language into a native-like pronunciation, making it easier to understand. For example, the pronunciation conversion system collects the conversation partner's pronunciation and analyzes it. Next, based on the analyzed pronunciation, the conversation partner's pronunciation is converted into a native-like pronunciation. The converted pronunciation is provided to the user in real time. This enables the user to understand the pronunciation of the foreign language and facilitates real-time conversation using the natural language. For example, using this system when conversing in real time with a person speaking a foreign language in a business setting can facilitate smooth communication. Furthermore, using this system when conversing with local people while traveling can enable communication without feeling a language barrier. As a result, the pronunciation conversion system makes it easier to understand the pronunciation of the foreign language and facilitates real-time conversation using the natural language. For example, using this system when conversing in real time with a person speaking a foreign language in a business setting can facilitate smooth communication. This system can also be used when talking to local people at travel destinations, allowing you to communicate without feeling the language barrier.

[0029] A pronunciation conversion system according to an embodiment includes a collection unit, an analysis unit, a conversion unit, and a provision unit. The collection unit collects the pronunciation of a dialogue partner. The pronunciation of a dialogue partner includes, but is not limited to, the pronunciation of a word, a sentence, or a specific language. For example, the collection unit records the pronunciation of the dialogue partner and inputs the data into an LLM. The collection unit can also collect the pronunciation of the dialogue partner in real time. For example, the collection unit collects the pronunciation of the dialogue partner using a microphone and inputs the data into the LLM in real time. The analysis unit uses the LLM to analyze the pronunciation collected by the collection unit. The analysis extracts, for example, features such as phonemes, intonation, and rhythm, but is not limited to, the example. For example, the analysis unit extracts phoneme features and identifies which parts differ from the pronunciation in the native language. The analysis unit can also extract intonation and rhythm features. For example, the analysis unit analyzes speech waveforms and extracts intonation and rhythm patterns. The conversion unit converts the pronunciation of the conversation partner into a native-like pronunciation based on the pronunciation characteristics analyzed by the analysis unit. The conversion is performed, for example, based on a phoneme substitution method, an accent adjustment method, or the like, but is not limited to these examples. For example, the conversion unit converts the pronunciation of the conversation partner into a native-like pronunciation using a phoneme substitution method. The conversion unit can also convert the pronunciation of the conversation partner into a native-like pronunciation using an accent adjustment method. The providing unit provides the pronunciation converted by the conversion unit to the user in real time. The providing is performed, for example, based on an audio streaming technology, but is not limited to these examples. For example, the providing unit streams the converted pronunciation to the user in real time. The providing unit can also record the converted pronunciation and play it back later. In this way, the pronunciation conversion system according to the embodiment can convert the pronunciation of a foreign language into a native-like pronunciation, making it easier to understand. For example, using this system when conversing in real time with someone speaking a foreign language in a business setting can facilitate communication. Furthermore, using this system when conversing with local people while traveling can allow communication without feeling a language barrier.

[0030] The pronunciation conversion system includes a learning unit that learns the pronunciation of the country of origin of the conversation partner and the user's own country of origin. The learning unit learns the pronunciation of the country of origin of the conversation partner and the user's own country of origin. The learning is performed, for example, using a machine learning algorithm, but is not limited to this example. For example, the learning unit collects pronunciation data of the country of origin of the conversation partner and pronunciation data of the user's own country of origin and learns using a machine learning algorithm. The learning unit can also perform learning based on the type of dataset. For example, the learning unit performs learning using a pronunciation dataset of a specific language. This makes it possible to more accurately convert the pronunciation of the conversation partner into one that sounds like the user's own language. For example, if the conversation partner speaks English, learning English pronunciation data and converting it into one that sounds like the user's own language makes it easier to understand.

[0031] The pronunciation conversion system includes a real-time providing unit that provides converted pronunciation in real time. The real-time providing unit provides the converted pronunciation in real time. The provision is performed, for example, using audio streaming technology, but is not limited to such an example. For example, the real-time providing unit streams the converted pronunciation to the user in real time. The real-time providing unit can also record the converted pronunciation and play it back later. This allows for smooth conversation by providing the converted pronunciation in real time. For example, using this system when conversing in real time with someone who speaks a different language in a business setting can facilitate communication. Also, using this system when conversing with local people while traveling can allow for communication without feeling a language barrier.

[0032] The collection unit can record the dialogue partner's pronunciation and input the data into the LLM. For example, the collection unit records the dialogue partner's pronunciation and inputs the data into the LLM. For example, the collection unit records the dialogue partner's pronunciation with a microphone and inputs the data into the LLM. The collection unit can also collect the dialogue partner's pronunciation in real time and input it into the LLM. For example, the collection unit collects the dialogue partner's pronunciation in real time and inputs it into the LLM. This allows the dialogue partner's pronunciation to be accurately collected and input into the LLM, improving analysis accuracy. For example, by accurately collecting the dialogue partner's pronunciation, the LLM can analyze the pronunciation characteristics in detail.

[0033] The analysis unit can analyze the pronunciation of the dialogue partner and extract its features. The analysis unit, for example, analyzes the pronunciation of the dialogue partner and extracts its features. Features include, but are not limited to, phonemes, intonation, rhythm, and the like. For example, the analysis unit extracts phoneme features and identifies which parts differ from the pronunciation of the dialogue partner's native language. The analysis unit can also extract intonation and rhythm features. For example, the analysis unit analyzes a speech waveform and extracts intonation and rhythm patterns. This allows for a detailed analysis of the pronunciation features of the dialogue partner, thereby improving conversion accuracy. For example, a detailed analysis of the pronunciation features of the dialogue partner allows the conversion unit to convert the pronunciation into a more natural, native-like pronunciation.

[0034] The conversion unit can convert the pronunciation of the dialogue partner into a native-like pronunciation based on the analyzed pronunciation features. The conversion unit converts the pronunciation of the dialogue partner into a native-like pronunciation based on, for example, the analyzed pronunciation features. The conversion is performed, for example, based on a phoneme substitution method, an accent adjustment method, or the like, but is not limited to these examples. For example, the conversion unit converts the pronunciation of the dialogue partner into a native-like pronunciation using a phoneme substitution method. The conversion unit can also convert the pronunciation of the dialogue partner into a native-like pronunciation using an accent adjustment method. In this way, by performing conversion based on the analyzed pronunciation features, a more natural native-like pronunciation can be obtained. For example, by converting the pronunciation of the dialogue partner into a native-like pronunciation, the user can more easily hear the pronunciation of the dialogue partner.

[0035] The providing unit can provide the converted pronunciation to the user in real time. The providing unit provides the converted pronunciation to the user in real time, for example. The provision is performed using, for example, audio streaming technology, but is not limited to such an example. For example, the providing unit streams the converted pronunciation to the user in real time. The providing unit can also record the converted pronunciation and play it back later. This allows for smooth conversation by providing the converted pronunciation in real time. For example, using this system when conversing in real time with someone who speaks a different language in a business setting can facilitate communication. Furthermore, using this system when conversing with local people while traveling can allow for communication without feeling a language barrier.

[0036] The collection unit can remove background noise from the dialogue partner's pronunciation and collect clear voice data. For example, the collection unit removes background noise from the dialogue partner's pronunciation and collects clear voice data. The removal of background noise can be performed using, for example, noise canceling technology, voice filtering technology, etc., but is not limited to these examples. For example, the collection unit can use noise canceling technology to remove background noise from the dialogue partner's pronunciation. The collection unit can also use voice filtering technology to emphasize sounds in a specific frequency band and collect clear voice data. Furthermore, the collection unit can use multiple microphones to collect the dialogue partner's pronunciation in a directional manner and remove background noise. For example, the collection unit can use multiple microphones to collect the dialogue partner's pronunciation from a specific direction and remove background noise. In this way, by removing background noise, clear voice data can be collected. For example, by clearly collecting the dialogue partner's pronunciation, LLM can analyze the pronunciation characteristics in detail.

[0037] The collection unit can adjust the speech rate and volume of the dialogue partner when collecting pronunciation. For example, the collection unit adjusts the speech rate and volume of the dialogue partner when collecting pronunciation. The adjustment of the speech rate and volume can be performed using, for example, a voice speed conversion algorithm, a volume adjustment method, or the like, but is not limited to these examples. For example, if the dialogue partner's speech rate is fast, the collection unit automatically slows down the speech rate before collection. Also, if the dialogue partner's volume is low, the collection unit can automatically amplify the volume before collection. Furthermore, the collection unit can monitor the speech rate and volume of the dialogue partner in real time and automatically adjust the optimal collection settings. For example, the collection unit monitors the speech rate and volume of the dialogue partner in real time and automatically adjusts the optimal collection settings. In this way, optimal voice data can be collected by adjusting the speech rate and volume. For example, by adjusting the speech rate and volume of the dialogue partner, the LLM can analyze pronunciation characteristics in detail.

[0038] The collection unit can analyze the mouth movements of the conversation partner when collecting the pronunciation and complement the pronunciation features. For example, the collection unit can analyze the mouth movements of the conversation partner when collecting the pronunciation and complement the pronunciation features. The analysis of mouth movements can be performed using, for example, video analysis technology, action recognition algorithms, etc., but is not limited to these examples. For example, the collection unit can analyze the mouth movements of the conversation partner using a camera and complement the pronunciation features. The collection unit can also synchronize the mouth movement data with the voice data to collect more accurate pronunciation data. Furthermore, the collection unit can learn mouth movement patterns and apply an algorithm to complement the pronunciation features. For example, the collection unit can learn mouth movement patterns and apply an algorithm to complement the pronunciation features. In this way, by analyzing the mouth movements, the pronunciation features can be complemented and more accurate voice data can be collected. For example, by analyzing the mouth movements of the conversation partner, the LLM can analyze the pronunciation features in detail.

[0039] When collecting pronunciations, the collection unit can prioritize collecting highly relevant pronunciations by taking into account the geographical location information of the conversation partner. For example, when collecting pronunciations, the collection unit prioritizes collecting highly relevant pronunciations by taking into account the geographical location information of the conversation partner. Consideration of geographical location information is performed using technologies such as, but not limited to, GPS data and location estimation from IP addresses. For example, if the conversation partner is in a specific region, the collection unit prioritizes collecting pronunciations specific to that region. The collection unit can also prioritize collecting related dialects and accents based on the geographical location information. Furthermore, the collection unit can update the location information of the conversation partner in real time to collect optimal pronunciations. For example, the collection unit updates the location information of the conversation partner in real time to collect optimal pronunciations. In this way, by taking geographical location information into account, highly relevant pronunciations can be prioritized. For example, if the conversation partner is in a specific region, priority is given to collecting pronunciations specific to that region, allowing the LLM to analyze pronunciation characteristics in detail.

[0040] The collection unit can analyze the social media activities of the conversation partner when collecting pronunciations and collect related pronunciations. For example, the collection unit analyzes the social media activities of the conversation partner when collecting pronunciations and collects related pronunciations. The analysis of social media activities is performed using techniques such as, but not limited to, analysis of post content and analysis of followers. For example, the collection unit analyzes the content of posts on social media of the conversation partner and collects related pronunciations. The collection unit can also collect related pronunciations based on the conversation partner's check-in information on social media. Furthermore, the collection unit can collect related pronunciations by referring to the activities of the conversation partner's friends on social media. For example, the collection unit collects related pronunciations by referring to the activities of the conversation partner's friends on social media. In this way, related pronunciations can be collected by analyzing social media activities. For example, by analyzing the content of posts on social media of the conversation partner, the LLM can analyze pronunciation characteristics in detail.

[0041] The collection unit can customize the collection method by reflecting the dialogue partner's past feedback when collecting pronunciation. For example, the collection unit customizes the collection method by reflecting the dialogue partner's past feedback when collecting pronunciation. Reflecting past feedback can be performed using data such as, for example, user reviews and survey results, but is not limited to these examples. For example, the collection unit customizes the collection method based on feedback provided by the dialogue partner in the past. The collection unit can also preferentially collect specific pronunciations from the dialogue partner's past feedback. Furthermore, the collection unit can reflect the dialogue partner's feedback in real time and adjust the collection method. For example, the collection unit reflects the dialogue partner's feedback in real time and adjusts the collection method. In this way, by reflecting past feedback, the collection method can be customized and more appropriate pronunciation data can be collected. For example, customizing the collection method based on the dialogue partner's past feedback allows the LLM to analyze pronunciation characteristics in detail.

[0042] The analysis unit can apply different analysis algorithms to each phoneme of the pronunciation during analysis. For example, the analysis unit applies different analysis algorithms to each phoneme of the pronunciation during analysis. Application of different analysis algorithms to each phoneme includes, but is not limited to, using different analysis algorithms for vowels and consonants. For example, the analysis unit applies different analysis algorithms to vowels and consonants to analyze pronunciation features in detail. The analysis unit can also apply dedicated analysis algorithms to specific phonemes to improve accuracy. Furthermore, the analysis unit can combine different analysis algorithms for each phoneme to provide a comprehensive analysis result. For example, the analysis unit combines different analysis algorithms for each phoneme to provide a comprehensive analysis result. In this way, by applying different analysis algorithms to each phoneme, pronunciation features can be analyzed in detail. For example, by applying different analysis algorithms to vowels and consonants, LLM can analyze pronunciation features in detail.

[0043] The analysis unit can analyze the intonation and rhythm of the dialogue partner's pronunciation during analysis. For example, the analysis unit analyzes the intonation and rhythm of the dialogue partner's pronunciation during analysis. The analysis of intonation and rhythm is performed using, for example, techniques such as speech waveform analysis and rhythm pattern extraction, but is not limited to these examples. For example, the analysis unit analyzes the intonation of the dialogue partner's pronunciation and extracts characteristics. The analysis unit can also analyze the rhythm of the dialogue partner's pronunciation and extract characteristics. Furthermore, the analysis unit can combine the results of the intonation and rhythm analysis to extract comprehensive pronunciation characteristics. For example, the analysis unit combines the results of the intonation and rhythm analysis to extract comprehensive pronunciation characteristics. In this way, by analyzing the intonation and rhythm, pronunciation characteristics can be grasped in detail. For example, by analyzing the intonation and rhythm of the dialogue partner's pronunciation, LLM can analyze pronunciation characteristics in detail.

[0044] The analysis unit can identify the accent of the dialogue partner's pronunciation during analysis and reflect it in the analysis results. For example, the analysis unit can identify the accent of the dialogue partner's pronunciation during analysis and reflect it in the analysis results. The identification of the accent is performed using techniques such as, for example, voice stress patterns and regional pronunciation characteristics, but is not limited to these examples. For example, the analysis unit can identify the accent of the dialogue partner's pronunciation and reflect it in the analysis results. The analysis unit can also analyze the position and strength of the accent and extract features. Furthermore, the analysis unit can analyze the pronunciation features in detail based on the accent analysis results. For example, the analysis unit analyzes the position and strength of the accent and extracts features. By identifying the accent, the pronunciation features can be analyzed in detail. For example, by identifying the accent of the dialogue partner's pronunciation, the LLM can analyze the pronunciation features in detail.

[0045] The analysis unit can determine the analysis priority based on the time of submission of the pronunciation during analysis. For example, the analysis unit determines the analysis priority based on the time of submission of the pronunciation during analysis. The submission time is taken into consideration using, for example, a timestamp, a method for recording the submission date and time, etc., but is not limited to such examples. For example, the analysis unit prioritizes analysis of the most recent pronunciation data. The analysis unit can also analyze pronunciation data that was submitted earlier later. Furthermore, the analysis unit can dynamically adjust the analysis priority based on the submission time. For example, the analysis unit dynamically adjusts the analysis priority based on the submission time. As a result, by determining the analysis priority based on the submission time, the most recent pronunciation data can be analyzed preferentially. For example, by prioritizing analysis of the most recent pronunciation data, the LLM can analyze the pronunciation features in detail.

[0046] The analysis unit can adjust the order of analysis based on the relevance of pronunciation during analysis. For example, the analysis unit adjusts the order of analysis based on the relevance of pronunciation during analysis. The evaluation of relevance is performed using, for example, techniques such as content similarity and a method for calculating a relevance score, but is not limited to these examples. For example, the analysis unit prioritizes analysis of highly relevant pronunciation data. The analysis unit can also postpone analysis of less relevant pronunciation data. Furthermore, the analysis unit can dynamically adjust the order of analysis based on the relevance of pronunciation. For example, the analysis unit dynamically adjusts the order of analysis based on the relevance of pronunciation. As a result, by adjusting the order of analysis based on the relevance of pronunciation, highly relevant pronunciation data can be prioritized for analysis. For example, by prioritizing analysis of highly relevant pronunciation data, the LLM can analyze pronunciation features in detail.

[0047] The analysis unit can adjust the use of technical terminology during analysis according to the expertise level of the interlocutor. For example, the analysis unit adjusts the use of technical terminology during analysis according to the expertise level of the interlocutor. The expertise level is evaluated using data such as, for example, qualification information and past utterance content, but is not limited to such examples. For example, the analysis unit can provide analysis results using detailed technical terminology to interlocutors with a high level of expertise. The analysis unit can also provide analysis results using concise and easy-to-understand terms to interlocutors with a low level of expertise. Furthermore, the analysis unit can dynamically adjust the way the analysis results are presented according to the expertise level. For example, the analysis unit dynamically adjusts the way the analysis results are presented according to the expertise level. This allows the use of technical terminology during analysis to be adjusted according to the expertise level, thereby providing analysis results appropriate for the interlocutor. For example, by providing analysis results using detailed technical terminology to interlocutors with a high level of expertise, the LLM can analyze pronunciation characteristics in detail.

[0048] The conversion unit can apply different conversion algorithms to each phoneme of the pronunciation during conversion. For example, the conversion unit applies different conversion algorithms to each phoneme of the pronunciation during conversion. Application of different conversion algorithms to each phoneme includes, but is not limited to, using different conversion algorithms for vowels and consonants. For example, the conversion unit applies different conversion algorithms to vowels and consonants to convert pronunciation features in detail. The conversion unit can also apply a dedicated conversion algorithm to a specific phoneme to improve accuracy. Furthermore, the conversion unit can combine different conversion algorithms for each phoneme to provide an overall conversion result. For example, the conversion unit combines different conversion algorithms for each phoneme to provide an overall conversion result. In this way, by applying different conversion algorithms to each phoneme, pronunciation features can be converted in detail. For example, by applying different conversion algorithms to vowels and consonants, LLM can convert pronunciation features in detail.

[0049] The conversion unit can take into account the intonation and rhythm of the dialogue partner's pronunciation during conversion. For example, the conversion unit takes into account the intonation and rhythm of the dialogue partner's pronunciation during conversion. Consideration of intonation and rhythm can be performed using, for example, techniques such as speech waveform analysis and rhythm pattern extraction, but is not limited to these examples. For example, the conversion unit takes into account the intonation of the dialogue partner's pronunciation during conversion. The conversion unit can also take into account the rhythm of the dialogue partner's pronunciation during conversion. Furthermore, the conversion unit can combine intonation and rhythm features to perform conversion that is closer to natural pronunciation. For example, the conversion unit combines intonation and rhythm features to perform conversion that is closer to natural pronunciation. By taking into account intonation and rhythm, conversion that is closer to more natural pronunciation is possible. For example, by taking into account the intonation and rhythm of the dialogue partner's pronunciation, the LLM can convert pronunciation features in detail.

[0050] The conversion unit can reflect the accent of the dialogue partner's pronunciation during conversion. For example, the conversion unit reflects the accent of the dialogue partner during conversion. Reflecting the accent can be performed using, for example, techniques such as voice stress patterns and regional pronunciation characteristics, but is not limited to these examples. For example, the conversion unit reflects the accent of the dialogue partner during conversion. The conversion unit can also take into account the position and strength of the accent during conversion. Furthermore, the conversion unit can perform conversion that is closer to natural pronunciation based on the accent characteristics. For example, the conversion unit performs conversion that is closer to natural pronunciation based on the accent characteristics. By reflecting the accent, conversion that is closer to more natural pronunciation is possible. For example, by reflecting the accent of the dialogue partner, the LLM can perform detailed conversion of pronunciation characteristics.

[0051] The conversion unit can perform the conversion while taking into account the geographical distribution of pronunciation. For example, the conversion unit performs the conversion while taking into account the geographical distribution of pronunciation. Consideration of the geographical distribution can be performed using, for example, techniques such as regional pronunciation characteristics and geographical data acquisition methods, but is not limited to these examples. For example, if the pronunciation of the dialogue partner is unique to a particular region, the conversion unit performs the conversion while taking into account the pronunciation characteristics of that region. The conversion unit can also perform the conversion while reflecting associated dialects and accents based on the geographical distribution. Furthermore, the conversion unit can apply an optimal conversion algorithm based on the geographical distribution data. For example, the conversion unit applies an optimal conversion algorithm based on the geographical distribution data. This allows for conversion that reflects regional pronunciation by taking into account the geographical distribution. For example, by taking into account the geographical distribution of the dialogue partner's pronunciation, the LLM can perform detailed conversion of pronunciation characteristics.

[0052] The conversion unit can improve the accuracy of the conversion by referring to literature related to pronunciation during conversion. For example, the conversion unit improves the accuracy of the conversion by referring to literature related to pronunciation during conversion. Referencing the related literature can be performed using, for example, techniques such as literature selection criteria and reference methods, but is not limited to these examples. For example, the conversion unit can refer to literature related to pronunciation and optimize the conversion algorithm. The conversion unit can also analyze pronunciation features in detail based on data from the related literature to improve the accuracy of the conversion. Furthermore, the conversion unit can refer to information from the related literature in real time and reflect the analysis in the conversion results. For example, the conversion unit can refer to information from the related literature in real time and reflect the analysis in the conversion results. As a result, the accuracy of the conversion is improved by referring to the related literature. For example, by referring to literature related to pronunciation, the LLM can convert pronunciation features in detail.

[0053] The conversion unit can perform the conversion while taking into account the market value of the pronunciation. For example, the conversion unit performs the conversion while taking into account the market value of the pronunciation. Consideration of market value can be performed using, for example, market research data, value assessment standards, and other technologies, but is not limited to these examples. For example, the conversion unit can preferentially convert pronunciations with high market value. The conversion unit can also adjust the pronunciation conversion algorithm based on the market value. Furthermore, the conversion unit can select the optimal conversion method based on the market value data. For example, the conversion unit selects the optimal conversion method based on the market value data. In this way, by taking market value into consideration, pronunciations with high value can be preferentially converted. For example, by preferentially converting pronunciations with high market value, the LLM can convert the pronunciation features in detail.

[0054] The providing unit can select the optimal delivery method by referring to the user's past operation history at the time of delivery. For example, the providing unit selects the optimal delivery method by referring to the user's past operation history at the time of delivery. Referencing the operation history is performed using, for example, a technology such as a history data acquisition method or a reference method, but is not limited to such examples. For example, the providing unit preferentially selects a delivery method that the user has used favorably in the past. The providing unit can also suggest the optimal audio delivery method based on the user's past operation history. Furthermore, the providing unit can refer to the user's operation history in real time and dynamically adjust the delivery method. For example, the providing unit refers to the user's operation history in real time and dynamically adjusts the delivery method. In this way, the optimal delivery method for the user can be selected by referring to the past operation history. For example, by preferentially selecting a delivery method that the user has used favorably in the past, the LLM can convert pronunciation features in detail.

[0055] The providing unit can customize the provided content according to the user's current task at the time of providing. For example, the providing unit customizes the provided content according to the user's current task at the time of providing. Task detection is performed using, for example, techniques such as user behavioral data and device usage status, but is not limited to such examples. For example, the providing unit provides the content in a quiet voice when the user is in a meeting. Furthermore, the providing unit can provide the content in a concise and easy-to-understand voice when the user is driving. Furthermore, the providing unit can detect the user's current task in real time and dynamically customize the provided content. For example, the providing unit detects the user's current task in real time and dynamically customizes the provided content. This enables more appropriate provision by customizing the provided content according to the current task. For example, when the user is in a meeting, providing the content in a quiet voice allows the LLM to convert pronunciation characteristics in detail.

[0056] The providing unit can select the optimal presentation method by taking into account the user's device information at the time of presentation. For example, the providing unit selects the optimal presentation method by taking into account the user's device information at the time of presentation. Consideration of device information is performed using, for example, technology such as device type and specification information, but is not limited to such examples. For example, if the user is using a smartphone, the providing unit selects a presentation method that matches the screen size. Also, if the user is using a tablet, the providing unit can select a presentation method optimized for a large screen. Furthermore, if the user is using a smartwatch, the providing unit can select a presentation method that is concise and highly visible. For example, if the user is using a smartwatch, the providing unit selects a presentation method that is concise and highly visible. In this way, the optimal presentation method for the user can be selected by taking into account the device information. For example, if the user is using a smartphone, selecting a presentation method that matches the screen size allows the LLM to convert pronunciation features in detail.

[0057] The providing unit can make the provided content multilingual based on the user's language setting at the time of provision. For example, the providing unit can make the provided content multilingual based on the user's language setting at the time of provision. Consideration of the language setting is performed using, for example, technology such as the user's language setting information and language selection criteria, but is not limited to such examples. For example, the providing unit automatically sets the provided content based on the language setting of the user's device. The providing unit can also provide a language switching function when the user uses multiple languages. Furthermore, the providing unit can provide the provided content in a specific language when the user selects that language. For example, when the user selects a specific language, the providing unit provides the provided content in that language. By making the provided content multilingual based on the language setting, the provided content can be provided in the language that is most suitable for the user. For example, by automatically setting the provided content based on the language setting of the user's device, the LLM can convert pronunciation characteristics in detail.

[0058] The learning unit can optimize the learning algorithm by referring to past learning data during learning. For example, the learning unit optimizes the learning algorithm by referring to past learning data during learning. Optimization of the learning algorithm is performed using, for example, techniques such as algorithm selection criteria and optimization methods, but is not limited to these examples. For example, the learning unit optimizes the learning algorithm based on past learning data. The learning unit can also analyze trends in the past learning data and select an optimal learning algorithm. Furthermore, the learning unit can dynamically adjust the learning algorithm by referring to past learning data in real time. For example, the learning unit dynamically adjusts the learning algorithm by referring to past learning data. This optimizes the learning algorithm by referring to past learning data, improving learning accuracy. For example, optimizing the learning algorithm based on past learning data allows the LLM to convert pronunciation features in detail.

[0059] The learning unit can update the learning data by reflecting user feedback during learning. For example, the learning unit updates the learning data by reflecting user feedback during learning. Reflecting feedback is performed using, for example, techniques such as the type of feedback and the reflection method, but is not limited to these examples. For example, the learning unit updates the learning data based on user feedback. The learning unit can also reflect user feedback in real time and dynamically update the learning data. Furthermore, the learning unit can analyze user feedback and select optimal learning data. For example, the learning unit analyzes user feedback and selects optimal learning data. In this way, the learning data is updated by reflecting user feedback, improving learning accuracy. For example, by updating the learning data based on user feedback, the LLM can convert pronunciation features in detail.

[0060] The learning unit can weight the learning data based on the time of submission of the pronunciation during learning. For example, the learning unit weights the learning data based on the time of submission of the pronunciation during learning. The weighting method is performed using, for example, techniques such as a data importance evaluation criterion and a weighting algorithm, but is not limited to these examples. For example, the learning unit weights the most recent pronunciation data during learning. The learning unit can also weight pronunciation data that was submitted earlier during learning. Furthermore, the learning unit can dynamically adjust the weighting of the learning data based on the time of submission. For example, the learning unit dynamically adjusts the weighting of the learning data based on the time of submission. As a result, by weighting the learning data based on the time of submission, learning can be performed with an emphasis on the most recent pronunciation data. For example, by weighting the most recent pronunciation data during learning, the LLM can convert pronunciation features in detail.

[0061] The learning unit can integrate information from different data sources during learning to enrich the training data. For example, the learning unit integrates information from different data sources during learning to enrich the training data. Data source integration can be performed using, for example, techniques based on the type of data source, integration method, etc., but is not limited to these examples. For example, the learning unit integrates pronunciation data from different data sources to perform learning. The learning unit can also enrich the training data based on information from different data sources. Furthermore, the learning unit can integrate data from different data sources in real time and dynamically update the training data. For example, the learning unit integrates data from different data sources in real time and dynamically update the training data. In this way, by integrating information from different data sources, the training data is enriched and the accuracy of learning is improved. For example, by integrating pronunciation data from different data sources and performing learning, the LLM can convert pronunciation features in detail.

[0062] The real-time providing unit can select the optimal delivery method by referring to the user's past operation history during real-time delivery. For example, the real-time providing unit selects the optimal delivery method by referring to the user's past operation history during real-time delivery. Referencing the operation history is performed using, for example, a technology such as a history data acquisition method or a reference method, but is not limited to such examples. For example, the real-time providing unit preferentially selects a delivery method that the user has used favorably in the past. The real-time providing unit can also suggest the optimal real-time delivery method based on the user's past operation history. Furthermore, the real-time providing unit can refer to the user's operation history in real time and dynamically adjust the delivery method. For example, the real-time providing unit refers to the user's operation history in real time and dynamically adjusts the delivery method. In this way, the optimal real-time delivery method for the user can be selected by referring to the past operation history. For example, by preferentially selecting a delivery method that the user has used favorably in the past, the LLM can convert pronunciation features in detail.

[0063] The real-time providing unit can customize the content to be provided in accordance with the user's current task during real-time provision. For example, the real-time providing unit customizes the content to be provided in accordance with the user's current task during real-time provision. Task detection is performed using, for example, techniques such as user behavioral data and device usage status, but is not limited to these examples. For example, if the user is in a meeting, the real-time providing unit provides real-time content in a quiet voice. Furthermore, if the user is driving, the real-time providing unit can provide real-time content in a concise and easy-to-understand voice. Furthermore, the real-time providing unit can detect the user's current task in real time and dynamically customize the content to be provided. For example, the real-time providing unit detects the user's current task in real time and dynamically customizes the content to be provided. This enables more appropriate content to be provided by customizing the content to the current task. For example, if the user is in a meeting, providing real-time content in a quiet voice allows the LLM to convert pronunciation characteristics in detail.

[0064] The real-time providing unit can select the optimal delivery method by taking into account the user's device information during real-time delivery. For example, the real-time providing unit selects the optimal delivery method by taking into account the user's device information during real-time delivery. Consideration of device information is performed using, for example, technology such as device type and specification information, but is not limited to such examples. For example, if the user is using a smartphone, the real-time providing unit selects a real-time delivery method that matches the screen size. Furthermore, if the user is using a tablet, the real-time providing unit can select a real-time delivery method that is optimized for a large screen. Furthermore, if the user is using a smartwatch, the real-time providing unit can select a real-time delivery method that is concise and highly visible. For example, if the user is using a smartwatch, the real-time providing unit selects a real-time delivery method that is concise and highly visible. In this way, the optimal real-time delivery method for the user can be selected by taking into account the device information. For example, if the user is using a smartphone, selecting a real-time delivery method that matches the screen size allows the LLM to convert pronunciation features in detail.

[0065] The real-time providing unit can provide multilingual content in accordance with the user's language setting during real-time provision. For example, the real-time providing unit provides multilingual content in accordance with the user's language setting during real-time provision. Consideration of the language setting is performed using, for example, technology such as the user's language setting information and language selection criteria, but is not limited to such examples. For example, the real-time providing unit automatically sets the real-time content based on the language setting of the user's device. The real-time providing unit can also provide a language switching function when the user uses multiple languages. Furthermore, the real-time providing unit can provide real-time content in a specific language when the user selects that language. For example, the real-time providing unit provides real-time content in that language when the user selects a specific language. This allows the content to be provided in multiple languages ​​according to the language setting, thereby providing the content in the language most suitable for the user. For example, automatically setting the real-time content based on the language setting of the user's device allows the LLM to convert pronunciation characteristics in detail.

[0066] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0067] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can filter the conversation partner's background sounds in real time and collect clear voice data. For example, the collection unit can use noise canceling technology to remove background sounds from the conversation partner's pronunciation. The collection unit can also emphasize sounds in specific frequency bands to collect the conversation partner's pronunciation more clearly. Furthermore, the collection unit can use multiple microphones to collect the conversation partner's pronunciation in a directional manner and effectively remove background sounds. In this way, by removing background sounds, clear voice data can be collected and analysis accuracy can be improved.

[0068] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can adjust the conversation partner's speaking speed and volume in real time to collect optimal voice data. For example, if the conversation partner's speaking speed is fast, the collection unit can automatically slow down the speech to collect the data. Also, if the conversation partner's volume is low, the collection unit can automatically amplify the volume to collect the data. Furthermore, the collection unit can monitor the conversation partner's speaking speed and volume in real time and automatically adjust the optimal collection settings. In this way, by adjusting the speaking speed and volume, optimal voice data can be collected and analysis accuracy can be improved.

[0069] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can analyze the conversation partner's mouth movements and complement the pronunciation features. For example, the collection unit can use a camera to analyze the conversation partner's mouth movements and complement the pronunciation features. The collection unit can also synchronize the mouth movement data with the voice data to collect more accurate pronunciation data. Furthermore, the collection unit can learn mouth movement patterns and apply an algorithm to complement the pronunciation features. In this way, by analyzing the mouth movements, pronunciation features can be complemented and more accurate voice data can be collected.

[0070] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can prioritize collecting highly relevant pronunciations by taking into account the conversation partner's geographical location information. For example, if the conversation partner is in a specific region, the collection unit can prioritize collecting pronunciations specific to that region. The collection unit can also prioritize collecting related dialects and accents based on the geographical location information. Furthermore, the collection unit can update the conversation partner's location information in real time and collect optimal pronunciations. In this way, by taking geographical location information into account, highly relevant pronunciations can be prioritized and analysis accuracy can be improved.

[0071] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can analyze the conversation partner's social media activities and collect related pronunciations. For example, the collection unit can analyze the conversation partner's social media posts and collect related pronunciations. The collection unit can also collect related pronunciations based on the conversation partner's social media check-in information. Furthermore, the collection unit can collect related pronunciations by referring to the activities of the conversation partner's friends on social media. In this way, by analyzing social media activities, related pronunciations can be collected and analysis accuracy can be improved.

[0072] When collecting pronunciation from a dialogue partner, the pronunciation conversion system can customize the collection method by reflecting the dialogue partner's past feedback. For example, the collection unit can customize the collection method based on feedback provided by the dialogue partner in the past. The collection unit can also preferentially collect specific pronunciations from the dialogue partner's past feedback. Furthermore, the collection unit can reflect the dialogue partner's feedback in real time and adjust the collection method. In this way, by reflecting past feedback, the collection method can be customized and more appropriate pronunciation data can be collected.

[0073] The processing flow of the first embodiment will be briefly explained below.

[0074] Step 1: The collection unit collects the pronunciation of the conversation partner. The conversation partner's pronunciation includes the pronunciation of words, sentences, and specific languages. The collection unit records the conversation partner's pronunciation and inputs the data into the LLM. The collection unit can also collect the conversation partner's pronunciation in real time. For example, the collection unit collects the conversation partner's pronunciation using a microphone and inputs the data into the LLM in real time. Step 2: The analyzer uses LLM to analyze the pronunciation collected by the collector. The analysis extracts features such as phonemes, intonation, and rhythm. For example, the analyzer extracts phoneme features and identifies which parts differ from the pronunciation in the native language. The analyzer also analyzes the speech waveform and extracts intonation and rhythm patterns. Step 3: The conversion unit converts the pronunciation of the interlocutor into one that sounds like the interlocutor's own language based on the pronunciation characteristics analyzed by the analysis unit. The conversion is performed based on a phoneme substitution method, an accent adjustment method, or the like. For example, the conversion unit converts the pronunciation of the interlocutor into one that sounds like the interlocutor's own language using a phoneme substitution method. The conversion unit can also convert the pronunciation of the interlocutor into one that sounds like the interlocutor's own language using an accent adjustment method. Step 4: The providing unit provides the pronunciation converted by the converting unit to the user in real time. The providing is performed using audio streaming technology. For example, the providing unit streams the converted pronunciation to the user in real time. The providing unit can also record the converted pronunciation and play it back later.

[0075] (Example 2) A pronunciation conversion system according to an embodiment of the present invention converts the pronunciation of a foreign language into a native-like pronunciation. The pronunciation conversion system learns the pronunciation of the language of the conversation partner's country of origin and the user's own country of origin, and converts the conversation partner's foreign language into a native-like pronunciation, making it easier to understand. For example, the pronunciation conversion system collects the conversation partner's pronunciation and analyzes it. Next, based on the analyzed pronunciation, the conversation partner's pronunciation is converted into a native-like pronunciation. The converted pronunciation is provided to the user in real time. This enables the user to understand the pronunciation of the foreign language and facilitates real-time conversation using the natural language. For example, using this system when conversing in real time with a person speaking a foreign language in a business setting can facilitate smooth communication. Furthermore, using this system when conversing with local people while traveling can enable communication without feeling a language barrier. As a result, the pronunciation conversion system makes it easier to understand the pronunciation of the foreign language and facilitates real-time conversation using the natural language. For example, using this system when conversing in real time with a person speaking a foreign language in a business setting can facilitate smooth communication. This system can also be used when talking to local people at travel destinations, allowing you to communicate without feeling the language barrier.

[0076] A pronunciation conversion system according to an embodiment includes a collection unit, an analysis unit, a conversion unit, and a provision unit. The collection unit collects the pronunciation of a dialogue partner. The pronunciation of a dialogue partner includes, but is not limited to, the pronunciation of a word, a sentence, or a specific language. For example, the collection unit records the pronunciation of the dialogue partner and inputs the data into an LLM. The collection unit can also collect the pronunciation of the dialogue partner in real time. For example, the collection unit collects the pronunciation of the dialogue partner using a microphone and inputs the data into the LLM in real time. The analysis unit uses the LLM to analyze the pronunciation collected by the collection unit. The analysis extracts, for example, features such as phonemes, intonation, and rhythm, but is not limited to, the example. For example, the analysis unit extracts phoneme features and identifies which parts differ from the pronunciation in the native language. The analysis unit can also extract intonation and rhythm features. For example, the analysis unit analyzes speech waveforms and extracts intonation and rhythm patterns. The conversion unit converts the pronunciation of the conversation partner into a native-like pronunciation based on the pronunciation characteristics analyzed by the analysis unit. The conversion is performed, for example, based on a phoneme substitution method, an accent adjustment method, or the like, but is not limited to these examples. For example, the conversion unit converts the pronunciation of the conversation partner into a native-like pronunciation using a phoneme substitution method. The conversion unit can also convert the pronunciation of the conversation partner into a native-like pronunciation using an accent adjustment method. The providing unit provides the pronunciation converted by the conversion unit to the user in real time. The providing is performed, for example, based on an audio streaming technology, but is not limited to these examples. For example, the providing unit streams the converted pronunciation to the user in real time. The providing unit can also record the converted pronunciation and play it back later. In this way, the pronunciation conversion system according to the embodiment can convert the pronunciation of a foreign language into a native-like pronunciation, making it easier to understand. For example, using this system when conversing in real time with someone speaking a foreign language in a business setting can facilitate communication. Furthermore, using this system when conversing with local people while traveling can allow communication without feeling a language barrier.

[0077] The pronunciation conversion system includes a learning unit that learns the pronunciation of the country of origin of the conversation partner and the user's own country of origin. The learning unit learns the pronunciation of the country of origin of the conversation partner and the user's own country of origin. The learning is performed, for example, using a machine learning algorithm, but is not limited to this example. For example, the learning unit collects pronunciation data of the country of origin of the conversation partner and pronunciation data of the user's own country of origin and learns using a machine learning algorithm. The learning unit can also perform learning based on the type of dataset. For example, the learning unit performs learning using a pronunciation dataset of a specific language. This makes it possible to more accurately convert the pronunciation of the conversation partner into one that sounds like the user's own language. For example, if the conversation partner speaks English, learning English pronunciation data and converting it into one that sounds like the user's own language makes it easier to understand.

[0078] The pronunciation conversion system includes a real-time providing unit that provides converted pronunciation in real time. The real-time providing unit provides the converted pronunciation in real time. The provision is performed, for example, using audio streaming technology, but is not limited to such an example. For example, the real-time providing unit streams the converted pronunciation to the user in real time. The real-time providing unit can also record the converted pronunciation and play it back later. This allows for smooth conversation by providing the converted pronunciation in real time. For example, using this system when conversing in real time with someone who speaks a different language in a business setting can facilitate communication. Also, using this system when conversing with local people while traveling can allow for communication without feeling a language barrier.

[0079] The collection unit can record the dialogue partner's pronunciation and input the data into the LLM. For example, the collection unit records the dialogue partner's pronunciation and inputs the data into the LLM. For example, the collection unit records the dialogue partner's pronunciation with a microphone and inputs the data into the LLM. The collection unit can also collect the dialogue partner's pronunciation in real time and input it into the LLM. For example, the collection unit collects the dialogue partner's pronunciation in real time and inputs it into the LLM. This allows the dialogue partner's pronunciation to be accurately collected and input into the LLM, improving analysis accuracy. For example, by accurately collecting the dialogue partner's pronunciation, the LLM can analyze the pronunciation characteristics in detail.

[0080] The analysis unit can analyze the pronunciation of the dialogue partner and extract its features. The analysis unit, for example, analyzes the pronunciation of the dialogue partner and extracts its features. Features include, but are not limited to, phonemes, intonation, rhythm, and the like. For example, the analysis unit extracts phoneme features and identifies which parts differ from the pronunciation of the dialogue partner's native language. The analysis unit can also extract intonation and rhythm features. For example, the analysis unit analyzes a speech waveform and extracts intonation and rhythm patterns. This allows for a detailed analysis of the pronunciation features of the dialogue partner, thereby improving conversion accuracy. For example, a detailed analysis of the pronunciation features of the dialogue partner allows the conversion unit to convert the pronunciation into a more natural, native-like pronunciation.

[0081] The conversion unit can convert the pronunciation of the dialogue partner into a native-like pronunciation based on the analyzed pronunciation features. The conversion unit converts the pronunciation of the dialogue partner into a native-like pronunciation based on, for example, the analyzed pronunciation features. The conversion is performed, for example, based on a phoneme substitution method, an accent adjustment method, or the like, but is not limited to these examples. For example, the conversion unit converts the pronunciation of the dialogue partner into a native-like pronunciation using a phoneme substitution method. The conversion unit can also convert the pronunciation of the dialogue partner into a native-like pronunciation using an accent adjustment method. In this way, by performing conversion based on the analyzed pronunciation features, a more natural native-like pronunciation can be obtained. For example, by converting the pronunciation of the dialogue partner into a native-like pronunciation, the user can more easily hear the pronunciation of the dialogue partner.

[0082] The providing unit can provide the converted pronunciation to the user in real time. The providing unit provides the converted pronunciation to the user in real time, for example. The provision is performed using, for example, audio streaming technology, but is not limited to such an example. For example, the providing unit streams the converted pronunciation to the user in real time. The providing unit can also record the converted pronunciation and play it back later. This allows for smooth conversation by providing the converted pronunciation in real time. For example, using this system when conversing in real time with someone who speaks a different language in a business setting can facilitate communication. Furthermore, using this system when conversing with local people while traveling can allow for communication without feeling a language barrier.

[0083] The collection unit can estimate the user's emotions and adjust the timing of pronunciation collection based on the estimated user's emotions. The collection unit, for example, estimates the user's emotions and adjusts the timing of pronunciation collection based on the estimated user's emotions. Emotion estimation is performed using, for example, technologies such as voice analysis, facial expression recognition, and text analysis, but is not limited to these examples. For example, the collection unit analyzes the user's voice data to estimate the emotions. The collection unit can also analyze the user's facial expression data to estimate the emotions. For example, the collection unit analyzes changes in the user's facial expressions to estimate the emotions. Furthermore, the collection unit can analyze the user's text data to estimate the emotions. For example, the collection unit analyzes the user's text messages to estimate the emotions. In this way, by adjusting the collection timing according to the user's emotions, more natural pronunciation data can be collected. For example, when the user is relaxed, pronunciation can be collected in accordance with the natural flow of conversation. Furthermore, when the user is nervous, pronunciation can be collected with appropriate pauses between conversations. Furthermore, when the user is in a hurry, pronunciation can be collected for each short phrase and processed quickly.

[0084] The collection unit can remove background noise from the dialogue partner's pronunciation and collect clear voice data. For example, the collection unit removes background noise from the dialogue partner's pronunciation and collects clear voice data. The removal of background noise can be performed using, for example, noise canceling technology, voice filtering technology, etc., but is not limited to these examples. For example, the collection unit can use noise canceling technology to remove background noise from the dialogue partner's pronunciation. The collection unit can also use voice filtering technology to emphasize sounds in a specific frequency band and collect clear voice data. Furthermore, the collection unit can use multiple microphones to collect the dialogue partner's pronunciation in a directional manner and remove background noise. For example, the collection unit can use multiple microphones to collect the dialogue partner's pronunciation from a specific direction and remove background noise. In this way, by removing background noise, clear voice data can be collected. For example, by clearly collecting the dialogue partner's pronunciation, LLM can analyze the pronunciation characteristics in detail.

[0085] The collection unit can adjust the speech rate and volume of the dialogue partner when collecting pronunciation. For example, the collection unit adjusts the speech rate and volume of the dialogue partner when collecting pronunciation. The adjustment of the speech rate and volume can be performed using, for example, a voice speed conversion algorithm, a volume adjustment method, or the like, but is not limited to these examples. For example, if the dialogue partner's speech rate is fast, the collection unit automatically slows down the speech rate before collection. Also, if the dialogue partner's volume is low, the collection unit can automatically amplify the volume before collection. Furthermore, the collection unit can monitor the speech rate and volume of the dialogue partner in real time and automatically adjust the optimal collection settings. For example, the collection unit monitors the speech rate and volume of the dialogue partner in real time and automatically adjusts the optimal collection settings. In this way, optimal voice data can be collected by adjusting the speech rate and volume. For example, by adjusting the speech rate and volume of the dialogue partner, the LLM can analyze pronunciation characteristics in detail.

[0086] The collection unit can analyze the mouth movements of the conversation partner when collecting the pronunciation and complement the pronunciation features. For example, the collection unit can analyze the mouth movements of the conversation partner when collecting the pronunciation and complement the pronunciation features. The analysis of mouth movements can be performed using, for example, video analysis technology, action recognition algorithms, etc., but is not limited to these examples. For example, the collection unit can analyze the mouth movements of the conversation partner using a camera and complement the pronunciation features. The collection unit can also synchronize the mouth movement data with the voice data to collect more accurate pronunciation data. Furthermore, the collection unit can learn mouth movement patterns and apply an algorithm to complement the pronunciation features. For example, the collection unit can learn mouth movement patterns and apply an algorithm to complement the pronunciation features. In this way, by analyzing the mouth movements, the pronunciation features can be complemented and more accurate voice data can be collected. For example, by analyzing the mouth movements of the conversation partner, the LLM can analyze the pronunciation features in detail.

[0087] The collection unit can estimate the user's emotion and determine the priority of pronunciations to be collected based on the estimated user's emotion. The collection unit, for example, estimates the user's emotion and determines the priority of pronunciations to be collected based on the estimated user's emotion. Emotion estimation is performed using, for example, technologies such as voice analysis, facial expression recognition, and text analysis, but is not limited to these examples. For example, the collection unit analyzes the user's voice data to estimate the emotion. The collection unit can also analyze the user's facial expression data to estimate the emotion. Furthermore, the collection unit can analyze the user's text data to estimate the emotion. For example, the collection unit analyzes the user's text messages to estimate the emotion. In this way, by determining the priority of pronunciations according to the user's emotion, important pronunciations can be preferentially collected. For example, when the user is excited, important pronunciations can be preferentially collected. Furthermore, when the user is relaxed, overall pronunciations can be collected in a balanced manner. Furthermore, when the user is nervous, pronunciations can be collected in order from easiest to easiest.

[0088] When collecting pronunciations, the collection unit can prioritize collecting highly relevant pronunciations by taking into account the geographical location information of the conversation partner. For example, when collecting pronunciations, the collection unit prioritizes collecting highly relevant pronunciations by taking into account the geographical location information of the conversation partner. Consideration of geographical location information is performed using technologies such as, but not limited to, GPS data and location estimation from IP addresses. For example, if the conversation partner is in a specific region, the collection unit prioritizes collecting pronunciations specific to that region. The collection unit can also prioritize collecting related dialects and accents based on the geographical location information. Furthermore, the collection unit can update the location information of the conversation partner in real time to collect optimal pronunciations. For example, the collection unit updates the location information of the conversation partner in real time to collect optimal pronunciations. In this way, by taking geographical location information into account, highly relevant pronunciations can be prioritized. For example, if the conversation partner is in a specific region, priority is given to collecting pronunciations specific to that region, allowing the LLM to analyze pronunciation characteristics in detail.

[0089] The collection unit can analyze the social media activities of the conversation partner when collecting pronunciations and collect related pronunciations. For example, the collection unit analyzes the social media activities of the conversation partner when collecting pronunciations and collects related pronunciations. The analysis of social media activities is performed using techniques such as, but not limited to, analysis of post content and analysis of followers. For example, the collection unit analyzes the content of posts on social media of the conversation partner and collects related pronunciations. The collection unit can also collect related pronunciations based on the conversation partner's check-in information on social media. Furthermore, the collection unit can collect related pronunciations by referring to the activities of the conversation partner's friends on social media. For example, the collection unit collects related pronunciations by referring to the activities of the conversation partner's friends on social media. In this way, related pronunciations can be collected by analyzing social media activities. For example, by analyzing the content of posts on social media of the conversation partner, the LLM can analyze pronunciation characteristics in detail.

[0090] The collection unit can customize the collection method by reflecting the dialogue partner's past feedback when collecting pronunciation. For example, the collection unit customizes the collection method by reflecting the dialogue partner's past feedback when collecting pronunciation. Reflecting past feedback can be performed using data such as, for example, user reviews and survey results, but is not limited to these examples. For example, the collection unit customizes the collection method based on feedback provided by the dialogue partner in the past. The collection unit can also preferentially collect specific pronunciations from the dialogue partner's past feedback. Furthermore, the collection unit can reflect the dialogue partner's feedback in real time and adjust the collection method. For example, the collection unit reflects the dialogue partner's feedback in real time and adjusts the collection method. In this way, by reflecting past feedback, the collection method can be customized and more appropriate pronunciation data can be collected. For example, customizing the collection method based on the dialogue partner's past feedback allows the LLM to analyze pronunciation characteristics in detail.

[0091] The analysis unit can estimate the user's emotion and adjust the presentation method of the analysis based on the estimated user's emotion. The analysis unit, for example, estimates the user's emotion and adjusts the presentation method of the analysis based on the estimated user's emotion. Emotion estimation is performed using, for example, but not limited to, technologies such as voice analysis, facial expression recognition, and text analysis. For example, the analysis unit can analyze the user's voice data to estimate the emotion. The analysis unit can also analyze the user's facial expression data to estimate the emotion. Furthermore, the analysis unit can analyze the user's text data to estimate the emotion. For example, the analysis unit can analyze the user's text messages to estimate the emotion. This allows the analysis presentation method to be adjusted according to the user's emotion, thereby providing more appropriate analysis results. For example, if the user is relaxed, detailed analysis results can be provided. If the user is nervous, concise and to the point analysis results can be provided. Furthermore, if the user is in a hurry, analysis results can be provided quickly.

[0092] The analysis unit can apply different analysis algorithms to each phoneme of the pronunciation during analysis. For example, the analysis unit applies different analysis algorithms to each phoneme of the pronunciation during analysis. Application of different analysis algorithms to each phoneme includes, but is not limited to, using different analysis algorithms for vowels and consonants. For example, the analysis unit applies different analysis algorithms to vowels and consonants to analyze pronunciation features in detail. The analysis unit can also apply dedicated analysis algorithms to specific phonemes to improve accuracy. Furthermore, the analysis unit can combine different analysis algorithms for each phoneme to provide a comprehensive analysis result. For example, the analysis unit combines different analysis algorithms for each phoneme to provide a comprehensive analysis result. In this way, by applying different analysis algorithms to each phoneme, pronunciation features can be analyzed in detail. For example, by applying different analysis algorithms to vowels and consonants, LLM can analyze pronunciation features in detail.

[0093] The analysis unit can analyze the intonation and rhythm of the dialogue partner's pronunciation during analysis. For example, the analysis unit analyzes the intonation and rhythm of the dialogue partner's pronunciation during analysis. The analysis of intonation and rhythm is performed using, for example, techniques such as speech waveform analysis and rhythm pattern extraction, but is not limited to these examples. For example, the analysis unit analyzes the intonation of the dialogue partner's pronunciation and extracts characteristics. The analysis unit can also analyze the rhythm of the dialogue partner's pronunciation and extract characteristics. Furthermore, the analysis unit can combine the results of the intonation and rhythm analysis to extract comprehensive pronunciation characteristics. For example, the analysis unit combines the results of the intonation and rhythm analysis to extract comprehensive pronunciation characteristics. In this way, by analyzing the intonation and rhythm, pronunciation characteristics can be grasped in detail. For example, by analyzing the intonation and rhythm of the dialogue partner's pronunciation, LLM can analyze pronunciation characteristics in detail.

[0094] The analysis unit can identify the accent of the dialogue partner's pronunciation during analysis and reflect it in the analysis results. For example, the analysis unit can identify the accent of the dialogue partner's pronunciation during analysis and reflect it in the analysis results. The identification of the accent is performed using techniques such as, for example, voice stress patterns and regional pronunciation characteristics, but is not limited to these examples. For example, the analysis unit can identify the accent of the dialogue partner's pronunciation and reflect it in the analysis results. The analysis unit can also analyze the position and strength of the accent and extract features. Furthermore, the analysis unit can analyze the pronunciation features in detail based on the accent analysis results. For example, the analysis unit analyzes the position and strength of the accent and extracts features. By identifying the accent, the pronunciation features can be analyzed in detail. For example, by identifying the accent of the dialogue partner's pronunciation, the LLM can analyze the pronunciation features in detail.

[0095] The analysis unit can estimate the user's emotion and adjust the level of detail of the analysis based on the estimated user's emotion. The analysis unit, for example, estimates the user's emotion and adjusts the level of detail of the analysis based on the estimated user's emotion. Emotion estimation is performed using, for example, but not limited to, technologies such as voice analysis, facial expression recognition, and text analysis. For example, the analysis unit can analyze the user's voice data to estimate the emotion. The analysis unit can also analyze the user's facial expression data to estimate the emotion. Furthermore, the analysis unit can analyze the user's text data to estimate the emotion. For example, the analysis unit can analyze the user's text messages to estimate the emotion. This allows the level of detail of the analysis to be adjusted according to the user's emotion, thereby providing more appropriate analysis results. For example, if the user is relaxed, a detailed analysis result can be provided. If the user is nervous, a concise and to-the-point analysis result can be provided. Furthermore, if the user is in a hurry, analysis results can be provided quickly.

[0096] The analysis unit can determine the analysis priority based on the time of submission of the pronunciation during analysis. For example, the analysis unit determines the analysis priority based on the time of submission of the pronunciation during analysis. The submission time is taken into consideration using, for example, a timestamp, a method for recording the submission date and time, etc., but is not limited to such examples. For example, the analysis unit prioritizes analysis of the most recent pronunciation data. The analysis unit can also analyze pronunciation data that was submitted earlier later. Furthermore, the analysis unit can dynamically adjust the analysis priority based on the submission time. For example, the analysis unit dynamically adjusts the analysis priority based on the submission time. As a result, by determining the analysis priority based on the submission time, the most recent pronunciation data can be analyzed preferentially. For example, by prioritizing analysis of the most recent pronunciation data, the LLM can analyze the pronunciation features in detail.

[0097] The analysis unit can adjust the order of analysis based on the relevance of pronunciation during analysis. For example, the analysis unit adjusts the order of analysis based on the relevance of pronunciation during analysis. The evaluation of relevance is performed using, for example, techniques such as content similarity and a method for calculating a relevance score, but is not limited to these examples. For example, the analysis unit prioritizes analysis of highly relevant pronunciation data. The analysis unit can also postpone analysis of less relevant pronunciation data. Furthermore, the analysis unit can dynamically adjust the order of analysis based on the relevance of pronunciation. For example, the analysis unit dynamically adjusts the order of analysis based on the relevance of pronunciation. As a result, by adjusting the order of analysis based on the relevance of pronunciation, highly relevant pronunciation data can be prioritized for analysis. For example, by prioritizing analysis of highly relevant pronunciation data, the LLM can analyze pronunciation features in detail.

[0098] The analysis unit can adjust the use of technical terminology during analysis according to the expertise level of the interlocutor. For example, the analysis unit adjusts the use of technical terminology during analysis according to the expertise level of the interlocutor. The expertise level is evaluated using data such as, for example, qualification information and past utterance content, but is not limited to such examples. For example, the analysis unit can provide analysis results using detailed technical terminology to interlocutors with a high level of expertise. The analysis unit can also provide analysis results using concise and easy-to-understand terms to interlocutors with a low level of expertise. Furthermore, the analysis unit can dynamically adjust the way the analysis results are presented according to the expertise level. For example, the analysis unit dynamically adjusts the way the analysis results are presented according to the expertise level. This allows the use of technical terminology during analysis to be adjusted according to the expertise level, thereby providing analysis results appropriate for the interlocutor. For example, by providing analysis results using detailed technical terminology to interlocutors with a high level of expertise, the LLM can analyze pronunciation characteristics in detail.

[0099] The conversion unit can estimate the user's emotion and adjust the conversion method based on the estimated user's emotion. For example, the conversion unit can estimate the user's emotion and adjust the conversion method based on the estimated user's emotion. Emotion estimation can be performed using, for example, technologies such as voice analysis, facial expression recognition, and text analysis, but is not limited to these examples. For example, the conversion unit can analyze the user's voice data to estimate the emotion. The conversion unit can also analyze the user's facial expression data to estimate the emotion. Furthermore, the conversion unit can analyze the user's text data to estimate the emotion. For example, the conversion unit can analyze the user's text message to estimate the emotion. This allows the conversion method to be adjusted according to the user's emotion, thereby providing more appropriate conversion results. For example, if the user is relaxed, conversion that is closer to natural pronunciation can be performed. Furthermore, if the user is nervous, concise and easy-to-understand conversion can be performed. Furthermore, if the user is in a hurry, conversion can be performed quickly.

[0100] The conversion unit can apply different conversion algorithms to each phoneme of the pronunciation during conversion. For example, the conversion unit applies different conversion algorithms to each phoneme of the pronunciation during conversion. Application of different conversion algorithms to each phoneme includes, but is not limited to, using different conversion algorithms for vowels and consonants. For example, the conversion unit applies different conversion algorithms to vowels and consonants to convert pronunciation features in detail. The conversion unit can also apply a dedicated conversion algorithm to a specific phoneme to improve accuracy. Furthermore, the conversion unit can combine different conversion algorithms for each phoneme to provide an overall conversion result. For example, the conversion unit combines different conversion algorithms for each phoneme to provide an overall conversion result. In this way, by applying different conversion algorithms to each phoneme, pronunciation features can be converted in detail. For example, by applying different conversion algorithms to vowels and consonants, LLM can convert pronunciation features in detail.

[0101] The conversion unit can take into account the intonation and rhythm of the dialogue partner's pronunciation during conversion. For example, the conversion unit takes into account the intonation and rhythm of the dialogue partner's pronunciation during conversion. Consideration of intonation and rhythm can be performed using, for example, techniques such as speech waveform analysis and rhythm pattern extraction, but is not limited to these examples. For example, the conversion unit takes into account the intonation of the dialogue partner's pronunciation during conversion. The conversion unit can also take into account the rhythm of the dialogue partner's pronunciation during conversion. Furthermore, the conversion unit can combine intonation and rhythm features to perform conversion that is closer to natural pronunciation. For example, the conversion unit combines intonation and rhythm features to perform conversion that is closer to natural pronunciation. By taking into account intonation and rhythm, conversion that is closer to more natural pronunciation is possible. For example, by taking into account the intonation and rhythm of the dialogue partner's pronunciation, the LLM can convert pronunciation features in detail.

[0102] The conversion unit can reflect the accent of the dialogue partner's pronunciation during conversion. For example, the conversion unit reflects the accent of the dialogue partner during conversion. Reflecting the accent can be performed using, for example, techniques such as voice stress patterns and regional pronunciation characteristics, but is not limited to these examples. For example, the conversion unit reflects the accent of the dialogue partner during conversion. The conversion unit can also take into account the position and strength of the accent during conversion. Furthermore, the conversion unit can perform conversion that is closer to natural pronunciation based on the accent characteristics. For example, the conversion unit performs conversion that is closer to natural pronunciation based on the accent characteristics. By reflecting the accent, conversion that is closer to more natural pronunciation is possible. For example, by reflecting the accent of the dialogue partner, the LLM can perform detailed conversion of pronunciation characteristics.

[0103] The conversion unit can estimate the user's emotion and determine the conversion priority based on the estimated user's emotion. The conversion unit can, for example, estimate the user's emotion and determine the conversion priority based on the estimated user's emotion. Emotion estimation can be performed using, for example, technologies such as voice analysis, facial expression recognition, and text analysis, but is not limited to these examples. For example, the conversion unit can analyze the user's voice data to estimate the emotion. The conversion unit can also analyze the user's facial expression data to estimate the emotion. Furthermore, the conversion unit can analyze the user's text data to estimate the emotion. For example, the conversion unit can analyze the user's text message to estimate the emotion. This allows the conversion priority to be determined according to the user's emotion, so that important pronunciations can be converted preferentially. For example, if the user is relaxed, the overall pronunciation can be converted in a balanced manner. Also, if the user is nervous, important pronunciations can be converted preferentially. Furthermore, if the user is in a hurry, quick conversion can be performed for each short phrase.

[0104] The conversion unit can perform the conversion while taking into account the geographical distribution of pronunciation. For example, the conversion unit performs the conversion while taking into account the geographical distribution of pronunciation. Consideration of the geographical distribution can be performed using, for example, techniques such as regional pronunciation characteristics and geographical data acquisition methods, but is not limited to these examples. For example, if the pronunciation of the dialogue partner is unique to a particular region, the conversion unit performs the conversion while taking into account the pronunciation characteristics of that region. The conversion unit can also perform the conversion while reflecting associated dialects and accents based on the geographical distribution. Furthermore, the conversion unit can apply an optimal conversion algorithm based on the geographical distribution data. For example, the conversion unit applies an optimal conversion algorithm based on the geographical distribution data. This allows for conversion that reflects regional pronunciation by taking into account the geographical distribution. For example, by taking into account the geographical distribution of the dialogue partner's pronunciation, the LLM can perform detailed conversion of pronunciation characteristics.

[0105] The conversion unit can improve the accuracy of the conversion by referring to literature related to pronunciation during conversion. For example, the conversion unit improves the accuracy of the conversion by referring to literature related to pronunciation during conversion. Referencing the related literature can be performed using, for example, techniques such as literature selection criteria and reference methods, but is not limited to these examples. For example, the conversion unit can refer to literature related to pronunciation and optimize the conversion algorithm. The conversion unit can also analyze pronunciation features in detail based on data from the related literature to improve the accuracy of the conversion. Furthermore, the conversion unit can refer to information from the related literature in real time and reflect the analysis in the conversion results. For example, the conversion unit can refer to information from the related literature in real time and reflect the analysis in the conversion results. As a result, the accuracy of the conversion is improved by referring to the related literature. For example, by referring to literature related to pronunciation, the LLM can convert pronunciation features in detail.

[0106] The conversion unit can perform the conversion while taking into account the market value of the pronunciation. For example, the conversion unit performs the conversion while taking into account the market value of the pronunciation. Consideration of market value can be performed using, for example, market research data, value assessment standards, and other technologies, but is not limited to these examples. For example, the conversion unit can preferentially convert pronunciations with high market value. The conversion unit can also adjust the pronunciation conversion algorithm based on the market value. Furthermore, the conversion unit can select the optimal conversion method based on the market value data. For example, the conversion unit selects the optimal conversion method based on the market value data. In this way, by taking market value into consideration, pronunciations with high value can be preferentially converted. For example, by preferentially converting pronunciations with high market value, the LLM can convert the pronunciation features in detail.

[0107] The providing unit can estimate the user's emotion and adjust the method of providing information based on the estimated user's emotion. The providing unit, for example, estimates the user's emotion and adjusts the method of providing information based on the estimated user's emotion. Emotion estimation is performed using, for example, technologies such as voice analysis, facial expression recognition, and text analysis, but is not limited to these examples. For example, the providing unit analyzes the user's voice data to estimate the emotion. The providing unit can also analyze the user's facial expression data to estimate the emotion. Furthermore, the providing unit can analyze the user's text data to estimate the emotion. For example, the providing unit analyzes the user's text message to estimate the emotion. This enables more appropriate information to be provided by adjusting the method of providing information according to the user's emotion. For example, if the user is relaxed, information can be provided in a natural voice. If the user is nervous, information can be provided in a concise and easy-to-understand voice. Furthermore, if the user is in a hurry, information can be provided quickly.

[0108] The providing unit can select the optimal delivery method by referring to the user's past operation history at the time of delivery. For example, the providing unit selects the optimal delivery method by referring to the user's past operation history at the time of delivery. Referencing the operation history is performed using, for example, a technology such as a history data acquisition method or a reference method, but is not limited to such examples. For example, the providing unit preferentially selects a delivery method that the user has used favorably in the past. The providing unit can also suggest the optimal audio delivery method based on the user's past operation history. Furthermore, the providing unit can refer to the user's operation history in real time and dynamically adjust the delivery method. For example, the providing unit refers to the user's operation history in real time and dynamically adjusts the delivery method. In this way, the optimal delivery method for the user can be selected by referring to the past operation history. For example, by preferentially selecting a delivery method that the user has used favorably in the past, the LLM can convert pronunciation features in detail.

[0109] The providing unit can customize the provided content according to the user's current task at the time of providing. For example, the providing unit customizes the provided content according to the user's current task at the time of providing. Task detection is performed using, for example, techniques such as user behavioral data and device usage status, but is not limited to such examples. For example, the providing unit provides the content in a quiet voice when the user is in a meeting. Furthermore, the providing unit can provide the content in a concise and easy-to-understand voice when the user is driving. Furthermore, the providing unit can detect the user's current task in real time and dynamically customize the provided content. For example, the providing unit detects the user's current task in real time and dynamically customizes the provided content. This enables more appropriate provision by customizing the provided content according to the current task. For example, when the user is in a meeting, providing the content in a quiet voice allows the LLM to convert pronunciation characteristics in detail.

[0110] The providing unit can estimate the user's emotion and determine the priority of provision based on the estimated user's emotion. The providing unit, for example, estimates the user's emotion and determines the priority of provision based on the estimated user's emotion. Emotion estimation is performed using, for example, technologies such as voice analysis, facial expression recognition, and text analysis, but is not limited to these examples. For example, the providing unit analyzes the user's voice data to estimate the emotion. The providing unit can also analyze the user's facial expression data to estimate the emotion. Furthermore, the providing unit can analyze the user's text data to estimate the emotion. For example, the providing unit analyzes the user's text message to estimate the emotion. This allows the provision priority to be determined according to the user's emotion, thereby prioritizing the provision of important pronunciations. For example, when the user is relaxed, overall pronunciations can be provided in a balanced manner. Furthermore, when the user is nervous, important pronunciations can be provided preferentially. Furthermore, when the user is in a hurry, short phrases can be provided quickly.

[0111] The providing unit can select the optimal presentation method by taking into account the user's device information at the time of presentation. For example, the providing unit selects the optimal presentation method by taking into account the user's device information at the time of presentation. Consideration of device information is performed using, for example, technology such as device type and specification information, but is not limited to such examples. For example, if the user is using a smartphone, the providing unit selects a presentation method that matches the screen size. Also, if the user is using a tablet, the providing unit can select a presentation method optimized for a large screen. Furthermore, if the user is using a smartwatch, the providing unit can select a presentation method that is concise and highly visible. For example, if the user is using a smartwatch, the providing unit selects a presentation method that is concise and highly visible. In this way, the optimal presentation method for the user can be selected by taking into account the device information. For example, if the user is using a smartphone, selecting a presentation method that matches the screen size allows the LLM to convert pronunciation features in detail.

[0112] The providing unit can make the provided content multilingual based on the user's language setting at the time of provision. For example, the providing unit can make the provided content multilingual based on the user's language setting at the time of provision. Consideration of the language setting is performed using, for example, technology such as the user's language setting information and language selection criteria, but is not limited to such examples. For example, the providing unit automatically sets the provided content based on the language setting of the user's device. The providing unit can also provide a language switching function when the user uses multiple languages. Furthermore, the providing unit can provide the provided content in a specific language when the user selects that language. For example, when the user selects a specific language, the providing unit provides the provided content in that language. By making the provided content multilingual based on the language setting, the provided content can be provided in the language that is most suitable for the user. For example, by automatically setting the provided content based on the language setting of the user's device, the LLM can convert pronunciation characteristics in detail.

[0113] The learning unit can estimate the user's emotions and select training data based on the estimated user emotions. For example, the learning unit can estimate the user's emotions and select training data based on the estimated user emotions. Emotion estimation can be performed using, for example, technologies such as voice analysis, facial expression recognition, and text analysis, but is not limited to these examples. For example, the learning unit can analyze the user's voice data to estimate the emotions. The learning unit can also analyze the user's facial expression data to estimate the emotions. Furthermore, the learning unit can analyze the user's text data to estimate the emotions. For example, the learning unit can analyze the user's text messages to estimate the emotions. This enables more appropriate learning by selecting training data according to the user's emotions. For example, if the user is relaxed, detailed training data can be selected. If the user is nervous, concise and to the point training data can be selected. Furthermore, if the user is in a hurry, training data can be selected quickly.

[0114] The learning unit can optimize the learning algorithm by referring to past learning data during learning. For example, the learning unit optimizes the learning algorithm by referring to past learning data during learning. Optimization of the learning algorithm is performed using, for example, techniques such as algorithm selection criteria and optimization methods, but is not limited to these examples. For example, the learning unit optimizes the learning algorithm based on past learning data. The learning unit can also analyze trends in the past learning data and select an optimal learning algorithm. Furthermore, the learning unit can dynamically adjust the learning algorithm by referring to past learning data in real time. For example, the learning unit dynamically adjusts the learning algorithm by referring to past learning data. This optimizes the learning algorithm by referring to past learning data, improving learning accuracy. For example, optimizing the learning algorithm based on past learning data allows the LLM to convert pronunciation features in detail.

[0115] The learning unit can update the learning data by reflecting user feedback during learning. For example, the learning unit updates the learning data by reflecting user feedback during learning. Reflecting feedback is performed using, for example, techniques such as the type of feedback and the reflection method, but is not limited to these examples. For example, the learning unit updates the learning data based on user feedback. The learning unit can also reflect user feedback in real time and dynamically update the learning data. Furthermore, the learning unit can analyze user feedback and select optimal learning data. For example, the learning unit analyzes user feedback and selects optimal learning data. In this way, the learning data is updated by reflecting user feedback, improving learning accuracy. For example, by updating the learning data based on user feedback, the LLM can convert pronunciation features in detail.

[0116] The learning unit can estimate the user's emotions and adjust the frequency of learning based on the estimated user emotions. For example, the learning unit estimates the user's emotions and adjusts the frequency of learning based on the estimated user emotions. Emotion estimation is performed using, for example, technologies such as voice analysis, facial expression recognition, and text analysis, but is not limited to these examples. For example, the learning unit analyzes the user's voice data to estimate the emotions. The learning unit can also analyze the user's facial expression data to estimate the emotions. Furthermore, the learning unit can analyze the user's text data to estimate the emotions. For example, the learning unit analyzes the user's text messages to estimate the emotions. This allows for more appropriate learning by adjusting the frequency of learning according to the user's emotions. For example, if the user is relaxed, the learning can be performed more frequently. Also, if the user is nervous, the learning frequency can be reduced. Furthermore, if the user is in a hurry, the learning can be performed quickly.

[0117] The learning unit can weight the learning data based on the time of submission of the pronunciation during learning. For example, the learning unit weights the learning data based on the time of submission of the pronunciation during learning. The weighting method is performed using, for example, techniques such as a data importance evaluation criterion and a weighting algorithm, but is not limited to these examples. For example, the learning unit weights the most recent pronunciation data during learning. The learning unit can also weight pronunciation data that was submitted earlier during learning. Furthermore, the learning unit can dynamically adjust the weighting of the learning data based on the time of submission. For example, the learning unit dynamically adjusts the weighting of the learning data based on the time of submission. As a result, by weighting the learning data based on the time of submission, learning can be performed with an emphasis on the most recent pronunciation data. For example, by weighting the most recent pronunciation data during learning, the LLM can convert pronunciation features in detail.

[0118] The learning unit can integrate information from different data sources during learning to enrich the training data. For example, the learning unit integrates information from different data sources during learning to enrich the training data. Data source integration can be performed using, for example, techniques based on the type of data source, integration method, etc., but is not limited to these examples. For example, the learning unit integrates pronunciation data from different data sources to perform learning. The learning unit can also enrich the training data based on information from different data sources. Furthermore, the learning unit can integrate data from different data sources in real time and dynamically update the training data. For example, the learning unit integrates data from different data sources in real time and dynamically update the training data. In this way, by integrating information from different data sources, the training data is enriched and the accuracy of learning is improved. For example, by integrating pronunciation data from different data sources and performing learning, the LLM can convert pronunciation features in detail.

[0119] The real-time providing unit can estimate the user's emotion and adjust the method of real-time provision based on the estimated user's emotion. The real-time providing unit, for example, estimates the user's emotion and adjusts the method of real-time provision based on the estimated user's emotion. Emotion estimation is performed using, for example, technologies such as voice analysis, facial expression recognition, and text analysis, but is not limited to these examples. For example, the real-time providing unit analyzes the user's voice data to estimate the emotion. The real-time providing unit can also analyze the user's facial expression data to estimate the emotion. Furthermore, the real-time providing unit can analyze the user's text data to estimate the emotion. For example, the real-time providing unit analyzes the user's text message to estimate the emotion. This enables more appropriate provision by adjusting the method of real-time provision according to the user's emotion. For example, if the user is relaxed, real-time provision can be performed in a natural voice. Furthermore, if the user is nervous, real-time provision can be performed in a concise and easy-to-understand voice. Furthermore, if the user is in a hurry, real-time provision can be performed quickly.

[0120] The real-time providing unit can select the optimal delivery method by referring to the user's past operation history during real-time delivery. For example, the real-time providing unit selects the optimal delivery method by referring to the user's past operation history during real-time delivery. Referencing the operation history is performed using, for example, a technology such as a history data acquisition method or a reference method, but is not limited to such examples. For example, the real-time providing unit preferentially selects a delivery method that the user has used favorably in the past. The real-time providing unit can also suggest the optimal real-time delivery method based on the user's past operation history. Furthermore, the real-time providing unit can refer to the user's operation history in real time and dynamically adjust the delivery method. For example, the real-time providing unit refers to the user's operation history in real time and dynamically adjusts the delivery method. In this way, the optimal real-time delivery method for the user can be selected by referring to the past operation history. For example, by preferentially selecting a delivery method that the user has used favorably in the past, the LLM can convert pronunciation features in detail.

[0121] The real-time providing unit can customize the content to be provided in accordance with the user's current task during real-time provision. For example, the real-time providing unit customizes the content to be provided in accordance with the user's current task during real-time provision. Task detection is performed using, for example, techniques such as user behavioral data and device usage status, but is not limited to these examples. For example, if the user is in a meeting, the real-time providing unit provides real-time content in a quiet voice. Furthermore, if the user is driving, the real-time providing unit can provide real-time content in a concise and easy-to-understand voice. Furthermore, the real-time providing unit can detect the user's current task in real time and dynamically customize the content to be provided. For example, the real-time providing unit detects the user's current task in real time and dynamically customizes the content to be provided. This enables more appropriate content to be provided by customizing the content to the current task. For example, if the user is in a meeting, providing real-time content in a quiet voice allows the LLM to convert pronunciation characteristics in detail.

[0122] The real-time providing unit can estimate the user's emotions and determine the priority of real-time provision based on the estimated user emotions. The real-time providing unit can, for example, estimate the user's emotions and determine the priority of real-time provision based on the estimated user emotions. Emotion estimation can be performed using, for example, technologies such as voice analysis, facial expression recognition, and text analysis, but is not limited to these examples. For example, the real-time providing unit can analyze the user's voice data to estimate the emotion. The real-time providing unit can also analyze the user's facial expression data to estimate the emotion. Furthermore, the real-time providing unit can analyze the user's text data to estimate the emotion. For example, the real-time providing unit can analyze the user's text messages to estimate the emotion. This allows the priority of real-time provision to be determined according to the user's emotions, thereby enabling important pronunciations to be provided preferentially. For example, when the user is relaxed, overall pronunciations can be provided in a balanced manner in real time. Furthermore, when the user is nervous, important pronunciations can be provided preferentially in real time. Furthermore, when the user is in a hurry, real-time provision can be performed quickly for each short phrase.

[0123] The real-time providing unit can select the optimal delivery method by taking into account the user's device information during real-time delivery. For example, the real-time providing unit selects the optimal delivery method by taking into account the user's device information during real-time delivery. Consideration of device information is performed using, for example, technology such as device type and specification information, but is not limited to such examples. For example, if the user is using a smartphone, the real-time providing unit selects a real-time delivery method that matches the screen size. Furthermore, if the user is using a tablet, the real-time providing unit can select a real-time delivery method that is optimized for a large screen. Furthermore, if the user is using a smartwatch, the real-time providing unit can select a real-time delivery method that is concise and highly visible. For example, if the user is using a smartwatch, the real-time providing unit selects a real-time delivery method that is concise and highly visible. In this way, the optimal real-time delivery method for the user can be selected by taking into account the device information. For example, if the user is using a smartphone, selecting a real-time delivery method that matches the screen size allows the LLM to convert pronunciation features in detail.

[0124] The real-time providing unit can provide multilingual content in accordance with the user's language setting during real-time provision. For example, the real-time providing unit provides multilingual content in accordance with the user's language setting during real-time provision. Consideration of the language setting is performed using, for example, technology such as the user's language setting information and language selection criteria, but is not limited to such examples. For example, the real-time providing unit automatically sets the real-time content based on the language setting of the user's device. The real-time providing unit can also provide a language switching function when the user uses multiple languages. Furthermore, the real-time providing unit can provide real-time content in a specific language when the user selects that language. For example, the real-time providing unit provides real-time content in that language when the user selects a specific language. This allows the content to be provided in multiple languages ​​according to the language setting, thereby providing the content in the language most suitable for the user. For example, automatically setting the real-time content based on the language setting of the user's device allows the LLM to convert pronunciation characteristics in detail. === Hard Collateral 1-1 === Each of the multiple elements, including the collection unit, analysis unit, conversion unit, provision unit, learning unit, and real-time provision unit, described above, is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the collection unit collects the pronunciation of the conversation partner using the microphone 38B of the smart device 14 and transmits it to the specific processing unit 290 of the data processing device 12. For example, the analysis unit analyzes the pronunciation collected by the specific processing unit 290 of the data processing device 12 and extracts phoneme and intonation characteristics. For example, the conversion unit converts the pronunciation of the conversation partner into a pronunciation similar to the user's own language based on the characteristics analyzed by the specific processing unit 290 of the data processing device 12. For example, the provision unit provides the converted pronunciation to the user in real time using the control unit 46A of the smart device 14. For example, the learning unit learns the pronunciation of the conversation partner's country of origin and the user's own country of origin using the specific processing unit 290 of the data processing device 12. For example, the real-time provision unit streams the converted pronunciation in real time using the control unit 46A of the smart device 14. For example, the collection unit estimates the user's emotions using the camera 42 and microphone 38B of the smart device 14, and adjusts the collection timing using the specific processing unit 290 of the data processing device 12. === Hard Collateral 1-2 === Each of the multiple elements, including the collection unit, analysis unit, conversion unit, provision unit, learning unit, and real-time provision unit, described above, is realized, for example, in at least one of the smart glasses 214 and the data processing device 12. For example, the collection unit collects the pronunciation of the conversation partner using the microphone 238 of the smart glasses 214 and transmits it to the specific processing unit 290 of the data processing device 12. For example, the analysis unit analyzes the pronunciation collected by the specific processing unit 290 of the data processing device 12 and extracts phoneme and intonation features. For example, the conversion unit converts the pronunciation of the conversation partner into a pronunciation similar to the conversation partner's own language based on the features analyzed by the specific processing unit 290 of the data processing device 12. For example, the provision unit provides the converted pronunciation to the user in real time using the control unit 46A of the smart glasses 214. For example, the learning unit learns the pronunciation of the conversation partner's country of origin and the user's own country of origin using the specific processing unit 290 of the data processing device 12. For example, the real-time provision unit streams the converted pronunciation in real time using the control unit 46A of the smart glasses 214. For example, the collection unit estimates the user's emotions using the camera 42 and microphone 238 of the smart glasses 214, and adjusts the collection timing using the specific processing unit 290 of the data processing device 12. === Hard Collateral 1-3 === Each of the multiple elements, including the collection unit, analysis unit, conversion unit, provision unit, learning unit, and real-time provision unit, described above, is realized, for example, in at least one of the headset-type terminal 314 and the data processing device 12. For example, the collection unit collects the pronunciation of the conversation partner using the microphone 238 of the headset-type terminal 314 and transmits it to the specific processing unit 290 of the data processing device 12. For example, the analysis unit analyzes the pronunciation collected by the specific processing unit 290 of the data processing device 12 and extracts phoneme and intonation features. For example, the conversion unit converts the pronunciation of the conversation partner into a pronunciation similar to the user's own language based on the features analyzed by the specific processing unit 290 of the data processing device 12. For example, the provision unit provides the converted pronunciation to the user in real time using the control unit 46A of the headset-type terminal 314. For example, the learning unit learns the pronunciation of the conversation partner's country of origin and the user's own country of origin using the specific processing unit 290 of the data processing device 12. For example, the real-time provision unit streams the converted pronunciation in real time using the control unit 46A of the headset-type terminal 314. For example, the collection unit estimates the user's emotions using the camera 42 and microphone 238 of the headset terminal 314, and adjusts the collection timing using the specific processing unit 290 of the data processing device 12. === Hard Collateral 1-4 === Each of the multiple elements, including the collection unit, analysis unit, conversion unit, provision unit, learning unit, and real-time provision unit, described above, is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the collection unit collects the pronunciation of the dialogue partner using the microphone 238 of the robot 414 and transmits it to the specific processing unit 290 of the data processing device 12. For example, the analysis unit analyzes the pronunciation collected by the specific processing unit 290 of the data processing device 12 and extracts phoneme and intonation features. For example, the conversion unit converts the pronunciation of the dialogue partner into a pronunciation similar to the dialogue partner's own language based on the features analyzed by the specific processing unit 290 of the data processing device 12. For example, the provision unit provides the converted pronunciation to the user in real time using the control unit 46A of the robot 414. For example, the learning unit learns the pronunciation of the dialogue partner's country of origin and the user's own country of origin using the specific processing unit 290 of the data processing device 12. For example, the real-time provision unit streams the converted pronunciation in real time using the control unit 46A of the robot 414. For example, the collection unit estimates the user's emotions using the camera 42 and microphone 238 of the robot 414, and adjusts the collection timing using the specific processing unit 290 of the data processing device 12.

[0125] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0126] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can estimate the conversation partner's emotions and adjust the collection method based on the estimated emotions. For example, if the conversation partner is nervous, the collection unit can wait until the conversation partner is relaxed before collecting the pronunciation. Also, if the conversation partner is excited, the collection unit can prioritize collecting pronunciation that reflects that emotion. Furthermore, if the conversation partner is tired, the collection unit can collect pronunciation in short phrases, reducing the burden on the conversation partner. In this way, by adopting a collection method that corresponds to the conversation partner's emotions, more natural and accurate pronunciation data can be collected.

[0127] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can filter the conversation partner's background sounds in real time and collect clear voice data. For example, the collection unit can use noise canceling technology to remove background sounds from the conversation partner's pronunciation. The collection unit can also emphasize sounds in specific frequency bands to collect the conversation partner's pronunciation more clearly. Furthermore, the collection unit can use multiple microphones to collect the conversation partner's pronunciation in a directional manner and effectively remove background sounds. In this way, by removing background sounds, clear voice data can be collected and analysis accuracy can be improved.

[0128] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can adjust the conversation partner's speaking speed and volume in real time to collect optimal voice data. For example, if the conversation partner's speaking speed is fast, the collection unit can automatically slow down the speech to collect the data. Also, if the conversation partner's volume is low, the collection unit can automatically amplify the volume to collect the data. Furthermore, the collection unit can monitor the conversation partner's speaking speed and volume in real time and automatically adjust the optimal collection settings. In this way, by adjusting the speaking speed and volume, optimal voice data can be collected and analysis accuracy can be improved.

[0129] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can analyze the conversation partner's mouth movements and complement the pronunciation features. For example, the collection unit can use a camera to analyze the conversation partner's mouth movements and complement the pronunciation features. The collection unit can also synchronize the mouth movement data with the voice data to collect more accurate pronunciation data. Furthermore, the collection unit can learn mouth movement patterns and apply an algorithm to complement the pronunciation features. In this way, by analyzing the mouth movements, pronunciation features can be complemented and more accurate voice data can be collected.

[0130] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can prioritize collecting highly relevant pronunciations by taking into account the conversation partner's geographical location information. For example, if the conversation partner is in a specific region, the collection unit can prioritize collecting pronunciations specific to that region. The collection unit can also prioritize collecting related dialects and accents based on the geographical location information. Furthermore, the collection unit can update the conversation partner's location information in real time and collect optimal pronunciations. In this way, by taking geographical location information into account, highly relevant pronunciations can be prioritized and analysis accuracy can be improved.

[0131] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can analyze the conversation partner's social media activities and collect related pronunciations. For example, the collection unit can analyze the conversation partner's social media posts and collect related pronunciations. The collection unit can also collect related pronunciations based on the conversation partner's social media check-in information. Furthermore, the collection unit can collect related pronunciations by referring to the activities of the conversation partner's friends on social media. In this way, by analyzing social media activities, related pronunciations can be collected and analysis accuracy can be improved.

[0132] When collecting pronunciation from a dialogue partner, the pronunciation conversion system can customize the collection method by reflecting the dialogue partner's past feedback. For example, the collection unit can customize the collection method based on feedback provided by the dialogue partner in the past. The collection unit can also preferentially collect specific pronunciations from the dialogue partner's past feedback. Furthermore, the collection unit can reflect the dialogue partner's feedback in real time and adjust the collection method. In this way, by reflecting past feedback, the collection method can be customized and more appropriate pronunciation data can be collected.

[0133] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can estimate the emotion of the conversation partner and determine the priority of the pronunciations to be collected based on the estimated emotion. For example, if the conversation partner is excited, important pronunciations can be collected with priority. Also, if the conversation partner is relaxed, overall pronunciations can be collected in a balanced manner. Furthermore, if the conversation partner is nervous, pronunciations can be collected in order from simple to complex. In this way, by determining the priority of pronunciations according to the emotion of the conversation partner, important pronunciations can be collected with priority, and analysis accuracy can be improved.

[0134] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can estimate the emotion of the conversation partner and adjust the collection timing based on the estimated emotion. For example, if the conversation partner is relaxed, the pronunciation can be collected in accordance with the natural flow of the conversation. If the conversation partner is nervous, the pronunciation can be collected with appropriate pauses between conversations. Furthermore, if the conversation partner is in a hurry, the pronunciation can be collected for each short phrase and processed quickly. In this way, by adjusting the collection timing according to the emotion of the conversation partner, more natural pronunciation data can be collected.

[0135] When collecting the pronunciation of a conversation partner, the pronunciation conversion system can estimate the emotion of the conversation partner and adjust the collection method based on the estimated emotion. For example, if the conversation partner is relaxed, the system can collect pronunciation in accordance with the natural flow of the conversation. If the conversation partner is nervous, the system can collect pronunciation with appropriate pauses between conversations. Furthermore, if the conversation partner is in a hurry, the system can collect pronunciation for each short phrase and process it quickly. In this way, by adjusting the collection method according to the emotion of the conversation partner, more natural pronunciation data can be collected.

[0136] The processing flow of the second embodiment will be briefly explained below.

[0137] Step 1: The collection unit collects the pronunciation of the conversation partner. The conversation partner's pronunciation includes the pronunciation of words, sentences, and specific languages. The collection unit records the conversation partner's pronunciation and inputs the data into the LLM. The collection unit can also collect the conversation partner's pronunciation in real time. For example, the collection unit collects the conversation partner's pronunciation using a microphone and inputs the data into the LLM in real time. Step 2: The analyzer uses LLM to analyze the pronunciation collected by the collector. The analysis extracts features such as phonemes, intonation, and rhythm. For example, the analyzer extracts phoneme features and identifies which parts differ from the pronunciation in the native language. The analyzer also analyzes the speech waveform and extracts intonation and rhythm patterns. Step 3: The conversion unit converts the pronunciation of the interlocutor into one that sounds like the interlocutor's own language based on the pronunciation characteristics analyzed by the analysis unit. The conversion is performed based on a phoneme substitution method, an accent adjustment method, or the like. For example, the conversion unit converts the pronunciation of the interlocutor into one that sounds like the interlocutor's own language using a phoneme substitution method. The conversion unit can also convert the pronunciation of the interlocutor into one that sounds like the interlocutor's own language using an accent adjustment method. Step 4: The providing unit provides the pronunciation converted by the converting unit to the user in real time. The providing is performed using audio streaming technology. For example, the providing unit streams the converted pronunciation to the user in real time. The providing unit can also record the converted pronunciation and play it back later.

[0138] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0139] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0140] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0141] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0142] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0143] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0144] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0145] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0146] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0147] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0148] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0149] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0150] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0151] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0152] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0153] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0154] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0155] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0156] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0157] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0158] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0159] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0160] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0161] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0162] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0163] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0164] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0165] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0166] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0167] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0168] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0169] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0170] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0171] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0172] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0173] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0174] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0175] 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0176] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0177] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0178] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0179] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0180] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0181] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0182] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0183] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0184] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0185] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0186] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0187] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0188] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0189] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0190] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0191] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0192] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0193] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0194] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0195] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0196] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0197] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0198] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0199] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0200] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0201] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0202] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0203] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0204] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0205] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0206] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0207] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0208] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0209] [Explanation of symbols]

[0210] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a collection unit that collects pronunciations; an analysis unit that analyzes the pronunciation collected by the collection unit; a conversion unit that converts the pronunciation analyzed by the analysis unit into a pronunciation similar to the native language; a providing unit that provides the pronunciation converted by the converting unit; Equipped with A system characterized by:

2. Equipped with a learning section that allows you to learn the pronunciation of your own country and the country of your conversation partner's origin 2. The system of claim 1.

3. A real-time providing unit is provided that provides converted pronunciation in real time.

2. The system of claim 1.

4. The collecting unit Record the pronunciation of your conversation partner and enter the data into LLM.

2. The system of claim 1.

5. The analysis unit Analyze the pronunciation of your conversation partner and extract its characteristics 2. The system of claim 1.

6. The conversion unit Based on the analyzed pronunciation characteristics, the pronunciation of the interlocutor is converted to sound more like the individual's own language.

2. The system of claim 1.

7. The providing unit Providing the converted pronunciation to the user in real time 2. The system of claim 1.

8. The collecting unit The system estimates the user's emotions and adjusts the timing of collecting the pronunciation based on the estimated user emotions.

2. The system of claim 1.

9. The collecting unit Eliminate background noise from the other person's speech and collect clear voice data 2. The system of claim 1.

Images