System

Abstract

To provide a system according to an embodiment, capable of detecting a scam telephone call in a real time, and issuing a warning to a user.SOLUTION: A system according to an embodiment, comprises a voice recognition part, a summary part, a determination part, a warning part, and a contact part. The voice recognition part converts a telephone conversation content into a text. The summary part summarizes the conversation content obtained by being converted into the text by the voice recognition part. The determination part verifies the content summarized by the summary part, and determines a possibility of a scam telephone call. The warning part issues a warning to a telephone user in the case where it is determined that the telephone content is the scam telephone call by the determination part. The contact part makes contact with a participant in the case where it is determined that the telephone content is the scam telephone call by the determination part.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology has the problem that it is difficult to detect fraudulent calls in real time and warn users.

[0005] The system according to the embodiment aims to detect fraudulent calls in real time and to warn users. [Means for solving the problem]

[0006] The system according to the embodiment includes a voice recognition unit, a summarization unit, a determination unit, a warning unit, and a contact unit. The voice recognition unit converts the contents of a telephone conversation into text. The summarization unit summarizes the contents of the conversation converted into text by the voice recognition unit. The determination unit compares the contents summarized by the summarization unit with a determination condition and determines whether the call is likely to be a fraudulent call. The warning unit warns the telephone user when the determination unit determines that the call is a fraudulent call. The contact unit contacts relevant parties when the determination unit determines that the call is a fraudulent call. Effect of the Invention

[0007] The system according to the embodiment can detect fraudulent calls in real time and warn users. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The telephone conversation content analysis system according to the embodiment of the present invention is a system that converts telephone conversation content into text using voice recognition technology and generation AI, and summarizes the content. The telephone conversation content analysis system converts telephone conversation content into text using voice recognition technology, and the generation AI summarizes the texted conversation content. Furthermore, if the conversation content meets certain conditions such as "a sudden request for money" or "a transfer to an unknown account," it is determined to be a fraudulent call, and crime prevention actions such as warning the telephone user and contacting related parties are automatically taken. For example, if the telephone conversation content is "I need money urgently," the content is converted into text. Next, the generation AI summarizes the texted conversation content. The generation AI analyzes the texted conversation content, extracts important information, and summarizes it. For example, the conversation content "I need money urgently" is summarized as "a request for money." Furthermore, if the conversation content meets certain conditions such as "a sudden request for money" or "a transfer to an unknown account," it is determined to be a fraudulent call. The generation AI compares the summarized conversation content with the judgment criteria, and if it determines that there is a possibility of a fraudulent call, it automatically takes crime prevention action such as warning the phone user and contacting the relevant parties. For example, it warns the phone user that "there is a possibility of fraud" and contacts the relevant parties that "there has been a call that may be fraudulent." This system can be built into the phone itself, or a dedicated device can be installed near the landline phone. This allows phone users to reduce the risk of fraudulent calls and use the phone safely. In this way, the phone conversation content analysis system can automatically analyze the content of phone conversations and reduce the risk of fraudulent calls.

[0029] The telephone conversation content analysis system according to the embodiment includes a voice recognition unit, a summarizing unit, a determination unit, a warning unit, and a contact unit. The voice recognition unit converts the contents of the telephone conversation into text. The voice recognition unit converts the contents of the telephone conversation into text in real time using, for example, a voice recognition technology. For example, the voice recognition unit converts the contents of the telephone conversation into text using a voice recognition algorithm. The summarizing unit uses a generation AI to summarize the contents of the conversation that have been converted into text by the voice recognition unit. The summarizing unit analyzes, for example, the contents of the conversation that have been converted into text using the generation AI, extracts important information, and summarizes it. For example, the generation AI summarizes the contents of the conversation using a text generation AI (for example, LLM). The determination unit checks the contents summarized by the summarizing unit against a determination condition and determines whether the call is likely to be fraudulent. For example, the determination unit checks the summarized contents of the conversation against a determination condition and determines whether the call is likely to be fraudulent. For example, the determination unit determines whether the summarized contents of the conversation meet a condition such as "a sudden request for money" or "a transfer to an unknown account." The warning unit warns the telephone user when the determination unit determines that there is a possibility of a fraudulent call. The warning unit, for example, warns the telephone user that "there is a possibility of fraud." The contact unit contacts the relevant party when the determination unit determines that there is a possibility of a fraudulent call. The contact unit, for example, contacts the relevant party by saying "You have received a possibly fraudulent call." In this way, the telephone conversation content analysis system according to the embodiment can automatically analyze the content of telephone conversations and reduce the risk of fraudulent calls.

[0030] The voice recognition unit converts the contents of the telephone conversation into text. The voice recognition unit converts the contents of the telephone conversation into text in real time, for example, using a voice recognition technology. Specifically, the voice recognition unit converts the contents of the telephone conversation into text accurately using an advanced voice recognition algorithm. This algorithm analyzes the voice signal, identifies phonemes and words, and converts them into text. The voice recognition unit can also use a noise reduction technology to eliminate background sounds and noise and clearly recognize the contents of the conversation. Furthermore, the voice recognition unit supports multiple languages ​​and dialects, and can accurately convert the conversation of users with different languages ​​and accents into text. Since the voice recognition unit converts the contents of the telephone conversation into text in real time, it instantly generates text data as the conversation progresses and provides it to a subsequent processing unit. As a result, the voice recognition unit can quickly and accurately convert the contents of the telephone conversation into text, improving the efficiency of the entire system.

[0031] The summarization unit uses a generation AI to summarize the conversation content that has been converted to text by the speech recognition unit. For example, the summarization unit analyzes the conversation content that has been converted to text by the generation AI, extracts important information, and summarizes it. Specifically, the summarization unit uses a large-scale language model (LLM) as the generation AI, analyzes the text data, and extracts important keywords and phrases. The generation AI makes full use of natural language processing technology to understand the context and intent of the conversation, and summarizes the main points by eliminating redundant parts. For example, it extracts important information such as "a sudden request for money" or "a transfer to an unknown account" from a long conversation and generates a concise summary. The summarization unit provides the generated summary to the judgment unit, which uses it as basic data for judging the possibility of a fraudulent call. This allows the summarization unit to efficiently process huge amounts of conversation data and quickly extract important information.

[0032] The determination unit compares the contents summarized by the summarization unit with the determination conditions to determine the possibility of a fraudulent call. Specifically, the determination unit compares the summarized conversation contents with the determination conditions set in advance to determine whether or not there is a possibility of a fraudulent call. The determination conditions include characteristic phrases and behavioral patterns common to fraudulent calls, such as "a sudden request for money" and "a transfer to an unknown account." The determination unit evaluates whether or not the summarized conversation contents are relevant based on these conditions. Furthermore, the determination unit increases the possibility of a fraudulent call by referring to a database of past fraudulent calls and detecting similar patterns. If the determination unit determines that there is a high possibility of a fraudulent call, it provides the information to the warning unit and the contact unit to prompt appropriate action. In this way, the determination unit can quickly and accurately evaluate the summarized conversation contents and reduce the risk of a fraudulent call.

[0033] The warning unit warns the telephone user when the determination unit determines that the call is likely to be a fraudulent call. Specifically, the warning unit warns the telephone user that "there is a possibility of fraud." The warning unit can, for example, display a warning message on the telephone display or issue a warning by voice. Furthermore, the warning unit has a function of repeatedly issuing a warning so that the user does not ignore the warning. For example, the warning message is redisplayed at regular intervals or a warning is issued again by voice. Furthermore, the warning unit can check whether the user has accepted the warning and provide additional instructions as necessary. In this way, the warning unit can promptly and reliably warn the user and prevent damage from fraudulent calls.

[0034] The contact unit contacts the relevant party when the determination unit determines that the call may be fraudulent. Specifically, the contact unit notifies the relevant party that "there has been a call that may be fraudulent." The contact unit contacts the relevant party by means of, for example, email, SMS, telephone, or the like. Furthermore, the contact unit has a function for recording the contact details so that the contact can be referenced later. For example, the contact date and time, contact details, contact information, etc. are stored in a database so that they can be searched and referenced as needed. Furthermore, the contact unit can accept feedback from the relevant party and use it to improve the contact details and optimize the system. This allows the contact unit to quickly and reliably contact the relevant party and reduce the risk of fraudulent calls.

[0035] The determination unit may include a means for updating the determination conditions. The determination unit may update the determination conditions using, for example, a machine learning algorithm. For example, the determination unit may learn past fraudulent call data and update the determination conditions to accommodate new fraud methods. The determination unit may also update the determination conditions using a database update method. For example, the determination unit may periodically update the database and add determination conditions to accommodate new fraud methods. This allows the determination unit to respond to new types of fraud by updating the determination conditions.

[0036] The voice recognition unit can instantly convert the contents of a telephone conversation into text. The voice recognition unit can instantly convert the contents of a telephone conversation into text, for example, by using real-time voice recognition technology. For example, the voice recognition unit can instantly convert the contents of a telephone conversation into text by using low-latency processing. The voice recognition unit can also instantly convert the contents of a telephone conversation into text by using a voice recognition algorithm. For example, the voice recognition unit can instantly convert the contents of a telephone conversation into text by using a voice recognition algorithm. This allows the voice recognition unit to instantly convert the contents of a telephone conversation into text, enabling a quick response.

[0037] The summarization unit can analyze the text-converted conversation content, and extract and summarize information based on specific criteria. The summarization unit can analyze the text-converted conversation content, for example, using a generation AI, and extract and summarize information based on specific criteria. For example, the summarization unit can extract and summarize information based on the frequency of keywords and the importance of context using a generation AI. The summarization unit can also extract and summarize information based on the length of a sentence and the importance of the information to be summarized using a generation AI. For example, the summarization unit can analyze the text-converted conversation content, and extract and summarize important information using a generation AI. As a result, the summarization unit can extract and summarize important information, enabling efficient information processing.

[0038] The warning unit can warn the telephone user when it is determined that the call is a fraudulent call. The warning unit can, for example, issue a voice warning to the telephone user. For example, the warning unit can warn the telephone user by voice, "This may be a fraud." The warning unit can also issue a warning to the telephone user by a text message. For example, the warning unit can warn the telephone user by a text message, "This may be a fraud." In this way, the warning unit can alert the user by issuing a warning when there is a possibility of a fraudulent call, thereby attracting the user's attention.

[0039] The communication unit can contact the relevant person when it is determined that the call is a fraudulent call. The communication unit can, for example, contact the relevant person by telephone. For example, the communication unit can contact the relevant person by telephone, saying, "You have received a call that may be a fraudulent call." The communication unit can also contact the relevant person by email. For example, the communication unit can contact the relevant person by email, saying, "You have received a call that may be a fraudulent call." The communication unit can also contact the relevant person by SMS. For example, the communication unit can contact the relevant person by SMS, saying, "You have received a call that may be a fraudulent call." This allows the communication unit to contact the relevant person when there is a possibility that the call is a fraudulent call, enabling a quick response.

[0040] The voice recognition unit can add a function of removing background noise during voice recognition. The voice recognition unit can remove background noise during voice recognition, for example, by using noise canceling technology. For example, the voice recognition unit can filter background noise occurring during a conversation in real time and convert clear voice into text. The voice recognition unit can also extract voice using noise canceling technology when recognizing a conversation in a noisy environment. Furthermore, the voice recognition unit can also apply a noise reduction algorithm before voice recognition to reduce background sound. For example, the voice recognition unit can reduce background sound using a noise reduction algorithm to improve the accuracy of voice recognition. As a result, the voice recognition unit can improve the accuracy of voice recognition by removing background noise.

[0041] The voice recognition unit can analyze the characteristics of the speaker's voice during voice recognition and apply an individually adjusted voice recognition model. The voice recognition unit can analyze the characteristics of the speaker's voice using, for example, voice waveform analysis. For example, the voice recognition unit can analyze the tone and pitch of the speaker's voice and select an optimal voice recognition model. The voice recognition unit can also analyze the characteristics of the speaker's voice using voiceprint recognition technology. For example, the voice recognition unit can learn the pronunciation habits and accent of the speaker and apply an individually optimized voice recognition model. Furthermore, the voice recognition unit can register the characteristics of the speaker's voice in advance and improve the recognition accuracy. For example, the voice recognition unit can register the characteristics of the speaker's voice in a database and improve the recognition accuracy by referring to the data during voice recognition. As a result, the voice recognition unit optimizes the voice recognition model based on the characteristics of the speaker's voice, thereby improving the recognition accuracy.

[0042] The speech recognition unit can improve the recognition accuracy by considering the geographical accent and dialect of the speaker during speech recognition. The speech recognition unit can consider the geographical accent and dialect of the speaker by using, for example, a speech database for each region. For example, the speech recognition unit can learn the geographical accent of the speaker in advance and reflect it in the speech recognition model. The speech recognition unit can also analyze the characteristics of the speaker's dialect by using a dialect dictionary and optimize the speech recognition model. For example, the speech recognition unit can improve the accuracy of speech recognition by considering differences in pronunciation for each region. As a result, the speech recognition unit improves the accuracy of speech recognition by considering geographical accents and dialects.

[0043] During speech recognition, the speech recognition unit can improve the recognition accuracy by referring to the speaker's past conversation history. The speech recognition unit can, for example, use a database to refer to the speaker's past conversation history. For example, the speech recognition unit can analyze the speaker's past conversation history and preferentially recognize specific phrases or words. The speech recognition unit can also learn the speaker's pronunciation habits from the past conversation history to improve the recognition accuracy. For example, the speech recognition unit can customize a speech recognition model based on the speaker's past conversation history. As a result, the speech recognition unit can improve the accuracy of speech recognition by referring to the past conversation history.

[0044] The summarization unit can adjust the level of detail of the summary based on the importance of the conversation when generating a summary. The summarization unit can evaluate the importance of the conversation based on, for example, the frequency of keywords or the importance of the context. For example, the summarization unit can generate a detailed summary in the case of important conversation content. Furthermore, the summarization unit can generate a concise summary in the case of general conversation content. Furthermore, the summarization unit can generate a summary in which only the main points are extracted in the case of low-importance conversation content. For example, the summarization unit can evaluate the importance of the conversation based on the frequency of keywords or the importance of the context, and adjust the level of detail of the summary based on the evaluation result. As a result, the summarization unit can adjust the level of detail of the summary based on the importance of the conversation, thereby enabling efficient information processing.

[0045] The summarization unit can apply different summarization algorithms depending on the category of the conversation when generating a summary. The summarization unit can classify the category of the conversation using, for example, a topic classification algorithm. For example, in the case of a business conversation, the summarization unit can generate a detailed summary including technical terms. In addition, in the case of a private conversation, the summarization unit can also generate a concise and easy-to-understand summary. Furthermore, in the case of an urgent conversation, the summarization unit can quickly generate a summary that extracts the main points. For example, the summarization unit can classify the category of the conversation using a topic classification algorithm and apply different summarization algorithms depending on the category. In this way, the summarization unit can apply a summarization algorithm depending on the category of the conversation, thereby improving the accuracy of the summary.

[0046] The summarization unit can determine the priority of summaries based on the time periods of conversations when generating summaries. The summarization unit can, for example, consider the time periods of conversations using an importance evaluation for each time period. For example, the summarization unit can generate summaries by prioritizing time periods in which important conversations took place. The summarization unit can also generate summaries with normal priority for time periods in which general conversations took place. Furthermore, the summarization unit can generate summaries by putting time periods in which less important conversations took place on a later date. For example, the summarization unit can consider the time periods of conversations using an importance evaluation for each time period and determine the priority of summaries based on the evaluation results. This enables efficient information processing by the summarization unit determining the priority of summaries based on the time periods of conversations.

[0047] The summarization unit can adjust the order of summaries based on the relevance of the conversation when generating summaries. The summarization unit can evaluate the relevance of the conversation using, for example, topic modeling. For example, the summarization unit can summarize important conversation contents first and prioritize highly relevant information. The summarization unit can also summarize general conversation contents in the middle and postpone less relevant information. Furthermore, the summarization unit can summarize less important conversation contents last and postpone less relevant information. For example, the summarization unit can evaluate the relevance of the conversation using topic modeling and adjust the order of summaries based on the evaluation result. As a result, the summarization unit can adjust the order of summaries based on the relevance of the conversation, thereby enabling efficient information processing.

[0048] The judgment unit can optimize the judgment algorithm by referring to past fraudulent call data when making a judgment. The judgment unit can, for example, use a database to refer to past fraudulent call data. For example, the judgment unit can analyze past fraudulent call data, learn fraud patterns, and reflect the patterns in the judgment algorithm. In addition, when a new fraud method is discovered, the judgment unit can update the database and optimize the judgment algorithm. Furthermore, the judgment unit can identify elements that are likely to be fraud based on past fraudulent call data and reflect the elements in the judgment. For example, the judgment unit can identify elements that are likely to be fraud based on past fraudulent call data, and reflect the elements in the judgment algorithm. As a result, the judgment unit can improve the accuracy of the judgment algorithm by referring to past fraudulent call data.

[0049] At the time of judgment, the judgment unit can identify elements that increase the possibility of fraud by considering the context of the conversation. The judgment unit can, for example, consider the context of the conversation using a context analysis algorithm. For example, the judgment unit can analyze the context of the conversation and identify phrases or words that are likely to be fraudulent. The judgment unit can also analyze the flow of the conversation and issue a warning when there are signs of fraud. Furthermore, the judgment unit can identify elements that are likely to be fraudulent based on the context of the conversation and reflect them in the judgment. For example, the judgment unit can analyze the context of the conversation using a context analysis algorithm and identify elements that are likely to be fraudulent based on the analysis result. This makes it easier for the judgment unit to identify elements that increase the possibility of fraud by considering the context of the conversation.

[0050] At the time of the determination, the determination unit can determine the possibility of fraud by taking into account the geographical information of the conversation. The determination unit can, for example, use a location information database to consider the geographical information of the conversation. For example, the determination unit can identify an area where fraud is highly likely based on the geographical information of the conversation. The determination unit can also identify an element where fraud is highly likely based on the geographical information. Furthermore, the determination unit can preferentially determine a conversation in an area where fraud is highly likely based on the geographical information. For example, the determination unit can consider the geographical information of the conversation using a location information database and identify an area where fraud is highly likely based on the information. This allows the determination unit to more accurately determine the possibility of fraud by taking into account the geographical information.

[0051] The determination unit can improve the accuracy of the determination by referring to literature related to the conversation at the time of determination. The determination unit can, for example, use a literature database to refer to literature related to the conversation. For example, the determination unit can compare the content of the conversation with the related literature to determine the possibility of fraud. The determination unit can also identify elements that are likely to be fraudulent based on the related literature. Furthermore, the determination unit can optimize the determination algorithm by referring to the related literature. For example, the determination unit can refer to literature related to the conversation using a literature database and identify elements that are likely to be fraudulent based on the information. Thereby, the determination unit can improve the accuracy of the determination by referring to the related literature.

[0052] The warning unit can select the most appropriate warning method by referring to the user's past reaction history at the time of warning. The warning unit can, for example, use a database to refer to the user's past reaction history. For example, the warning unit can preferentially use a warning method to which the user responded quickly in the past. The warning unit can also try a new warning method, avoiding warning methods that the user has ignored in the past. Furthermore, the warning unit can analyze the user's past reaction history and select the most effective warning method. For example, the warning unit can use a database to refer to the user's past reaction history and select the most appropriate warning method based on that information. In this way, the warning unit can select the most appropriate warning method by referring to the past reaction history.

[0053] The warning unit can adjust the timing of the warning based on the user's current situation at the time of warning. The warning unit can, for example, use real-time data analysis to evaluate the user's current situation. For example, the warning unit can issue a warning at an important timing when the user is busy. Also, the warning unit can issue a warning at a normal timing when the user is relaxed. Furthermore, the warning unit can adjust the timing to issue a warning quickly when the user is in a hurry. For example, the warning unit can evaluate the user's current situation using real-time data analysis and adjust the timing of the warning based on the information. Thereby, the warning unit can adjust the timing of the warning according to the user's situation, thereby improving the effectiveness of the warning.

[0054] The warning unit can select the optimal warning method in consideration of the geographical information of the user at the time of warning. The warning unit can consider the geographical information of the user by using, for example, a location information database. For example, when the user is at home, the warning unit can preferentially use a voice warning. Also, when the user is out, the warning unit can issue a warning by a text message. Furthermore, the warning unit can select the optimal warning method based on the geographical information of the user. For example, the warning unit can consider the geographical information of the user by using a location information database, and select the optimal warning method based on the information. As a result, the warning unit can select the optimal warning method by considering the geographical information.

[0055] At the time of issuing a warning, the warning unit can analyze the user's social media activity and suggest a warning means. The warning unit can, for example, analyze the user's social media activity using an analysis of the social media activity. For example, the warning unit can issue a warning through a social media platform that the user frequently uses. The warning unit can also analyze the user's social media activity and suggest an optimal warning means. Furthermore, the warning unit can select a warning means based on information shared by the user on social media. For example, the warning unit can analyze the user's social media activity using an analysis of the social media activity and suggest an optimal warning means based on the information. In this way, the warning unit can suggest an optimal warning means by analyzing the social media activity.

[0056] When making contact, the contact unit can select the optimal contact method by referring to the past response history of the relevant person. The contact unit can, for example, use a database to refer to the past response history of the relevant person. For example, the contact unit can preferentially use a contact method to which the relevant person has responded quickly in the past. The contact unit can also try a new contact method, avoiding contact methods that the relevant person has ignored in the past. Furthermore, the contact unit can analyze the past response history of the relevant person and select the most effective contact method. For example, the contact unit can use a database to refer to the past response history of the relevant person and select the optimal contact method based on that information. In this way, the contact unit can select the optimal contact method by referring to the past response history.

[0057] The communication unit can adjust the timing of the contact based on the current situation of the relevant person at the time of contact. The communication unit can, for example, use real-time data analysis to evaluate the current situation of the relevant person. For example, the communication unit can contact the relevant person at an important timing when the relevant person is busy. Also, the communication unit can contact the relevant person at a normal timing when the relevant person is relaxed. Furthermore, the communication unit can adjust the timing to contact the relevant person quickly when the relevant person is in a hurry. For example, the communication unit can evaluate the current situation of the relevant person using real-time data analysis and adjust the timing of the contact based on the information. Thereby, the communication unit can adjust the timing of the contact according to the situation of the relevant person, thereby improving the effectiveness of the contact.

[0058] The contact unit can select the optimal contact method when making contact by taking into consideration geographical information of the relevant person. The contact unit can, for example, use a location information database to take into consideration the geographical information of the relevant person. For example, the contact unit can preferentially use voice contact when the relevant person is at home. Also, the contact unit can contact the relevant person by text message when the relevant person is out. Furthermore, the contact unit can select the optimal contact method based on the geographical information of the relevant person. For example, the contact unit can consider geographical information of the relevant person using a location information database and select the optimal contact method based on that information. In this way, the contact unit can select the optimal contact method by taking into consideration geographical information.

[0059] At the time of contact, the contact department can analyze the social media activity of the relevant person and suggest a means of contact. The contact department can, for example, analyze the social media activity of the relevant person using an analysis of social media activity. For example, the contact department can contact the relevant person through a social media platform that the relevant person frequently uses. The contact department can also analyze the social media activity of the relevant person and suggest the optimal means of contact. Furthermore, the contact department can select a means of contact based on information shared by the relevant person on social media. For example, the contact department can analyze the social media activity of the relevant person using an analysis of social media activity and suggest the optimal means of contact based on the information. In this way, the contact department can suggest the optimal means of contact by analyzing social media activity.

[0060] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0061] The telephone conversation content analysis system may further include a translation unit. The translation unit may translate the textualized conversation content into a plurality of languages. For example, the translation unit may support major languages ​​such as English, Japanese, and Chinese. The translation unit may also perform natural translation taking into account the context of the conversation content. Furthermore, the translation unit may also appropriately translate technical terms and expressions specific to an industry. This allows the telephone conversation content analysis system to be compatible with international users and facilitate communication between different languages.

[0062] The determination unit can improve the accuracy of the determination by referring to the user's past behavior history. For example, the determination unit can record in a database what kind of calls the user has received in the past and how the user responded, and make a determination based on that information. The determination unit can also learn the user's behavior patterns and more accurately determine calls that are likely to be fraudulent. Furthermore, the determination unit can strengthen the warning when there are signs of fraud based on the user's past behavior history. In this way, the determination unit can improve the accuracy of determining fraudulent calls by utilizing the user's behavior history.

[0063] The voice recognition unit can learn the characteristics of a user's voice and apply an individually optimized voice recognition model. For example, the voice recognition unit can learn the tone, pitch, and pronunciation habits of a user's voice and customize the voice recognition model based on that information. The voice recognition unit can also register the characteristics of a user's voice in a database and improve recognition accuracy by referring to that data during voice recognition. Furthermore, the voice recognition unit can also learn the characteristics of the voices of multiple users simultaneously and apply a voice recognition model optimized for each user. As a result, the voice recognition unit can optimize the voice recognition model based on the characteristics of the user's voice, thereby improving recognition accuracy.

[0064] The speech recognition unit can improve the accuracy of speech recognition by taking into account the geographical accent or dialect of the speaker during speech recognition. For example, the speech recognition unit can take into account the geographical accent or dialect of the speaker by using a speech database for each region. The speech recognition unit can also analyze the characteristics of the speaker's dialect by using a dialect dictionary and optimize the speech recognition model. Furthermore, the speech recognition unit can improve the accuracy of speech recognition by taking into account differences in pronunciation for each region. As a result, the speech recognition unit improves the accuracy of speech recognition by taking into account geographical accents and dialects.

[0065] At the time of judgment, the judgment unit can identify elements that increase the possibility of fraud by considering the context of the conversation. For example, the judgment unit can analyze the context of the conversation and identify phrases or words that are likely to be fraudulent. The judgment unit can also analyze the flow of the conversation and issue a warning if there are signs of fraud. Furthermore, the judgment unit can identify elements that are likely to be fraudulent based on the context of the conversation and reflect them in the judgment. This makes it easier for the judgment unit to identify elements that increase the possibility of fraud by considering the context of the conversation.

[0066] The summarization unit can apply different summarization algorithms depending on the category of the conversation when generating a summary. For example, in the case of a business conversation, the summarization unit can generate a detailed summary including technical terms. In addition, in the case of a private conversation, the summarization unit can also generate a concise and easy-to-understand summary. Furthermore, in the case of an urgent conversation, the summarization unit can quickly generate a summary that extracts the main points. In this way, the summarization unit can apply a summarization algorithm depending on the category of the conversation, thereby improving the accuracy of the summary.

[0067] When issuing a warning, the warning unit may analyze the user's social media activity and suggest a warning method. For example, the warning unit may issue a warning through a social media platform that the user frequently uses. The warning unit may also analyze the user's social media activity and suggest an optimal warning method. Furthermore, the warning unit may select a warning method based on information shared by the user on social media. In this way, the warning unit may suggest an optimal warning method by analyzing social media activity.

[0068] The process flow of the first embodiment will be briefly described below.

[0069] Step 1: The voice recognition unit converts the contents of the telephone conversation into text. The voice recognition unit converts the contents of the telephone conversation into text in real time using voice recognition technology. For example, the voice recognition unit converts the contents of the telephone conversation into text using a voice recognition algorithm. Step 2: The summarization unit uses a generation AI to summarize the conversation content that has been converted to text by the speech recognition unit. The summarization unit analyzes the converted conversation content, extracts important information, and summarizes it. For example, the conversation content is summarized using a text generation AI (e.g., LLM). Step 3: The judgment unit checks the contents summarized by the summarization unit against the judgment criteria to determine whether it is a fraudulent call. The judgment unit determines whether the summarized conversation content meets the criteria, such as "a sudden request for money" or "a transfer to an unknown account." Step 4: If the determination unit determines that the call is likely to be a fraudulent call, the warning unit warns the telephone user. For example, the warning unit warns the telephone user that "there is a possibility of fraud." Step 5: If the judgment unit judges that the call may be fraudulent, the communication unit contacts the relevant person. For example, the communication unit may inform the relevant person that "there has been a call that may be fraudulent."

[0070] (Example 2) The telephone conversation content analysis system according to the embodiment of the present invention is a system that converts telephone conversation content into text using voice recognition technology and generation AI, and summarizes the content. The telephone conversation content analysis system converts telephone conversation content into text using voice recognition technology, and the generation AI summarizes the texted conversation content. Furthermore, if the conversation content meets certain conditions such as "a sudden request for money" or "a transfer to an unknown account," it is determined to be a fraudulent call, and crime prevention actions such as warning the telephone user and contacting related parties are automatically taken. For example, if the telephone conversation content is "I need money urgently," the content is converted into text. Next, the generation AI summarizes the texted conversation content. The generation AI analyzes the texted conversation content, extracts important information, and summarizes it. For example, the conversation content "I need money urgently" is summarized as "a request for money." Furthermore, if the conversation content meets certain conditions such as "a sudden request for money" or "a transfer to an unknown account," it is determined to be a fraudulent call. The generation AI compares the summarized conversation content with the judgment criteria, and if it determines that there is a possibility of a fraudulent call, it automatically takes crime prevention action such as warning the phone user and contacting the relevant parties. For example, it warns the phone user that "there is a possibility of fraud" and contacts the relevant parties that "there has been a call that may be fraudulent." This system can be built into the phone itself, or a dedicated device can be installed near the landline phone. This allows phone users to reduce the risk of fraudulent calls and use the phone safely. In this way, the phone conversation content analysis system can automatically analyze the content of phone conversations and reduce the risk of fraudulent calls.

[0071] The telephone conversation content analysis system according to the embodiment includes a voice recognition unit, a summarizing unit, a determination unit, a warning unit, and a contact unit. The voice recognition unit converts the contents of the telephone conversation into text. The voice recognition unit converts the contents of the telephone conversation into text in real time using, for example, a voice recognition technology. For example, the voice recognition unit converts the contents of the telephone conversation into text using a voice recognition algorithm. The summarizing unit uses a generation AI to summarize the contents of the conversation that have been converted into text by the voice recognition unit. The summarizing unit analyzes, for example, the contents of the conversation that have been converted into text using the generation AI, extracts important information, and summarizes it. For example, the generation AI summarizes the contents of the conversation using a text generation AI (for example, LLM). The determination unit checks the contents summarized by the summarizing unit against a determination condition and determines whether the call is likely to be fraudulent. For example, the determination unit checks the summarized contents of the conversation against a determination condition and determines whether the call is likely to be fraudulent. For example, the determination unit determines whether the summarized contents of the conversation meet a condition such as "a sudden request for money" or "a transfer to an unknown account." The warning unit warns the telephone user when the determination unit determines that there is a possibility of a fraudulent call. The warning unit, for example, warns the telephone user that "there is a possibility of fraud." The contact unit contacts the relevant party when the determination unit determines that there is a possibility of a fraudulent call. The contact unit, for example, contacts the relevant party by saying "You have received a possibly fraudulent call." In this way, the telephone conversation content analysis system according to the embodiment can automatically analyze the content of telephone conversations and reduce the risk of fraudulent calls.

[0072] The voice recognition unit converts the contents of the telephone conversation into text. The voice recognition unit converts the contents of the telephone conversation into text in real time, for example, using a voice recognition technology. Specifically, the voice recognition unit converts the contents of the telephone conversation into text accurately using an advanced voice recognition algorithm. This algorithm analyzes the voice signal, identifies phonemes and words, and converts them into text. The voice recognition unit can also use a noise reduction technology to eliminate background sounds and noise and clearly recognize the contents of the conversation. Furthermore, the voice recognition unit supports multiple languages ​​and dialects, and can accurately convert the conversation of users with different languages ​​and accents into text. Since the voice recognition unit converts the contents of the telephone conversation into text in real time, it instantly generates text data as the conversation progresses and provides it to a subsequent processing unit. As a result, the voice recognition unit can quickly and accurately convert the contents of the telephone conversation into text, improving the efficiency of the entire system.

[0073] The summarization unit uses a generation AI to summarize the conversation content that has been converted to text by the speech recognition unit. For example, the summarization unit analyzes the conversation content that has been converted to text by the generation AI, extracts important information, and summarizes it. Specifically, the summarization unit uses a large-scale language model (LLM) as the generation AI, analyzes the text data, and extracts important keywords and phrases. The generation AI makes full use of natural language processing technology to understand the context and intent of the conversation, and summarizes the main points by eliminating redundant parts. For example, it extracts important information such as "a sudden request for money" or "a transfer to an unknown account" from a long conversation and generates a concise summary. The summarization unit provides the generated summary to the judgment unit, which uses it as basic data for judging the possibility of a fraudulent call. This allows the summarization unit to efficiently process huge amounts of conversation data and quickly extract important information.

[0074] The determination unit compares the contents summarized by the summarization unit with the determination conditions to determine the possibility of a fraudulent call. Specifically, the determination unit compares the summarized conversation contents with the determination conditions set in advance to determine whether or not there is a possibility of a fraudulent call. The determination conditions include characteristic phrases and behavioral patterns common to fraudulent calls, such as "a sudden request for money" and "a transfer to an unknown account." The determination unit evaluates whether or not the summarized conversation contents are relevant based on these conditions. Furthermore, the determination unit increases the possibility of a fraudulent call by referring to a database of past fraudulent calls and detecting similar patterns. If the determination unit determines that there is a high possibility of a fraudulent call, it provides the information to the warning unit and the contact unit to prompt appropriate action. In this way, the determination unit can quickly and accurately evaluate the summarized conversation contents and reduce the risk of a fraudulent call.

[0075] The warning unit warns the telephone user when the determination unit determines that the call is likely to be a fraudulent call. Specifically, the warning unit warns the telephone user that "there is a possibility of fraud." The warning unit can, for example, display a warning message on the telephone display or issue a warning by voice. Furthermore, the warning unit has a function of repeatedly issuing a warning so that the user does not ignore the warning. For example, the warning message is redisplayed at regular intervals or a warning is issued again by voice. Furthermore, the warning unit can check whether the user has accepted the warning and provide additional instructions as necessary. In this way, the warning unit can promptly and reliably warn the user and prevent damage from fraudulent calls.

[0076] The contact unit contacts the relevant party when the determination unit determines that the call may be fraudulent. Specifically, the contact unit notifies the relevant party that "there has been a call that may be fraudulent." The contact unit contacts the relevant party by means of, for example, email, SMS, telephone, or the like. Furthermore, the contact unit has a function for recording the contact details so that the contact can be referenced later. For example, the contact date and time, contact details, contact information, etc. are stored in a database so that they can be searched and referenced as needed. Furthermore, the contact unit can accept feedback from the relevant party and use it to improve the contact details and optimize the system. This allows the contact unit to quickly and reliably contact the relevant party and reduce the risk of fraudulent calls.

[0077] The determination unit may include a means for updating the determination conditions. The determination unit may update the determination conditions using, for example, a machine learning algorithm. For example, the determination unit may learn past fraudulent call data and update the determination conditions to accommodate new fraud methods. The determination unit may also update the determination conditions using a database update method. For example, the determination unit may periodically update the database and add determination conditions to accommodate new fraud methods. This allows the determination unit to respond to new types of fraud by updating the determination conditions.

[0078] The voice recognition unit can instantly convert the contents of a telephone conversation into text. The voice recognition unit can instantly convert the contents of a telephone conversation into text, for example, by using real-time voice recognition technology. For example, the voice recognition unit can instantly convert the contents of a telephone conversation into text by using low-latency processing. The voice recognition unit can also instantly convert the contents of a telephone conversation into text by using a voice recognition algorithm. For example, the voice recognition unit can instantly convert the contents of a telephone conversation into text by using a voice recognition algorithm. This allows the voice recognition unit to instantly convert the contents of a telephone conversation into text, enabling a quick response.

[0079] The summarization unit can analyze the text-converted conversation content, and extract and summarize information based on specific criteria. The summarization unit can analyze the text-converted conversation content, for example, using a generation AI, and extract and summarize information based on specific criteria. For example, the summarization unit can extract and summarize information based on the frequency of keywords and the importance of context using a generation AI. The summarization unit can also extract and summarize information based on the length of a sentence and the importance of the information to be summarized using a generation AI. For example, the summarization unit can analyze the text-converted conversation content, and extract and summarize important information using a generation AI. As a result, the summarization unit can extract and summarize important information, enabling efficient information processing.

[0080] The warning unit can warn the telephone user when it is determined that the call is a fraudulent call. The warning unit can, for example, issue a voice warning to the telephone user. For example, the warning unit can warn the telephone user by voice, "This may be a fraud." The warning unit can also issue a warning to the telephone user by a text message. For example, the warning unit can warn the telephone user by a text message, "This may be a fraud." In this way, the warning unit can alert the user by issuing a warning when there is a possibility of a fraudulent call, thereby attracting the user's attention.

[0081] The communication unit can contact the relevant person when it is determined that the call is a fraudulent call. The communication unit can, for example, contact the relevant person by telephone. For example, the communication unit can contact the relevant person by telephone, saying, "You have received a call that may be a fraudulent call." The communication unit can also contact the relevant person by email. For example, the communication unit can contact the relevant person by email, saying, "You have received a call that may be a fraudulent call." The communication unit can also contact the relevant person by SMS. For example, the communication unit can contact the relevant person by SMS, saying, "You have received a call that may be a fraudulent call." This allows the communication unit to contact the relevant person when there is a possibility that the call is a fraudulent call, enabling a quick response.

[0082] The voice recognition unit can analyze the user's emotions and adjust the accuracy of the voice recognition based on the analyzed user's emotions. The voice recognition unit can analyze the user's emotions, for example, using voice tone analysis. For example, the voice recognition unit can analyze the tone of the user's voice and determine whether the user is nervous or relaxed. The voice recognition unit can also analyze the user's emotions using facial expression recognition technology. For example, the voice recognition unit can analyze the user's facial expression and determine whether the user is nervous or relaxed. Furthermore, the voice recognition unit can adjust the accuracy of the voice recognition based on the analyzed user's emotions. For example, if the user is nervous, the sensitivity of the voice recognition can be increased to convert the voice into text more accurately. Furthermore, if the user is relaxed, the sensitivity of the voice recognition can be set to normal to convert natural conversation into text. Furthermore, if the user is in a hurry, the speed of the voice recognition can be increased to convert the voice into text quickly. As a result, the voice recognition unit adjusts the accuracy of the voice recognition according to the user's emotions, thereby improving the recognition accuracy. The estimation of emotions is realized using an emotion estimation function, for example, using an emotion engine or a generation AI. Examples of the generation AI include, but are not limited to, a text generation AI (eg, LLM) and a multimodal generation AI.

[0083] The voice recognition unit can add a function of removing background noise during voice recognition. The voice recognition unit can remove background noise during voice recognition, for example, by using noise canceling technology. For example, the voice recognition unit can filter background noise occurring during a conversation in real time and convert clear voice into text. The voice recognition unit can also extract voice using noise canceling technology when recognizing a conversation in a noisy environment. Furthermore, the voice recognition unit can also apply a noise reduction algorithm before voice recognition to reduce background sound. For example, the voice recognition unit can reduce background sound using a noise reduction algorithm to improve the accuracy of voice recognition. As a result, the voice recognition unit can improve the accuracy of voice recognition by removing background noise.

[0084] The voice recognition unit can analyze the characteristics of the speaker's voice during voice recognition and apply an individually adjusted voice recognition model. The voice recognition unit can analyze the characteristics of the speaker's voice using, for example, voice waveform analysis. For example, the voice recognition unit can analyze the tone and pitch of the speaker's voice and select an optimal voice recognition model. The voice recognition unit can also analyze the characteristics of the speaker's voice using voiceprint recognition technology. For example, the voice recognition unit can learn the pronunciation habits and accent of the speaker and apply an individually optimized voice recognition model. Furthermore, the voice recognition unit can register the characteristics of the speaker's voice in advance and improve the recognition accuracy. For example, the voice recognition unit can register the characteristics of the speaker's voice in a database and improve the recognition accuracy by referring to the data during voice recognition. As a result, the voice recognition unit optimizes the voice recognition model based on the characteristics of the speaker's voice, thereby improving the recognition accuracy.

[0085] The voice recognition unit can analyze the user's emotions and adjust the timing of voice recognition based on the analyzed user's emotions. The voice recognition unit can analyze the user's emotions using, for example, voice tone analysis. For example, the voice recognition unit can analyze the tone of the user's voice and determine whether the user is nervous or relaxed. The voice recognition unit can also analyze the user's emotions using facial expression recognition technology. For example, the voice recognition unit can analyze the user's facial expression and determine whether the user is nervous or relaxed. Furthermore, the voice recognition unit can adjust the timing of voice recognition based on the analyzed user's emotions. For example, if the user is nervous, the timing of voice recognition can be delayed to encourage the user to speak slowly. Furthermore, if the user is relaxed, voice recognition can be performed at normal timing. Furthermore, if the user is in a hurry, the timing of voice recognition can be advanced to quickly convert the speech into text. As a result, the voice recognition unit adjusts the timing of voice recognition according to the user's emotions, thereby improving recognition accuracy. Estimation of emotions is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. Examples of the generation AI include, but are not limited to, a text generation AI (eg, LLM) and a multimodal generation AI.

[0086] The speech recognition unit can improve the recognition accuracy by considering the geographical accent and dialect of the speaker during speech recognition. The speech recognition unit can consider the geographical accent and dialect of the speaker by using, for example, a speech database for each region. For example, the speech recognition unit can learn the geographical accent of the speaker in advance and reflect it in the speech recognition model. The speech recognition unit can also analyze the characteristics of the speaker's dialect by using a dialect dictionary and optimize the speech recognition model. For example, the speech recognition unit can improve the accuracy of speech recognition by considering differences in pronunciation for each region. As a result, the speech recognition unit improves the accuracy of speech recognition by considering geographical accents and dialects.

[0087] During speech recognition, the speech recognition unit can improve the recognition accuracy by referring to the speaker's past conversation history. The speech recognition unit can, for example, use a database to refer to the speaker's past conversation history. For example, the speech recognition unit can analyze the speaker's past conversation history and preferentially recognize specific phrases or words. The speech recognition unit can also learn the speaker's pronunciation habits from the past conversation history to improve the recognition accuracy. For example, the speech recognition unit can customize a speech recognition model based on the speaker's past conversation history. As a result, the speech recognition unit can improve the accuracy of speech recognition by referring to the past conversation history.

[0088] The summarization unit can estimate the user's emotion and adjust the expression method of the summary based on the estimated user's emotion. The summarization unit can estimate the user's emotion using, for example, voice tone analysis. For example, the summarization unit can analyze the tone of the user's voice and determine whether the user is nervous or relaxed. The summarization unit can also estimate the user's emotion using facial expression recognition technology. For example, the summarization unit can analyze the user's facial expression and determine whether the user is nervous or relaxed. Furthermore, the summarization unit can adjust the expression method of the summary based on the estimated user's emotion. For example, if the user is nervous, a concise and easy-to-understand summary can be generated. Furthermore, if the user is relaxed, a summary including detailed information can be generated. Furthermore, if the user is in a hurry, a summary that extracts only important points can be generated. Thereby, the summarization unit adjusts the expression method of the summary according to the user's emotion, thereby improving understanding of the summary. The estimation of the emotion is realized using, for example, an emotion estimation function using an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0089] The summarization unit can adjust the level of detail of the summary based on the importance of the conversation when generating a summary. The summarization unit can evaluate the importance of the conversation based on, for example, the frequency of keywords or the importance of the context. For example, the summarization unit can generate a detailed summary in the case of important conversation content. Furthermore, the summarization unit can generate a concise summary in the case of general conversation content. Furthermore, the summarization unit can generate a summary in which only the main points are extracted in the case of low-importance conversation content. For example, the summarization unit can evaluate the importance of the conversation based on the frequency of keywords or the importance of the context, and adjust the level of detail of the summary based on the evaluation result. As a result, the summarization unit can adjust the level of detail of the summary based on the importance of the conversation, thereby enabling efficient information processing.

[0090] The summarization unit can apply different summarization algorithms depending on the category of the conversation when generating a summary. The summarization unit can classify the category of the conversation using, for example, a topic classification algorithm. For example, in the case of a business conversation, the summarization unit can generate a detailed summary including technical terms. In addition, in the case of a private conversation, the summarization unit can also generate a concise and easy-to-understand summary. Furthermore, in the case of an urgent conversation, the summarization unit can quickly generate a summary that extracts the main points. For example, the summarization unit can classify the category of the conversation using a topic classification algorithm and apply different summarization algorithms depending on the category. In this way, the summarization unit can apply a summarization algorithm depending on the category of the conversation, thereby improving the accuracy of the summary.

[0091] The summarization unit can estimate the user's emotion and adjust the length of the summary based on the estimated user's emotion. The summarization unit can estimate the user's emotion using, for example, voice tone analysis. For example, the summarization unit can analyze the tone of the user's voice and determine whether the user is nervous or relaxed. The summarization unit can also estimate the user's emotion using facial expression recognition technology. For example, the summarization unit can analyze the user's facial expression and determine whether the user is nervous or relaxed. Furthermore, the summarization unit can adjust the length of the summary based on the estimated user's emotion. For example, if the user is nervous, a short summary that is to the point can be generated. Furthermore, if the user is relaxed, a longer summary that includes detailed information can be generated. Furthermore, if the user is in a hurry, a short summary that extracts only important points can be generated. Thereby, the summarization unit adjusts the length of the summary according to the user's emotion, thereby improving understanding of the summary. The estimation of the emotion is realized using, for example, an emotion estimation function using an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0092] The summarization unit can determine the priority of summaries based on the time periods of conversations when generating summaries. The summarization unit can, for example, consider the time periods of conversations using an importance evaluation for each time period. For example, the summarization unit can generate summaries by prioritizing time periods in which important conversations took place. The summarization unit can also generate summaries with normal priority for time periods in which general conversations took place. Furthermore, the summarization unit can generate summaries by putting time periods in which less important conversations took place on a later date. For example, the summarization unit can consider the time periods of conversations using an importance evaluation for each time period and determine the priority of summaries based on the evaluation results. This enables efficient information processing by the summarization unit determining the priority of summaries based on the time periods of conversations.

[0093] The summarization unit can adjust the order of summaries based on the relevance of the conversation when generating summaries. The summarization unit can evaluate the relevance of the conversation using, for example, topic modeling. For example, the summarization unit can summarize important conversation contents first and prioritize highly relevant information. The summarization unit can also summarize general conversation contents in the middle and postpone less relevant information. Furthermore, the summarization unit can summarize less important conversation contents last and postpone less relevant information. For example, the summarization unit can evaluate the relevance of the conversation using topic modeling and adjust the order of summaries based on the evaluation result. As a result, the summarization unit can adjust the order of summaries based on the relevance of the conversation, thereby enabling efficient information processing.

[0094] The determination unit can estimate the user's emotion and adjust the judgment criteria based on the estimated user's emotion. The determination unit can estimate the user's emotion using, for example, voice tone analysis. For example, the determination unit can analyze the tone of the user's voice and determine whether the user is nervous or relaxed. The determination unit can also estimate the user's emotion using facial expression recognition technology. For example, the determination unit can analyze the user's facial expression and determine whether the user is nervous or relaxed. Furthermore, the determination unit can adjust the judgment criteria based on the estimated user's emotion. For example, if the user is nervous, strict judgment criteria can be applied to increase the possibility of fraud. Furthermore, if the user is relaxed, normal judgment criteria can be applied. Furthermore, if the user is in a hurry, the criteria can be adjusted to make a quick judgment. Thereby, the determination unit adjusts the judgment criteria according to the user's emotion, thereby improving the judgment accuracy. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0095] The judgment unit can optimize the judgment algorithm by referring to past fraudulent call data when making a judgment. The judgment unit can, for example, use a database to refer to past fraudulent call data. For example, the judgment unit can analyze past fraudulent call data, learn fraud patterns, and reflect the patterns in the judgment algorithm. In addition, when a new fraud method is discovered, the judgment unit can update the database and optimize the judgment algorithm. Furthermore, the judgment unit can identify elements that are likely to be fraud based on past fraudulent call data and reflect the elements in the judgment. For example, the judgment unit can identify elements that are likely to be fraud based on past fraudulent call data, and reflect the elements in the judgment algorithm. As a result, the judgment unit can improve the accuracy of the judgment algorithm by referring to past fraudulent call data.

[0096] At the time of judgment, the judgment unit can identify elements that increase the possibility of fraud by considering the context of the conversation. The judgment unit can, for example, consider the context of the conversation using a context analysis algorithm. For example, the judgment unit can analyze the context of the conversation and identify phrases or words that are likely to be fraudulent. The judgment unit can also analyze the flow of the conversation and issue a warning when there are signs of fraud. Furthermore, the judgment unit can identify elements that are likely to be fraudulent based on the context of the conversation and reflect them in the judgment. For example, the judgment unit can analyze the context of the conversation using a context analysis algorithm and identify elements that are likely to be fraudulent based on the analysis result. This makes it easier for the judgment unit to identify elements that increase the possibility of fraud by considering the context of the conversation.

[0097] The determination unit can estimate the user's emotion and adjust the display method of the determination result based on the estimated user's emotion. The determination unit can estimate the user's emotion using, for example, voice tone analysis. For example, the determination unit can analyze the tone of the user's voice and determine whether the user is tense or relaxed. The determination unit can also estimate the user's emotion using facial expression recognition technology. For example, the determination unit can analyze the user's facial expression and determine whether the user is tense or relaxed. Furthermore, the determination unit can adjust the display method of the determination result based on the estimated user's emotion. For example, when the user is tense, a simple and easy-to-understand display method can be provided. Furthermore, when the user is relaxed, a display method including detailed information can be provided. Furthermore, when the user is in a hurry, a display method that focuses on the main points can be provided. Thereby, the determination unit adjusts the display method of the determination result according to the user's emotion, thereby enabling an easy-to-understand display. The estimation of the emotion is realized using, for example, an emotion estimation function using an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0098] At the time of the determination, the determination unit can determine the possibility of fraud by taking into account the geographical information of the conversation. The determination unit can, for example, use a location information database to consider the geographical information of the conversation. For example, the determination unit can identify an area where fraud is highly likely based on the geographical information of the conversation. The determination unit can also identify an element where fraud is highly likely based on the geographical information. Furthermore, the determination unit can preferentially determine a conversation in an area where fraud is highly likely based on the geographical information. For example, the determination unit can consider the geographical information of the conversation using a location information database and identify an area where fraud is highly likely based on the information. This allows the determination unit to more accurately determine the possibility of fraud by taking into account the geographical information.

[0099] The determination unit can improve the accuracy of the determination by referring to literature related to the conversation at the time of determination. The determination unit can, for example, use a literature database to refer to literature related to the conversation. For example, the determination unit can compare the content of the conversation with the related literature to determine the possibility of fraud. The determination unit can also identify elements that are likely to be fraudulent based on the related literature. Furthermore, the determination unit can optimize the determination algorithm by referring to the related literature. For example, the determination unit can refer to literature related to the conversation using a literature database and identify elements that are likely to be fraudulent based on the information. Thereby, the determination unit can improve the accuracy of the determination by referring to the related literature.

[0100] The warning unit can estimate the user's emotion and adjust the expression method of the warning based on the estimated user's emotion. The warning unit can estimate the user's emotion, for example, by using voice tone analysis. For example, the warning unit can analyze the tone of the user's voice and determine whether the user is tense or relaxed. The warning unit can also estimate the user's emotion by using facial expression recognition technology. For example, the warning unit can analyze the user's facial expression and determine whether the user is tense or relaxed. Furthermore, the warning unit can adjust the expression method of the warning based on the estimated user's emotion. For example, if the user is tense, the warning can be issued in a calm tone. Furthermore, if the user is relaxed, the warning can be issued in a normal tone. Furthermore, if the user is in a hurry, a concise expression can be used to issue a warning quickly. Thereby, the warning unit adjusts the expression method of the warning according to the user's emotion, thereby improving the effect of the warning. The estimation of the emotion is realized using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0101] The warning unit can select the most appropriate warning method by referring to the user's past reaction history at the time of warning. The warning unit can, for example, use a database to refer to the user's past reaction history. For example, the warning unit can preferentially use a warning method to which the user responded quickly in the past. The warning unit can also try a new warning method, avoiding warning methods that the user has ignored in the past. Furthermore, the warning unit can analyze the user's past reaction history and select the most effective warning method. For example, the warning unit can use a database to refer to the user's past reaction history and select the most appropriate warning method based on that information. In this way, the warning unit can select the most appropriate warning method by referring to the past reaction history.

[0102] The warning unit can adjust the timing of the warning based on the user's current situation at the time of warning. The warning unit can, for example, use real-time data analysis to evaluate the user's current situation. For example, the warning unit can issue a warning at an important timing when the user is busy. Also, the warning unit can issue a warning at a normal timing when the user is relaxed. Furthermore, the warning unit can adjust the timing to issue a warning quickly when the user is in a hurry. For example, the warning unit can evaluate the user's current situation using real-time data analysis and adjust the timing of the warning based on the information. Thereby, the warning unit can adjust the timing of the warning according to the user's situation, thereby improving the effectiveness of the warning.

[0103] The warning unit can estimate the user's emotion and determine the priority of the warning based on the estimated user's emotion. The warning unit can estimate the user's emotion using, for example, voice tone analysis. For example, the warning unit can analyze the tone of the user's voice and determine whether the user is tense or relaxed. The warning unit can also estimate the user's emotion using facial expression recognition technology. For example, the warning unit can analyze the user's facial expression and determine whether the user is tense or relaxed. Furthermore, the warning unit can determine the priority of the warning based on the estimated user's emotion. For example, if the user is tense, an important warning can be displayed with priority. Furthermore, if the user is relaxed, a warning can be displayed with normal priority. Furthermore, if the user is in a hurry, a warning that requires a quick response can be displayed with priority. Thereby, the warning unit can display important warnings with priority by determining the priority of the warning according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0104] The warning unit can select the optimal warning method in consideration of the geographical information of the user at the time of warning. The warning unit can consider the geographical information of the user by using, for example, a location information database. For example, when the user is at home, the warning unit can preferentially use a voice warning. Also, when the user is out, the warning unit can issue a warning by a text message. Furthermore, the warning unit can select the optimal warning method based on the geographical information of the user. For example, the warning unit can consider the geographical information of the user by using a location information database, and select the optimal warning method based on the information. As a result, the warning unit can select the optimal warning method by considering the geographical information.

[0105] At the time of issuing a warning, the warning unit can analyze the user's social media activity and suggest a warning means. The warning unit can, for example, analyze the user's social media activity using an analysis of the social media activity. For example, the warning unit can issue a warning through a social media platform that the user frequently uses. The warning unit can also analyze the user's social media activity and suggest an optimal warning means. Furthermore, the warning unit can select a warning means based on information shared by the user on social media. For example, the warning unit can analyze the user's social media activity using an analysis of the social media activity and suggest an optimal warning means based on the information. In this way, the warning unit can suggest an optimal warning means by analyzing the social media activity.

[0106] The communication unit can estimate the user's emotion and adjust the manner of expression of the communication based on the estimated user's emotion. The communication unit can estimate the user's emotion, for example, by using voice tone analysis. For example, the communication unit can analyze the tone of the user's voice and determine whether the user is tense or relaxed. The communication unit can also estimate the user's emotion by using facial expression recognition technology. For example, the communication unit can analyze the user's facial expression and determine whether the user is tense or relaxed. Furthermore, the communication unit can adjust the manner of expression of the communication based on the estimated user's emotion. For example, if the user is tense, the communication can be made in a calm tone. Furthermore, if the user is relaxed, the communication can be made in a normal tone. Furthermore, if the user is in a hurry, a concise expression can be used to make a quick contact. Thereby, the communication unit adjusts the manner of expression of the communication according to the user's emotion, thereby improving the effect of the communication. The emotion estimation is realized by using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0107] When making contact, the contact unit can select the optimal contact method by referring to the past response history of the relevant person. The contact unit can, for example, use a database to refer to the past response history of the relevant person. For example, the contact unit can preferentially use a contact method to which the relevant person has responded quickly in the past. The contact unit can also try a new contact method, avoiding contact methods that the relevant person has ignored in the past. Furthermore, the contact unit can analyze the past response history of the relevant person and select the most effective contact method. For example, the contact unit can use a database to refer to the past response history of the relevant person and select the optimal contact method based on that information. In this way, the contact unit can select the optimal contact method by referring to the past response history.

[0108] The communication unit can adjust the timing of the contact based on the current situation of the relevant person at the time of contact. The communication unit can, for example, use real-time data analysis to evaluate the current situation of the relevant person. For example, the communication unit can contact the relevant person at an important timing when the relevant person is busy. Also, the communication unit can contact the relevant person at a normal timing when the relevant person is relaxed. Furthermore, the communication unit can adjust the timing to contact the relevant person quickly when the relevant person is in a hurry. For example, the communication unit can evaluate the current situation of the relevant person using real-time data analysis and adjust the timing of the contact based on the information. Thereby, the communication unit can adjust the timing of the contact according to the situation of the relevant person, thereby improving the effectiveness of the contact.

[0109] The communication unit can estimate the user's emotion and determine the priority of the communication based on the estimated user's emotion. The communication unit can estimate the user's emotion using, for example, voice tone analysis. For example, the communication unit can analyze the tone of the user's voice and determine whether the user is nervous or relaxed. The communication unit can also estimate the user's emotion using facial expression recognition technology. For example, the communication unit can analyze the user's facial expression and determine whether the user is nervous or relaxed. Furthermore, the communication unit can determine the priority of the communication based on the estimated user's emotion. For example, when the user is nervous, important communication can be prioritized. Furthermore, when the user is relaxed, communication can be made with normal priority. Furthermore, when the user is in a hurry, communication that requires a quick response can be made with priority. Thereby, the communication unit can prioritize important communication by determining the priority of the communication according to the user's emotion. The emotion estimation is realized using, for example, an emotion estimation function using an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0110] The contact unit can select the optimal contact method when making contact by taking into consideration geographical information of the relevant person. The contact unit can, for example, use a location information database to take into consideration the geographical information of the relevant person. For example, the contact unit can preferentially use voice contact when the relevant person is at home. Also, the contact unit can contact the relevant person by text message when the relevant person is out. Furthermore, the contact unit can select the optimal contact method based on the geographical information of the relevant person. For example, the contact unit can consider geographical information of the relevant person using a location information database and select the optimal contact method based on that information. In this way, the contact unit can select the optimal contact method by taking into consideration geographical information.

[0111] At the time of contact, the contact department can analyze the social media activity of the relevant person and suggest a means of contact. The contact department can, for example, analyze the social media activity of the relevant person using an analysis of social media activity. For example, the contact department can contact the relevant person through a social media platform that the relevant person frequently uses. The contact department can also analyze the social media activity of the relevant person and suggest the optimal means of contact. Furthermore, the contact department can select a means of contact based on information shared by the relevant person on social media. For example, the contact department can analyze the social media activity of the relevant person using an analysis of social media activity and suggest the optimal means of contact based on the information. In this way, the contact department can suggest the optimal means of contact by analyzing social media activity.

[0112] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0113] The telephone conversation content analysis system may further include a translation unit. The translation unit may translate the textualized conversation content into a plurality of languages. For example, the translation unit may support major languages ​​such as English, Japanese, and Chinese. The translation unit may also perform natural translation taking into account the context of the conversation content. Furthermore, the translation unit may also appropriately translate technical terms and expressions specific to an industry. This allows the telephone conversation content analysis system to be compatible with international users and facilitate communication between different languages.

[0114] The determination unit can improve the accuracy of the determination by referring to the user's past behavior history. For example, the determination unit can record in a database what kind of calls the user has received in the past and how the user responded, and make a determination based on that information. The determination unit can also learn the user's behavior patterns and more accurately determine calls that are likely to be fraudulent. Furthermore, the determination unit can strengthen the warning when there are signs of fraud based on the user's past behavior history. In this way, the determination unit can improve the accuracy of determining fraudulent calls by utilizing the user's behavior history.

[0115] The voice recognition unit can learn the characteristics of a user's voice and apply an individually optimized voice recognition model. For example, the voice recognition unit can learn the tone, pitch, and pronunciation habits of a user's voice and customize the voice recognition model based on that information. The voice recognition unit can also register the characteristics of a user's voice in a database and improve recognition accuracy by referring to that data during voice recognition. Furthermore, the voice recognition unit can also learn the characteristics of the voices of multiple users simultaneously and apply a voice recognition model optimized for each user. As a result, the voice recognition unit can optimize the voice recognition model based on the characteristics of the user's voice, thereby improving recognition accuracy.

[0116] The summarization unit can analyze the emotional tone of the conversation content and generate a summary according to the emotion. For example, if the conversation content is tense, the summarization unit can generate a concise and easy-to-understand summary. If the conversation content is relaxed, the summarization unit can also generate a summary including detailed information. Furthermore, if the conversation content is rushed, the summarization unit can generate a summary that extracts only the important points. In this way, the summarization unit can adjust the way the summary is expressed according to the emotional tone of the conversation content, thereby improving understanding of the summary.

[0117] The warning unit can estimate the user's emotion and adjust the way the warning is expressed based on the estimated user's emotion. For example, if the user is nervous, the warning can be given in a calm tone. Also, if the user is relaxed, the warning can be given in a normal tone. Furthermore, if the user is in a hurry, a concise expression can be used to give the warning quickly. In this way, the warning unit can improve the effectiveness of the warning by adjusting the way the warning is expressed depending on the user's emotion.

[0118] The communication unit can estimate the user's emotions and adjust the manner of communication expression based on the estimated user's emotions. For example, if the user is nervous, the communication can be made in a calm tone. Also, if the user is relaxed, the communication can be made in a normal tone. Furthermore, if the user is in a hurry, concise expressions can be used to quickly communicate. In this way, the communication unit can improve the effectiveness of communication by adjusting the manner of communication expression according to the user's emotions.

[0119] The speech recognition unit can improve the accuracy of speech recognition by taking into account the geographical accent or dialect of the speaker during speech recognition. For example, the speech recognition unit can take into account the geographical accent or dialect of the speaker by using a speech database for each region. The speech recognition unit can also analyze the characteristics of the speaker's dialect by using a dialect dictionary and optimize the speech recognition model. Furthermore, the speech recognition unit can improve the accuracy of speech recognition by taking into account differences in pronunciation for each region. As a result, the speech recognition unit improves the accuracy of speech recognition by taking into account geographical accents and dialects.

[0120] At the time of judgment, the judgment unit can identify elements that increase the possibility of fraud by considering the context of the conversation. For example, the judgment unit can analyze the context of the conversation and identify phrases or words that are likely to be fraudulent. The judgment unit can also analyze the flow of the conversation and issue a warning if there are signs of fraud. Furthermore, the judgment unit can identify elements that are likely to be fraudulent based on the context of the conversation and reflect them in the judgment. This makes it easier for the judgment unit to identify elements that increase the possibility of fraud by considering the context of the conversation.

[0121] The summarization unit can apply different summarization algorithms depending on the category of the conversation when generating a summary. For example, in the case of a business conversation, the summarization unit can generate a detailed summary including technical terms. In addition, in the case of a private conversation, the summarization unit can also generate a concise and easy-to-understand summary. Furthermore, in the case of an urgent conversation, the summarization unit can quickly generate a summary that extracts the main points. In this way, the summarization unit can apply a summarization algorithm depending on the category of the conversation, thereby improving the accuracy of the summary.

[0122] When issuing a warning, the warning unit may analyze the user's social media activity and suggest a warning method. For example, the warning unit may issue a warning through a social media platform that the user frequently uses. The warning unit may also analyze the user's social media activity and suggest an optimal warning method. Furthermore, the warning unit may select a warning method based on information shared by the user on social media. In this way, the warning unit may suggest an optimal warning method by analyzing social media activity.

[0123] The process flow of the second embodiment will be briefly described below.

[0124] Step 1: The voice recognition unit converts the contents of the telephone conversation into text. The voice recognition unit converts the contents of the telephone conversation into text in real time using voice recognition technology. For example, the voice recognition unit converts the contents of the telephone conversation into text using a voice recognition algorithm. Step 2: The summarization unit uses a generation AI to summarize the conversation content that has been converted to text by the speech recognition unit. The summarization unit analyzes the converted conversation content, extracts important information, and summarizes it. For example, the conversation content is summarized using a text generation AI (e.g., LLM). Step 3: The judgment unit checks the contents summarized by the summarization unit against the judgment criteria to determine whether it is a fraudulent call. The judgment unit determines whether the summarized conversation content meets the criteria, such as "a sudden request for money" or "a transfer to an unknown account." Step 4: If the determination unit determines that the call is likely to be a fraudulent call, the warning unit warns the telephone user. For example, the warning unit warns the telephone user that "there is a possibility of fraud." Step 5: If the judgment unit judges that the call may be fraudulent, the communication unit contacts the relevant person. For example, the communication unit may inform the relevant person that "there has been a call that may be fraudulent."

[0125] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0126] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0127] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0128] Each of the multiple elements including the above-mentioned voice recognition unit, summarization unit, judgment unit, warning unit, and contact unit is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the voice recognition unit is realized by the processor 46 of the smart device 14, and converts the contents of the telephone conversation into text in real time. The summarization unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and summarizes the text-converted conversation contents using a generation AI. The judgment unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and determines the possibility of a fraudulent call by comparing the summarized conversation contents with a judgment condition. The warning unit is realized, for example, by the control unit 46A of the smart device 14, and issues a warning to the telephone user. The contact unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and contacts the relevant parties. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0129] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0130] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0131] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0132] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0133] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0134] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0135] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0136] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0137] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0138] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0139] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0140] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0141] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0142] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0143] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0144] Each of the multiple elements including the above-mentioned voice recognition unit, summarization unit, determination unit, warning unit, and contact unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the voice recognition unit is realized by the processor 46 of the smart glasses 214, and converts the contents of a telephone conversation into text in real time. The summarization unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and summarizes the text-converted conversation contents using a generation AI. The determination unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and determines the possibility of a fraudulent call by comparing the summarized conversation contents with a determination condition. The warning unit is realized, for example, by the control unit 46A of the smart glasses 214, and warns the telephone user. The contact unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and contacts the relevant parties. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0145] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0146] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0147] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0148] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0149] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0150] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0151] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0152] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0153] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0154] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0155] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0156] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0157] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0158] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0159] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0160] Each of the multiple elements including the above-mentioned voice recognition unit, summarization unit, judgment unit, warning unit, and contact unit is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the voice recognition unit is realized by the processor 46 of the headset type terminal 314, and converts the contents of the telephone conversation into text in real time. The summarization unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and summarizes the text-converted conversation contents using a generation AI. The judgment unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and checks the summarized conversation contents against a judgment condition to determine the possibility of a fraudulent call. The warning unit is realized, for example, by the control unit 46A of the headset type terminal 314, and issues a warning to the telephone user. The contact unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and contacts the relevant parties. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0161] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0162] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0163] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0164] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0165] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0166] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0167] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0168] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0169] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0170] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0171] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0172] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0173] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0174] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0175] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0176] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0177] Each of the multiple elements including the above-mentioned voice recognition unit, summarization unit, judgment unit, warning unit, and contact unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the voice recognition unit is realized by the processor 46 of the robot 414, and converts the contents of the telephone conversation into text in real time. The summarization unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and summarizes the text-converted conversation contents using a generation AI. The judgment unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and determines the possibility of a fraudulent call by comparing the summarized conversation contents with a judgment condition. The warning unit is realized, for example, by the control unit 46A of the robot 414, and issues a warning to the telephone user. The contact unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and contacts the relevant parties. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0178] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0179] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0180] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0181] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0182] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0183] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0184] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0185] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0186] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0187] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0188] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0189] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0190] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0191] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0192] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The above specific processes are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0193] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0194] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0195] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0196] (Appendix 1) A voice recognition unit that converts telephone conversations into text; a summarizing unit that summarizes the conversation content converted into text by the speech recognition unit; a judgment unit that compares the contents summarized by the summarizing unit with judgment conditions and judges whether the call is a fraudulent call; a warning unit that warns a telephone user when the judgment unit judges that the call is a fraudulent call; a contact unit that contacts relevant parties when the determination unit determines that the call is a fraudulent call. A system characterized by: (Appendix 2) The determination unit is Equipped with a means for updating judgment conditions 2. The system of claim 1, (Appendix 3) The voice recognition unit Instantly transcribe phone conversations into text 2. The system of claim 1, (Appendix 4) The summary portion comprises: Analyze the transcribed conversation and extract and summarize information based on specific criteria 2. The system of claim 1, (Appendix 5) The warning unit is Warn the caller if the call is determined to be fraudulent 2. The system of claim 1, (Appendix 6) The communication unit includes: Contacting relevant parties if a call is determined to be fraudulent 2. The system of claim 1, (Appendix 7) The voice recognition unit Analyze user emotions and adjust speech recognition accuracy based on the analyzed user emotions 2. The system of claim 1, (Appendix 8) The voice recognition unit Add a feature to remove background noise during voice recognition 2. The system of claim 1, (Appendix 9) The voice recognition unit During speech recognition, the system analyzes the characteristics of the speaker's voice and applies an individually tuned speech recognition model 2. The system of claim 1, (Appendix 10) The voice recognition unit Analyze user emotions and adjust the timing of speech recognition based on the analyzed user emotions. 2. The system of claim 1, (Appendix 11) The voice recognition unit Improve speech recognition by taking into account the speaker's geographical accent and dialect 2. The system of claim 1, (Appendix 12) The voice recognition unit During speech recognition, the accuracy of the recognition is improved by referring to the speaker's past conversation history. 2. The system of claim 1, (Appendix 13) The summary portion comprises: Estimate the user's emotions and adjust the way summaries are presented based on the estimated user emotions. 2. The system of claim 1, (Appendix 14) The summary portion comprises: When generating summaries, adjust the level of detail based on the importance of the conversation 2. The system of claim 1, (Appendix 15) The summary portion comprises: When generating summaries, different summarization algorithms are applied depending on the category of the conversation. 2. The system of claim 1, (Appendix 16) The summary portion comprises: Estimating user sentiment and adjusting the length of a summary based on the estimated user sentiment 2. The system of claim 1, (Appendix 17) The summary portion comprises: Prioritize summaries based on time of conversation during summary generation 2. The system of claim 1, (Appendix 18) The summary portion comprises: When generating summaries, adjust the order of summaries based on the relevance of the conversation 2. The system of claim 1, (Appendix 19) The determination unit is To estimate a user's emotion and adjust a judgment criterion based on the estimated user's emotion. 2. The system of claim 1, (Appendix 20) The determination unit is When making a judgment, the judgment algorithm is optimized by referencing past fraudulent call data. 2. The system of claim 1, (Appendix 21) The determination unit is When adjudicating, consider the context of the conversation to identify factors that increase the likelihood of fraud 2. The system of claim 1, (Appendix 22) The determination unit is To estimate a user's emotion and adjust a method of displaying a judgment result based on the estimated user's emotion. 2. The system of claim 1, (Appendix 23) The determination unit is When assessing, the geographical location of the conversation is taken into account to determine the likelihood of fraud. 2. The system of claim 1, (Appendix 24) The determination unit is When making a judgment, the accuracy of the judgment is improved by referring to related literature on the conversation. 2. The system of claim 1, (Appendix 25) The warning unit is Infer user emotions and adjust warning wording based on the inferred user emotions 2. The system of claim 1, (Appendix 26) The warning unit is When a warning is issued, the system selects the most appropriate warning method by referring to the user's past response history. 2. The system of claim 1, (Appendix 27) The warning unit is When alerting, adjust the timing of the alert based on the user's current situation 2. The system of claim 1, (Appendix 28) The warning unit is Infer user emotions and prioritize alerts based on the inferred user emotions 2. The system of claim 1, (Appendix 29) The warning unit is When alerting, the most appropriate alert method is selected taking into account the user's geographical information. 2. The system of claim 1, (Appendix 30) The warning unit is When a warning is issued, the system analyzes the user's social media activity to suggest a method of warning. 2. The system of claim 1, (Appendix 31) The communication unit includes: Inferring a user's emotions and adjusting the way a communication is expressed based on the inferred user's emotions 2. The system of claim 1, (Appendix 32) The communication unit includes: When contacting someone, choose the best way to contact them by looking at their past responses 2. The system of claim 1, (Appendix 33) The communication unit includes: When communicating, adjust the timing of your contact based on the stakeholders' current situation 2. The system of claim 1, (Appendix 34) The communication unit includes: To estimate a user's emotion and determine a priority of contacts based on the estimated user's emotion. 2. The system of claim 1, (Appendix 35) The communication unit includes: When making contact, choose the most appropriate contact method based on the geographic location of the relevant parties. 2. The system of claim 1, (Appendix 36) The communication unit includes: When contacting someone, analyze their social media activity to suggest ways to contact them 2. The system of claim 1, [Explanation of symbols]

[0197] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A voice recognition unit that converts telephone conversations into text; a summarizing unit that summarizes the conversation content converted into text by the speech recognition unit; a judgment unit that compares the contents summarized by the summarizing unit with judgment conditions and judges whether the call is a fraudulent call; a warning unit that warns a telephone user when the judgment unit judges that the call is a fraudulent call; a contact unit that contacts relevant parties when the determination unit determines that the call is a fraudulent call. A system characterized by:

2. The determination unit is Equipped with a means for updating judgment conditions 2. The system of claim 1 .

3. The voice recognition unit Instantly transcribe phone conversations into text 2. The system of claim 1 .

4. The summary portion comprises: Analyze the transcribed conversation and extract and summarize information based on specific criteria 2. The system of claim 1 .

5. The warning unit is Warn the caller if the call is determined to be fraudulent 2. The system of claim 1 .

6. The voice recognition unit Analyze user emotions and adjust speech recognition accuracy based on the analyzed user emotions 2. The system of claim 1 .

7. The voice recognition unit Add a feature to remove background noise during voice recognition 2. The system of claim 1 .

8. The voice recognition unit During speech recognition, the system analyzes the characteristics of the speaker's voice and applies an individually tuned speech recognition model 2. The system of claim 1 .

Images