Information terminal, information processing device, and automatic response system

Abstract

In the present invention, a smartphone is provided with an answerphone function for carrying out an automatic conversation with a caller if there is an unattended incoming call. A main control unit of the smartphone generates speech text by converting the speech of the caller into text, and collects, from schedule information and on the basis of the speech text, schedules for creating an input sentence. The main control unit inputs the input sentence created on the basis of the speech text and the collected schedules to a generative AI server so as to cause the generative AI server to create response text. The main control unit corrects the response text at a disclosure granularity corresponding to the identity of the caller, creates response speech by converting the corrected response text back into speech, andl carries out an automatic conversation with the caller by using the response speech.

Description

Information terminal, information processing device and automatic response system

[0001] The present invention relates to an information terminal, an information processing device, and an automatic response system.

[0002] The use of automated voice responses utilizing AI (Artificial Intelligence) is becoming more widespread. Patent Document 1 discloses an information processing terminal that takes into consideration the attributes of the access source, the access content, and the status of the user and the information processing terminal, and automatically transmits an appropriate response to the accessing party according to the situation, without requiring the user to go through complicated procedures.

[0003] Japanese Patent Application Laid-Open No. 2021-40351

[0004] The inventors of the present application are considering applying generation AI to answering machines and the like to equip them with an automatic dialogue function. In this case, depending on the identity of the caller, it is possible that a caller may create a response message containing content that should not be included in the response message from a confidentiality perspective. The present invention has been made to solve the above problem. That is, one object of the present invention is to provide an information terminal, an information processing device, and an automatic response system that can appropriately control the content to be included in the response message.

[0005] In order to solve the above problem, the information terminal of the present invention is an information terminal equipped with a control unit and an automatic response unit including the control unit, and when the information terminal responds using the automatic response unit, the control unit controls the automatic response unit to convert the content received by the automatic response unit into text data, collects response information from internal and / or external information based on the text data, creates input data based on the text data and the response information, sends the input data to an internal and / or external language generation AI, receives output data from the language generation AI based on the input data, and sends the output data to the automatic response unit.

[0006] The information processing device of the present invention is an information processing device that has a calculation device and communicates with an information terminal, wherein the calculation device converts content received from the information terminal into text data when the information terminal performs an automatic response, collects response information from internal and / or external information based on the text data, creates input data based on the text data and the response information, transmits the input data to an internal and / or external language generation AI, receives output data from the language generation AI based on the input data, and transmits the output data or modified data of the output data to the information terminal.

[0007] The automatic response system of the present invention is an automatic response system having an information terminal and a server device, wherein when the information terminal makes an automatic response, it converts the received content into text data, collects response information from internal and / or external information based on the text data, creates input data based on the text data and the response information, and sends the input data to the server device, and the server device receives the input data, sends the input data to an internal and / or external language generation AI, receives output data from the language generation AI, and sends the output data to the information terminal, and the information terminal receives the output data.

[0008] According to the present invention, it is possible to appropriately control the content to be included in a response message. Note that the effects described herein are not necessarily limited to those described herein, and may be any of the effects described in this disclosure.

[0009] FIG. 1 is a diagram showing an example of the configuration of a system including an answering machine system according to a first embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of a smartphone. FIG. 3 is a diagram illustrating an overview of a method for creating an LLM. FIG. 4 is a diagram illustrating an identity determination table. FIG. 5 is a diagram illustrating schedule information. FIG. 6 is a diagram illustrating an example of the hardware configuration of a computer. FIG. 7 is a diagram illustrating a usage mode of the answering machine function of the answering machine system. FIG. 8 is a diagram illustrating the operation of the above-mentioned automatic dialogue function of the answering machine system. FIG. 9 is a diagram illustrating an example of a message according to the identity of the caller. FIG. 10 is a diagram illustrating an example of a message according to the identity of the caller. FIG. 11 is a diagram illustrating a schedule approval function. FIG. 12 is a diagram illustrating the schedule approval function. FIG. 13 is a diagram illustrating a schedule confirmation screen. FIG. 14 is a diagram illustrating the schedule approval function. FIG. 15 is a diagram illustrating an example of an approval notification message. FIG. 16 is a diagram illustrating a schedule confirmation screen. FIG. 17 is a diagram illustrating the schedule approval function. FIG. 18 is a diagram illustrating an example of a cancellation notification message.

[0010] Hereinafter, each embodiment of the present invention will be described with reference to the drawings. In all the drawings of the embodiments, the same or corresponding parts may be denoted by the same reference numerals.

[0011] In the following description, various types of information may be described using expressions such as "tables," but the various types of information may be expressed using data structures other than these.

[0012] In the following description, processing may be described using functional blocks as the subject, but the subject of the processing may be a CPU or a device instead of a functional block. The subject of processing performed by executing a program may be a computing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit may be, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0013] In the following description, a program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource for storing the program to be distributed, and the processor of the program distribution server may distribute the program to other computers. Furthermore, in each embodiment, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs. <<First Embodiment>> An answering machine system, which is an example of an automatic answering system according to a first embodiment of the present invention, will be described. FIG. 1 is a diagram illustrating an example system configuration including an answering machine system according to the first embodiment. The system includes a recipient smartphone 100a, a caller smartphone 100b, a generation AI server 200, and a conversation database server 300. Hereinafter, the recipient smartphone 100a and the caller smartphone 100b will be referred to as "smartphones 100" unless there is a need to distinguish them. A system including the recipient smartphone 100a, the generation AI server 200, and the conversation database server 300 corresponds to the answering machine system according to the first embodiment.

[0014] The receiver smartphone 100a, the sender smartphone 100b, the generation AI server 200, and the conversation database server 300 are connected to each other via a network NW1 (Internet) so as to be able to communicate with each other. The receiver smartphone 100a and the sender smartphone 100b are connected to each other via a public telephone network PNW1 and a base station BS1 so as to be able to communicate with each other.

[0015] 2 is a block diagram illustrating an example configuration of the smartphone 100. As shown in FIG. 2, the smartphone 100 includes a main control unit 110, a system bus 120, a storage unit 130, a sensor unit 140, an operation input unit 150, a video processing unit 160, an audio processing unit 170, and a communication processing unit 180.

[0016] The main control unit 110 is a microprocessor unit that controls the entire smartphone 100 in accordance with a predetermined operation program such as an application. The main control unit 110 may also be referred to as a "computing device."

[0017] The system bus 120 is a data communication path for transmitting and receiving various commands and data between the main control unit 110 and each component block within the smartphone 100 .

[0018] The storage unit 130 includes a program unit 131 , a various data unit 132 , and a program function unit 133 .

[0019] The program unit 131 stores programs for controlling the operation of the smartphone 100, etc. The programs include an information collection agent. The information collection agent is a program designed to accomplish a specific task. Tasks include searching for information, collecting information, etc. The program unit 131 includes a program for implementing an automatic answering function (for example, an answering machine function). The various data unit 132 stores various data such as operation setting values, detection values ​​from the sensor unit 140 described below, content (such as standard voice data used for the answering machine function), an identity determination table 4000 (see Figure 4), schedule information 5000 (see Figure 5), an address book, a phone book, and search history.

[0020] The program function unit 133 is a rewritable storage area such as a work area used for various program operations.

[0021] The storage unit 130 can store operation programs downloaded from devices such as servers on the network NW1 and various data created by the operation programs. The storage unit 130 can also store content such as videos, still images, and audio downloaded from devices such as servers on the network NW1. The storage unit 130 can also store data such as videos and still images captured using the imaging unit 161 (described below). The storage unit 130 must retain stored information even when the smartphone 100 is not receiving external power. Therefore, devices such as semiconductor memory devices, such as flash ROMs and solid-state drives (SSDs), are used. The operation programs stored in the storage unit 130 can be updated and expanded by downloading them from devices on the network NW1.

[0022] The sensor unit 140 includes a group of various sensors for detecting the state of the smartphone 100. The group of sensors includes a position sensor receiving unit 141, a geomagnetic sensor unit 142, a distance sensor unit 143, an acceleration sensor unit 144, and a gyro sensor unit 145. These sensors enable the smartphone 100 to detect its position, orientation, direction of movement, motion, tilt, and the like. The smartphone 100 may also include other sensors, such as an illuminance sensor and a proximity sensor. Furthermore, by wearing devices paired with these sensors on the hand or arm, the movement of the hand or arm can be detected. By comprehensively utilizing these sensors, the walking direction of the user carrying the smartphone 100, the tilt and direction of the smartphone 100, and the like can be detected.

[0023] The operation input unit 150 is an instruction input unit that inputs operation instructions to the smartphone 100. The operation input unit 150 is configured with a touch panel on a display screen, an operation key with an array of button switches, etc., etc. The operation input unit 150 may further include other operation devices. The smartphone 100 may be operated using a separate mobile terminal device connected by wired or wireless communication using a communication processing unit 180 described below. Furthermore, the smartphone 100 may be operated using voice commands as operation instructions using a voice recognition unit 172 of a voice processing unit 170 described below.

[0024] The video processing unit 160 includes an imaging unit 161 and a display unit 162. The imaging unit 161 is a camera unit that inputs image data of an object to be imaged by converting light input through a lens into an electrical signal using an electronic device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The display unit 162 is a display device (display device) such as a liquid crystal panel, and provides the image data to the user of the smartphone 100. The display unit 162 includes a video RAM (not shown). An image is displayed on a display screen based on the image data input to the video RAM.

[0025] The audio processing unit 170 includes an audio input / output unit 171, a voice recognition unit 172, and an audio decoding unit 173. The audio input of the audio input / output unit 171 is via a microphone, and the user's voice, etc. is converted into audio data and input. The audio output of the audio input / output unit 171 is via a speaker or earphones, and audio information required by the user is output. The voice recognition unit 172 analyzes the input audio information and extracts instruction commands, etc. The audio decoding unit 173 has a function of performing decoding processing (voice synthesis processing) of the encoded audio signal as necessary.

[0026] The communication processing unit 180 includes a LAN (Local Area Network) communication unit 181, a telephone network communication unit 182, and a near-field communication unit 183. The LAN communication unit 181 is connected to a network NW1 via an access point or the like, and transmits and receives data to and from each server device (the generation AI server 200, the conversation database server 300) on the network NW1. The connection to the access point or the like may be made wirelessly via Wi-Fi (registered trademark) or the like.

[0027] The telephone network communication unit 182 performs telephone communication (calls) and data transmission and reception via wireless communication with base stations BS1 and the like of the public telephone network PNW1. Communication with the base stations BS1 and the like may be performed using the W-CDMA (Wideband Code Division Multiple Access) (registered trademark) system, the GSM (Global System for Mobile communications) (registered trademark) system, the LTE (Long Term Evolution) system, or other communication systems. The close-proximity wireless communication unit 183 performs communication via Bluetooth (registered trademark) communication or infrared communication. The LAN communication unit 181, the telephone network communication unit 182, and the close-proximity wireless communication unit 183 each include an encoding circuit, a decoding circuit, an antenna, and the like.

[0028] 2 includes many components that are not essential to this embodiment, but the effects of this embodiment are not impaired even if these components are not provided. Furthermore, components not shown, such as a digital broadcast receiving unit and an electronic money settlement unit, may also be added.

[0029] The smartphone 100 is equipped with an automatic answering function (for example, an answering machine function). The automatic answering function can be realized by the main control unit 110, the memory unit 130 (the program unit 131, the various data unit 132, and the program function unit 133), the voice processing unit 170, etc. These elements that realize the automatic answering function (for example, an answering machine function) are referred to as the automatic answering unit (for example, an answering machine unit).

[0030] The generation AI server 200 includes an AI agent and a language generation AI. The language generation AI includes large language models (LLMs). The large language models (LLMs) are language models constructed using large amounts of data and deep learning technology. The generation AI server 200 may obtain and use the LLMs created by the conversation database server 300 from the conversation database server 300.

[0031] An AI agent is a program designed to accomplish a specific task, either automatically or under user direction. Tasks include text generation. The AI ​​agent inputs information (text) entered into the generation AI server 200 into the LLM, causing the LLM to generate an appropriate response message (response text).

[0032] The conversation database server 300 has a large number of teaching data and LLMs. In this example, as shown in Figure 3, the conversation database server 300 is composed of an LLM learning server group 3010 and a data management server (not shown) that holds (memorizes, stores) a large number of teaching data.

[0033] The LLM learning server group 3010 includes an LLM learning server A 3011a, an LLM learning server B 3011b, an LLM learning server C 3011c, an LLM learning server D 3011d, and multiple other LLM learning servers (not shown) for each classification to which the caller belongs. Hereinafter, these servers may be referred to as "LLM learning server 3011" unless there is a need to distinguish them. Each LLM learning server 3011 has an LLM corresponding to each caller's classification (own department / project, other departments, group company / partner company, customer, etc.).

[0034] When creating these LLMs, the LLM learning server 3011 uses the sender's utterance 3001 and the receiver's utterance 3002 in response to the sender's utterance 3001 as training data.

[0035] A data management server (not shown) sorts a large number of pieces of training data into categories of sender affiliation, and creates a training data set 3003 including a large number of pieces of training data for each category of sender affiliation.

[0036] A data management server (not shown) inputs to each LLM learning server 3011 a teacher dataset 3003 of an affiliation classification corresponding to the affiliation classification of the LLM held by each LLM learning server 3011. In this example, the affiliation classification of the teacher dataset 3003 is "in-house (other department)," so the data management server (not shown) inputs the teacher dataset 3003 to LLM learning server B 3011b corresponding to the in-house (other department). When the teacher dataset 3003 is input, the LLM learning server 3011 creates (updates) an LLM through machine learning. As described above, the LLM learning server 3011 creates an LLM and retains (memorizes, stores) the created LLM.

[0037] As will be described in detail later, when the answering machine function is active and the recipient smartphone 100a generates a response message in response to a caller's speech, the recipient smartphone 100a may determine the caller's affiliation, select an LLM learning server 3011 corresponding to the determined affiliation, and have the generation AI server 200 use the LLM held by the selected LLM learning server 3011 to generate a response sentence in response to the caller's speech. In this case, the LLM learning server 3011 suitable for creating a response message is selected depending on the caller's affiliation, thereby improving the accuracy of creating an appropriate response message.

[0038] 4 is a diagram illustrating the background determination table 4000. The recipient smartphone 100a holds (memorizes, stores) the background determination table 4000 in the various data section 132 of the storage unit 130. The background determination table 4000 includes columns for storing information, such as a level 4001, a caller classification 4002, a determination method 4003, and confidential information [& response method] 4004. The background determination table 4000 stores information related to the background determination of callers in association with each other.

[0039] Specifically, the level of the identity is stored in level 4001. In this example, A, B, C, D, and E are stored as identity levels. The identity levels are determined based on the affiliation of the sender (the other party), for example, and increase in order from A > B > C > D > E. The lower the identity level, the higher the security level that is required.

[0040] The classification of the call partner corresponding to the level of the feature is stored in the call partner classification 4002. In this example, the classifications of the call partner are "internal - own department / project", "internal - other department", "external - group company", "external - stakeholder (cooperative company)", "external - customer", and "external - other".

[0041] The determination method corresponding to the level of the identity is stored in the determination method 4003. In this example, the methods of determining the level of the identity are [department / name] ∩ [telephone number], [telephone number] ∩ [department / name], and (options: [voiceprint], [password], [content analysis], etc.).

[0042] [Affiliation / Name] ∩ [Phone Number] means that if the caller's [Affiliation / Name] belongs to the corresponding caller classification 4002 and the caller's [Phone Number] belongs to the corresponding caller classification 4002, the caller's affiliation is determined to be the corresponding caller classification 4002 and the corresponding feature level 4001.

[0043] [Phone number] ∩ [Affiliation / Name] and (options: [Voiceprint], [PW], [Content analysis], etc.) mean that if the caller's [Affiliation / Name] belongs to the corresponding caller party classification 4002 and the caller's [Phone number] also belongs to the corresponding caller party classification 4002, the caller's affiliation is determined to be the corresponding caller party classification 4002 and the corresponding feature level 4001. It also means that the caller's affiliation may be determined to be the corresponding caller party classification 4002 and the corresponding feature level 4001 by [Voiceprint], [PW], [Content analysis], etc.

[0044] Information to be anonymized from text corresponding to the level of the identity is stored in confidential information [& response method] 4004. In this example, the information to be anonymized is stored as "(1) personal information (address, date of birth, health / family related information, etc.)", "(1) + (2) department and project information", "(1) + (2) + (3) internal company information (confidential)", and "(1) + (2) + (3) + (4) internal company information (general)".

[0045] The recipient smartphone 100a determines the sender's identity level using a method in accordance with the identity determination table 4000, and conceals the information contained in the text according to the confidentiality information [& response method] 4004 corresponding to the identity level.

[0046] 5 is a diagram illustrating schedule information 5000. The recipient smartphone 100a holds (memorizes, stores) the schedule information 5000 in the various data section 132 of the storage unit 130. In this example, the schedule information 5000 is information indicating a business schedule of the user of the recipient smartphone 100a. Note that the schedule information 5000 may be stored in a server device (not shown) on the network NW1.

[0047] 6 is a diagram showing an example of the hardware configuration of a computer 6000 applied to each of the generation AI server 200 and the conversation database server 300. The computer 6000 may be referred to as a "computer" or an "information processing device." The computer 6000 includes a CPU 6001, a ROM 6002, a RAM 6003, a non-volatile storage device 6004 capable of reading and writing data, a network interface 6005, and an input / output interface 6006. These are connected to each other via a bus 6007 so as to be able to communicate with each other.

[0048] The CPU 6001 is a computing device that loads various programs (not shown) stored in the ROM 6002 and / or storage device 6004 into the RAM 6003 and executes the programs loaded into the RAM 6003, thereby realizing various functions.

[0049] As described above, the various programs executed by the CPU 6001 are loaded into the RAM 6003, and data used when the CPU 6001 executes the various programs is temporarily stored in the RAM 6003. The ROM 6002 and / or the storage device 6004 are non-volatile storage media, and the ROM 6002 and / or the storage device 6004 store various programs.

[0050] The network interface 6005 is an interface for connecting the computer 6000 to the network NW1. The input / output interface 6006 is an interface for connecting the computer 6000 to an operation device and a display (display device) capable of displaying images.

[0051] Note that instead of the computer 6000, a hardware device configured as a part or the whole of the computer 6000 using an FPGA (Field Programmable Gate Array) or the like may be used. Such a hardware device may also be referred to as an "arithmetic unit."

[0052] Each of the generation AI server 200 and the conversation database server 300 may be composed of multiple computers 6000, and may be not limited to physical computers 6000 but may also be virtual computers 6000. The computers 6000 may be computational resources and storage resources provided by a cloud service, and the functions provided by each of the generation AI server 200 and the conversation database server 300 may be provided by the cloud service.

[0053] The present invention provides an overview thereof. As conversational AI such as ChatGPT advances, various applications and uses of conversational AI are expanding, such as the adoption of conversational AI in conversational reception robots.

[0054] The smartphone 100 stores all kinds of personal information related to work and life in general. The smartphone 100 is equipped with an answering machine function. A typical answering machine function automatically answers calls that come in when you are not at home with a fixed message and records a message from the caller.

[0055] If the answering machine function of the smartphone 100 is equipped with an automatic dialogue function (two-way communication), it is possible to improve communication efficiency (preventing delays in judgment due to absence, adjustments, etc.).

[0056] Therefore, the present inventors are considering adding a dialogue function using generative AI (LLM) to the answering function of the answering machine.

[0057] When an interactive function using the generation AI (LLM) is added, the information collection agent of the smartphone 100 accesses information related to the other party's purpose (schedule, web, search / call history, sensor information (location, etc.)) as appropriate depending on the other party's purpose, references this information, generates a response message using the generation AI (LLM) in response to the other party's speech, and performs simple schedule adjustments, etc. by speaking the generated response message into the other party's phone.

[0058] However, if the response message generated using the generation AI (LLM) is used as is, there is a possibility that confidential information may be leaked if the response message contains information that should not be disclosed to the other party from a confidentiality standpoint.

[0059] Therefore, in the present invention, when a response message is created using a generated AI (LLM), if the generated AI (LLM) on the network NW1 is used, specific information is concealed and the concealed information is applied to the generated AI (LLM) to create the response message, thereby reducing the possibility of confidential information leaks when the generated AI (LLM) on the network NW1 is used.

[0060] Furthermore, in the present invention, when the answering machine function makes a call, the granularity of the disclosed information is appropriately controlled at a security level required according to the identity (level of identity) of the other party. As a result, the present invention can appropriately control the content to be included in the response message from the perspective of confidentiality, thereby reducing the possibility of confidential information being leaked. Furthermore, when the identity (level of identity) of the other party is high, the present invention can efficiently communicate by fine-tuning the granularity of the disclosed information.

[0061] FIG. 7 is a diagram for explaining a usage mode of the answering machine function of the answering machine system (recipient smartphone 100a) according to the first embodiment.

[0062] Step 700: When receiving a call from the caller smartphone 100b, the recipient smartphone 100a starts an answering machine operation (automatic response) if the recipient is not at home.

[0063] Step 710: The recipient smartphone 100a utters a standard phrase to the sender smartphone 100b. For example, the recipient smartphone 100a utters a standard phrase to the sender smartphone 100b, such as "I am currently unavailable to answer the phone. Depending on your business, we can handle the call on your behalf. Would you like to speak to me?", which indicates that the caller is unavailable to answer the phone and that a substitute can handle the call using the automatic dialogue function of the recipient smartphone 100a.

[0064] Step 720: The recipient smartphone 100a starts an automatic dialogue operation and engages in a dialogue (conversation) with the sender through the automatic dialogue. When the dialogue (conversation) with the sender ends, the recipient smartphone 100a ends the automatic dialogue operation. The dialogue (conversation) with the sender is recorded and stored in the various data unit 132.

[0065] Step 730: The recipient smartphone 100a utters a standard phrase to the sender smartphone 100b. For example, the recipient smartphone 100a utters a standard phrase to the sender smartphone 100b, such as "Is that all?", to confirm whether the sender has any other business to discuss.

[0066] Step 740: If the caller has additional matters to attend to, the caller responds by speaking whether or not there are any additional matters to attend to.

[0067] Step 750: The recipient smartphone 100a determines whether there is any additional matter based on the response. If there is any additional matter, the process returns to step 720. If there is no additional matter, the process proceeds to step 760.

[0068] Step 760: The recipient smartphone 100a speaks a standard phrase to the sender smartphone 100b to inform the sender smartphone 100b that the message has been accepted, such as, for example, "After confirming the content, the person in question will contact you if necessary. Thank you for calling. I apologize for the inconvenience."

[0069] Step 795: The recipient smartphone 100a ends the answering machine operation (automatic response).

[0070] FIG. 8 is a diagram for explaining the details of the automatic interactive operation (step 720) of the answering machine system.

[0071] Step 801: The voice data D1 of the sender is input to the voice input / output unit 171.

[0072] Step 802: The main control unit 110 creates a message MS1 by converting the voice data into text (text data) using the voice recognition unit 172. The converted text is sometimes referred to as "spoken text." In this example, the message MS1 is "Hello, this is Suzuki from the sales department at XX Manufacturing Co., Ltd. I'd like to visit you today to explain our new XX series of projectors, and I'm calling to inquire about a convenient date and time for you to visit." In this example, the message MS1 includes content (schedule adjustment) inquiring about a convenient date and time for the sender to visit the recipient's company.

[0073] Step 803: The main control unit 110 creates a message MS2 by concealing some of the information in the text so that the concealed portion can be restored. Methods of concealment include encryption, which encrypts information so that it can be restored, and masking, which masks information so that it can be restored. In this example, the message MS2 is "Hello, this is S from the sales department at Company A. I would like to visit you today to explain our new projector product, the X series, and I am calling to ask about a convenient date and time for us to meet."

[0074] The main control unit 110 analyzes the text message MS1 based on predetermined anonymization criteria and conceals the information contained in the message MS1 that should be concealed. In this example, for example, the message MS1 includes the name of a product series, "△△ series," as information that should be concealed. Therefore, in this example, the message MS2 conceals part of the information in the message MS1 (△△ series) by converting part of the information in the message MS1 (△△ series) to the X series. By concealing part of the information in the message MS1, security can be improved by using the generation AI server 200 in a form that protects important information contained in the sender's speech (message MS1).

[0075] Step 804: The main control unit 110 extracts keywords from the message MS1 to collect information necessary for generating a response message. In this example, the keyword "convenient date and time" is extracted from the message MS1.

[0076] Step 805: The main control unit 110 uses the information collection agent to collect information corresponding to the keyword. The main control unit 110 searches for and collects information corresponding to the keyword from the schedule information 5000, address book, search history, etc. stored in the various data section 132 of the storage unit 130. In this example, the main control unit 110 collects information about inconvenient schedules from the schedule information 5000 (Monday AM: product planning meeting, Thursday: student tour, Tuesday: AI technology seminar at Shinjuku, Wednesday: Yamada Parts Osaka Factory business trip, Thursday PM: student tour).

[0077] The information corresponding to the keyword may include, for example, information necessary for negotiation if the sender's business is one requiring negotiation, or information necessary for adjustment if the sender's business is one requiring adjustment. The information corresponding to the keyword may be referred to as "answer information." The main control unit 110 may use an information collection agent to collect answer information from a database or the like held by a server device on the network NW1 located outside the smartphone 100.

[0078] Step 806: The main control unit 110 converts the collected information into text and conceals some of the information in the text. In this example, the main control unit 110 converts the information necessary for adjusting the schedule collected in step 805 into text and further conceals some of the information to create "A (date and time): <Monday,: AM meeting>, <Thursday: PM visitor>, <Tuesday, Wednesday, Friday: all-day business trip>".

[0079] Step 807: Based on the message MS2 and the information that was converted to text and made confidential in step 806, the main control unit 110 creates (synthesizes) an input sentence MS3 (input data) to be input to the generation AI server 200. In this example, the input sentence MS3 is, "Q: Hello, my name is S, a sales representative at Company A. I would like to visit you today to explain our new X series of projectors, so I am calling to ask about a convenient date and time for you to come. A (date and time): <Monday, AM meeting>, <Thursday, PM visitor>, <Tuesday, Wednesday, Friday, all day business trip>."

[0080] The main control unit 110 sends (transmits) the input sentence MS3 to the generation AI server 200, causing the generation AI server 200 to create an LLM generation original text. The main control unit 110 may also select an LLM learning server 3011 having an LLM model to be used by the generation AI server 200, and cause the generation AI server 200 to use the LLM possessed by the selected LLM learning server 3011 to create the LLM generation original text. In this case, the main control unit 110 may analyze the identity of the sender and select an LLM learning server 3011 corresponding to the identity (affiliation classification) of the analyzed sender.

[0081] Step 808: The generation AI server 200 creates an LLM generation original text by inputting the input sentence MS3 into the LLM using the AI ​​agent and outputting the LLM generation original text (output data), and outputs the created LLM generation original text to the recipient smartphone 100a.

[0082] Step 809: The main control unit 110 extracts keywords from the message MS1 to determine the identity of the sender.

[0083] Step 810: The main control unit 110 uses the information collection agent to collect information corresponding to the keyword from the schedule information 5000, address book, search history, etc. stored in the various data section 132 of the storage unit 130 based on the keyword.

[0084] Step 811: The main control unit 110 determines the identity level of the other party (caller) based on the information collected in step 810, using a method in accordance with the identity determination table 4000. Note that the main control unit 110 may also collect information on the registration number, callback, incoming call history, voiceprint, etc., as necessary, to determine the identity level of the other party (caller).

[0085] Step 812: The main control unit 110 corrects the anonymized information portion of the LLM-generated original text by restoring the anonymized information contained in the LLM-generated original text (the information anonymized in steps 803 and 806).

[0086] Step 813: The main control unit 110 adjusts the granularity of the disclosed information in the LLM-generated original text to a granularity that corresponds to the level of the determined identity of the other party (sender) using a method in accordance with the identity determination table 4000, and creates a response message MS4 from the "LLM-generated original text after correcting the anonymized information portion" based on the adjusted granularity. In this example, the response message MS4 reads, "If you would like to visit, how about Monday afternoon or Thursday morning?"

[0087] Step 814: The main control unit 110 converts the response message MS4 into voice (voice data D11) using the voice decoding unit 173, outputs the voice data D11 from the voice input / output unit 171, and transmits the voice data D11 to the caller smartphone 100b using the communication processing unit 180.

[0088] The answering machine system operates as described above to perform automatic interactive operations.

[0089] 9 is a diagram for explaining examples of messages according to the identity of the caller. In the case of an automated dialogue on an answering machine, the caller's question MS10 is "May I come to your office on Monday at 1:00 PM?" and the identity level determination result is B.

[0090] In this case, the LLM generation text "Meeting A is scheduled until 1:00 PM. If you are not at your desk, please call me on my cell phone (XXXXXX) and I will return to my desk immediately." is generated in step 808. The main control unit 110 modifies the confidential portion of the LLM generation text and creates message MS12 "Meeting A is scheduled until 1:00 PM. If you are not at your desk, please call me on my cell phone (123-4567) and I will return to my desk immediately."

[0091] The main control unit 110 adjusts the message MS12 to the disclosure granularity according to judgment B, thereby creating a response message MS13a saying, "A product planning meeting is scheduled until 1:00 PM. If you are not at your desk, please call me on my mobile phone (123-4567) and I will return to my desk immediately." The main control unit 110 converts the response message MS13a into voice (voice data D11) using the voice processing unit 170, outputs the voice data D11 from the voice input / output unit 171, and transmits the voice data D11 to the caller smartphone 100b using the communication processing unit 180.

[0092] 10 is a diagram illustrating an example of a message depending on the identity of the caller. In the automated dialogue of an answering machine, the caller's question MS10 is "May I come to your office on Monday at 1:00 PM?" as in FIG. 9, and the identity level determination result is D.

[0093] 9, the LLM generation text "Meeting A is scheduled until 1:00 PM. If you are not at your desk, please call me on my cell phone (XXXXXX) and I will return to my desk immediately." is generated in step 808. The main control unit 110, like in Fig. 9, modifies the confidential portion of the LLM generation text and creates message MS12 "Meeting A is scheduled until 1:00 PM. If you are not at your desk, please call me on my cell phone (123-4567) and I will return to my desk immediately."

[0094] The main control unit 110 adjusts the message MS12 to a disclosure granularity according to the determination D, thereby creating a response message MS13b saying, "I have a meeting scheduled until 1:00 PM. If I'm not at my desk, please call someone nearby and call me. I'll be back soon." The main control unit 110 converts the response message MS13b into voice (voice data D11) using the voice processing unit 170, outputs the voice data D11 from the voice input / output unit 171, and transmits the voice data D11 to the caller smartphone 100b using the communication processing unit 180.

[0095] 9, the result of the identity level determination is determination B, whereas in Fig. 10, the result of the identity level determination is determination D, which requires a higher security level than determination B (determination D, which has a lower identity level than determination B). Therefore, compared to response message MS13a in Fig. 9, response message MS13b in Fig. 10 has stricter restrictions on disclosed information (disclosure granularity is coarser). That is, response message MS13a in Fig. 9 allows disclosure of the specific name of the meeting (product planning meeting) and a personal mobile phone number (123-4567) (disclosure granularity is fine), whereas response message MS13b in Fig. 10 does not allow disclosure of the specific name of the meeting or a personal mobile phone number (disclosure granularity is coarse).

[0096] In this way, the answering machine system restricts disclosure of information at a granularity according to the identity of the caller in the automatic answering machine function that utilizes language generation AI. Therefore, the answering machine system can improve security when using the automatic interactive answering machine function and can provide efficient responses according to the identity of the caller. <Effects> As described above, the answering machine system according to the first embodiment of the present invention can appropriately control the content included in the response message to the caller in the automatic interactive function of the answering machine by adjusting the disclosure granularity according to the caller's identity. As a result, the answering machine system according to the first embodiment of the present invention can improve security and provide efficient responses (responses) according to the identity of the caller. <<Second Embodiment>> An answering machine system as an example of an automatic response system according to the second embodiment of the present invention will be described. The answering machine system according to the second embodiment differs from the answering machine system according to the first embodiment only in that the following functions have been added to the answering machine system according to the first embodiment. The answering machine system according to the second embodiment has a schedule approval function.

[0097] The following description will focus on this difference.

[0098] When the caller's business is to adjust a schedule and the schedule adjustment is made through automatic dialogue, it is preferable to have a method for the caller to actually check the schedule in person, since there may be cases where the caller is free on schedule information 5000 but has plans that have not yet been reflected in schedule information 5000. Therefore, the answering machine system according to the second embodiment has a schedule approval function that finalizes the schedule when the caller approves the schedule adjusted through automatic dialogue on the answering machine.

[0099] 11 is a diagram for explaining the schedule approval function. When a conversation to secure the recipient's schedule is held during the automatic response dialogue, the recipient smartphone 100a updates the schedule information 5000.

[0100] For example, after starting the automatic dialogue operation, assume that the sender utters a message MS21, "May I visit your office at 1:00 PM on Monday?" to the recipient smartphone 100a via the sender smartphone 100b in step 1101. In this case, the recipient smartphone 100a checks the schedule in the schedule information 5000.

[0101] 12, according to the schedule information 5000, the recipient smartphone 100a makes a tentative reservation for Monday from 1 PM to 5 PM because there is no appointment available from 1 PM to 5 PM. The recipient smartphone 100a then speaks a message MS22 to the sender smartphone 100b, saying, "You have tentatively reserved a schedule from 1 PM to 5 PM on Monday."

[0102] When the user of the recipient smartphone 100a performs a predetermined operation after the answering machine operation is completed, the recipient smartphone 100a displays a schedule confirmation screen 1300 shown in FIG. 13 . The schedule confirmation screen 1300 includes a message 1301, a matter 1302, a schedule button 1303, an approve button 1304, and a reject button 1305. The message 1301 includes a statement indicating that the schedule has been tentatively reserved. The matter 1302 includes the subject, date, and content of the schedule. The schedule button 1303 is a button operated to display a schedule table on the screen. When the schedule button 1303 is operated, the recipient smartphone 100a displays a schedule table (not shown) based on the schedule information 5000 on the screen.

[0103] The approval button 1304 is a button that is operated to approve the tentative reservation. When the approval button 1304 is operated, the recipient smartphone 100a confirms the tentative reservation schedule in the schedule information 5000, as shown in FIG. 14 . Then, the recipient smartphone 100a notifies the sender smartphone 100b of an approval notification message 1500 shown in FIG. 15 by a communication means such as e-mail. This allows the user (sender) of the sender smartphone 100b to recognize that the tentative reservation schedule has been confirmed via the interactive function of the answering machine.

[0104] The reject button 1305 is a button operated to reject a tentative reservation. As shown in FIG. 16 , when the reject button 1305 is operated, the recipient smartphone 100a deletes the tentatively reserved schedule from the schedule information 5000, as shown in FIG. 17 . The recipient smartphone 100a then notifies the sender smartphone 100b of a cancellation notification message 1800 shown in FIG. 18 via a communication means such as e-mail. This allows the user of the sender smartphone 100b to recognize that the tentatively reserved schedule has been canceled via the answering machine's interactive function. <Effects> As described above, the answering machine system according to the second embodiment of the present invention achieves the same effects as the first embodiment. Furthermore, the answering machine system according to the second embodiment provides a mechanism for obtaining approval from the user of the recipient smartphone 100a for a schedule tentatively adjusted using the answering machine's automatic interactive function, and a mechanism for notifying the sender of the approval or rejection of the schedule, thereby reducing the possibility of schedule adjustment errors. <<Modifications>> The present invention is not limited to the above-described embodiments, and various modifications can be adopted within the scope of the present invention. Furthermore, the above-described embodiments can be combined with each other without departing from the scope of the present invention.

[0105] In each of the above embodiments, an example has been described in which the sender's business is one that requires a schedule adjustment, but each of the above embodiments can also be applied to various types of business, such as one that requires adjustment other than a schedule adjustment or one that requires negotiation.

[0106] The features of the above-described embodiments may be applied to a video conference system, which is another example of an automatic response system on the network NW1. In this case, responses to comments made by conference participants in the video conference system may be generated.

[0107] In each of the above embodiments, an information terminal with a calling function other than the smartphone 100 may be used instead of the smartphone 100. Examples of information terminals with a calling function include mobile phones, landlines, tablet terminals, and PCs (Personal Computers). A server device (information processing device) connected to the network NW1 may provide an automatic answering function (e.g., an answering machine automatic dialogue function) to an information terminal with a calling function such as the smartphone 100. In this case, the server device (information processing device) may have the following configuration: [1] An information processing device having a calculation device and communicating with an information terminal, wherein the calculation device converts content received from the information terminal into text data when the information terminal performs an automatic answering, collects response information from internal and / or external information based on the text data, creates input data based on the text data and the response information, transmits the input data to an internal and / or external language generation AI, receives output data from the language generation AI based on the input data, and transmits the output data or data obtained by modifying the output data to the information terminal. [2] In the information processing device of [1], when transmitting data obtained by modifying the output data to the information terminal, the calculation device modifies the output data at a disclosure granularity according to the identity of the target person receiving the response.

[0108] DESCRIPTION OF SYMBOLS 100a... Receiver smartphone, 100b... Sender smartphone, 110... Main control unit, 130... Memory unit, 131... Program unit, 132... Data unit, 133... Program function unit, 170... Voice processing unit, 171... Voice input / output unit, 172... Voice recognition unit, 173... Voice decoding unit, 180... Communication processing unit, 181... LAN communication unit, 181... Communication unit, 182... Telephone network communication unit, 183... Near field wireless communication unit, 200... Generation AI server, 300... Conversation database server

Claims

1. An information terminal comprising a control unit and an automatic response unit including the control unit, wherein the control unit: when the information terminal responds using the automatic response unit, controls the automatic response unit to convert the content received by the automatic response unit into text data; collects response information from internal and / or external information based on the text data; creates input data based on the text data and the response information; sends the input data to an internal and / or external language generation AI; receives output data from the language generation AI based on the input data; and controls the output data to be sent to the automatic response unit.

2. An information terminal according to claim 1, wherein the control unit controls the information terminal to search for the answer information corresponding to the text data from the external and / or internal information and collect the answer information.

3. An information terminal according to claim 1, wherein the control unit extracts keywords from the text data, and based on the keywords, searches for information corresponding to the keywords from the external and / or internal information as the answer information, and controls the information terminal to collect the answer information.

4. An information terminal as described in claim 1, wherein the control unit conceals the text data, conceals the answer information, inputs the input data created based on the concealed text data and the concealed answer information to the external language generation AI, thereby causing the language generation AI to create the output data, and controls the language generation AI to restore the concealed information in the output data before modifying the output data.

5. An information terminal as claimed in claim 1, wherein the control unit controls the collection of response information by referring to external and / or internal schedule information and searching for information necessary for the schedule adjustment as the response information, when the matter of the person receiving the response requires a schedule adjustment.

6. An information terminal according to claim 5, comprising a display device capable of displaying images, wherein the control unit modifies the output data at a disclosure granularity according to the identity of the person receiving the response, converts the modified output data back into voice to create a response voice, and after the response operation by the automatic response unit is completed, displays on the display device an operation screen for deciding whether or not to approve the content of the modified output data, and controls the information terminal of the person receiving the response to be notified of information indicating whether or not the content of the modified output data has been approved based on the operation performed on the operation screen.

7. An information terminal according to claim 6, wherein the control unit controls the schedule information to reflect the approved schedule when the contents of the output data are approved based on the operation performed on the operation screen.

8. An information terminal as described in claim 1, wherein the control unit controls to modify the output data at a disclosure granularity according to the identity of the person receiving the response, and to convert the modified output data back into voice to create a response voice.

9. An information terminal according to claim 8, wherein the control unit determines a level of identity according to the identity of the subject receiving the response, and controls to modify the output data so that the lower the level of identity, the coarser the disclosure granularity becomes.

10. An information terminal as described in claim 8, wherein the control unit, when the matter of the person receiving the response is one that requires negotiation, searches for information necessary for the negotiation as the response information and controls to collect the response information.

11. An information terminal according to claim 8, wherein the control unit, when the matter of the person receiving the response is one that requires adjustment, searches for information necessary for the adjustment as the response information and controls to collect the response information.

12. An information terminal according to claim 10, comprising a display device capable of displaying images, wherein the control unit, after the response operation by the automatic response unit has finished, displays an operation screen on the display device for deciding whether or not to approve the content of the modified output data, and controls the information terminal of the person receiving the response to notify the information terminal of the person receiving the response of information indicating whether or not the content of the modified output data has been approved, based on the operation performed on the operation screen.

13. An information terminal according to claim 11, comprising a display device capable of displaying images, wherein the control unit, after the response operation by the automatic response unit has finished, displays an operation screen on the display device for deciding whether or not to approve the content of the modified output data, and controls the information terminal of the person receiving the response to notify the information terminal of the person receiving the response of information indicating whether or not the content of the modified output data has been approved, based on the operation performed on the operation screen.

14. An information terminal according to claim 1, wherein the automatic response unit is an answering machine unit.

15. An information processing device having a calculation device and communicating with an information terminal, wherein the calculation device converts content received from the information terminal into text data when the information terminal makes an automatic response, collects response information from internal and / or external information based on the text data, creates input data based on the text data and the response information, sends the input data to an internal and / or external language generation AI, receives output data from the language generation AI based on the input data, and sends the output data or data obtained by modifying the output data to the information terminal.

16. An automatic response system having an information terminal and a server device, wherein when the information terminal performs an automatic response, it converts received content into text data, collects response information from internal and / or external information based on the text data, creates input data based on the text data and the response information, and sends the input data to the server device, the server device receives the input data, sends the input data to an internal and / or external language generation AI, receives output data from the language generation AI, and sends the output data to the information terminal, and the information terminal receives the output data.

Images