Program, information processing device, method, and system

Abstract

To immediately execute a response process in response to a response from the called party. [Solution] A program to be executed by a computer having a processor and a memory. The program causes the processor to execute the following steps: acquiring audio stream data indicating a response sound from the called party when a call is made from an automated voice telephone; extracting features from the acquired audio stream data each time a certain amount of the audio stream data is accumulated; calculating an index for determining whether the response sound is a beep sound from an answering machine based on the extracted features; determining whether the multiple indexes calculated for each certain amount of audio stream data satisfy predetermined conditions; and controlling the automated voice telephone to respond to the response from the called party based on the determination result.

Description

[Technical Field]

[0001] The present disclosure relates to a program, an information processing device, a method, and a system. [Background technology]

[0002] In recent years, various technologies related to IVR (Interactive Voice Response: Automatic Voice Response System) have been researched and developed. For example, Patent Document 1 discloses a technology for identifying user input information based on the speech of a user who accepts an incoming call, extracting dialogue information based on the identified user input information, and setting an available input time based on input setting information in the extracted dialogue information. Here, the available input time for accepting user input information is set in advance in the input setting information according to the user input information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-076869 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology disclosed in Patent Document 1 does not automatically determine whether the answer from the called party is a human or a beep sound played on an answering machine when an automated voice call is made. In other words, the technology disclosed in Patent Document 1 does not automatically control the content of the response process in response to the answer from the called party according to the type of the answer sound on the called party. Therefore, the technology disclosed in Patent Document 1 has room for improvement in terms of immediately executing the response process in response to the answer from the called party.

[0005] An object of the present disclosure is to immediately execute a response process in response to a response from the called party. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, a program according to one aspect of the present disclosure is a program to be executed by a computer including a processor and a memory, which causes the processor to execute the following steps: acquiring audio stream data indicating a response sound from a call recipient when a call is made from an automated voice telephone; extracting features from the acquired audio stream data each time a certain amount of the audio stream data is accumulated; calculating an index for determining whether the response sound is a beep sound of an answering machine based on the extracted features; determining whether the multiple indexes calculated for each certain amount of the audio stream data satisfy predetermined conditions; and controlling a response process of the automated voice telephone in response to the response from the call recipient based on the determination result. [Effects of the Invention]

[0007] According to the present disclosure, a response process can be immediately executed in response to a response from the called party. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of the overall configuration of a system 1. FIG. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of a server 20. [Figure 3] FIG. 2 is a diagram showing the data structure of a user information database 2022. [Figure 4] FIG. 2 is a diagram showing the data structure of a voice guidance database 2023. [Figure 5] FIG. 10 is a diagram showing the data structure of an inquiry information database 2024. [Figure 6] FIG. 2 is a diagram showing the data structure of a system log database 2025. [Figure 7] 10 is a flowchart showing an example of the operation of the server 20. [Figure 8] FIG. 2 is a block diagram showing the basic hardware configuration of a computer 91. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings describing the embodiments, common components are designated by the same reference numerals, and repeated explanations will be omitted. Note that the following embodiments do not unduly limit the content of the present disclosure described in the claims. Furthermore, not all components shown in the embodiments are necessarily essential components of the present disclosure. Furthermore, each drawing is a schematic diagram and is not necessarily a precise illustration.

[0010] [1. Overview] A server according to this embodiment acquires audio stream data indicating a response sound from the called party when a call is made from an automated voice telephone. Each time a certain amount of acquired audio stream data is accumulated, the server according to this embodiment extracts a frequency spectrum (feature) from the certain amount of acquired audio stream data. Based on the extracted frequency spectrum, the server according to this embodiment calculates an index for determining whether the response sound from the called party is a beep sound from an answering machine. The server according to this embodiment determines whether multiple indexes calculated for each certain amount of audio stream data satisfy predetermined conditions, and controls the automated voice telephone's response process in response to the response from the called party according to the determination result.

[0011] [2 System Configuration] <2-1 Overall system configuration> Fig. 1 is a block diagram showing an example of the overall configuration of system 1. System 1 is, for example, a system for providing users with an IVR (Interactive Voice Response) service. The IVR service responds to phone calls from users with automated voice guidance, and based on information obtained through push buttons, responds with the answer the user desires through automated voice, or transfers the call to an operator in a section appropriate for that answer, if necessary.

[0012] 1 includes, for example, a telephone 10, a server 20, and an automatic voice telephone 30. The telephone 10 and the automatic voice telephone 30 are connected, for example, via a public network 90. ​​The automatic voice telephone 30 is assigned, for example, an outside line number in the public network 90.

[0013] The server 20 and the automated voice telephone 30 are connected for communication via, for example, a network 80. That is, in this embodiment, the system 1 is a cloud-based or on-premise IVR system. When the system 1 is an on-premise IVR system, the server 20 and the automated voice telephone 30 may be connected via, for example, an analog line, a digital line, or CTI (Computer Telephony Integration) middleware.

[0014] 1, for the sake of simplicity, an example is shown in which the system 1 includes one telephone 10. However, in reality, the system 1 includes as many telephones 10 as there are users.

[0015] 1 shows an example in which the system 1 includes one server 20, but for example, a collection of multiple devices may be one server 20. The way in which the multiple functions required to realize the server 20 are allocated to one or multiple pieces of hardware can be determined appropriately depending on the processing capacity of each piece of hardware and / or the specifications required for the server 20.

[0016] 1 shows an example in which the system 1 includes one automated voice telephone 30, but the system 1 may include two or more automated voice telephones 30. Also, in FIG. 1, an example in which the automated voice telephone 30 is independent from the server 20 is shown, but whether the system 1 is a cloud-based or on-premise IVR system, the server 20 may include the functionality of the automated voice telephone 30.

[0017] The telephone 10 is a telephone operated by a user, and may be any telephone that has the function of making voice calls via the public network 90. ​​The telephone 10 may be, for example, a landline telephone, an ISDN (Integrated Services Digital Network) telephone, a smartphone, a feature phone, or a tablet.

[0018] The server 20 is, for example, an information processing device for managing and operating an IVR service, and is realized by a computer connected to a network 80. As shown in Fig. 1, the server 20 includes a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29. The input / output IF 23 functions as an input device for receiving input operations from an IVR service representative (hereinafter, a service representative), and as an interface for an output device for outputting information to the service representative.

[0019] The automatic voice telephone 30 automatically responds to voice input from the telephone 10 and to user operations on the telephone 10, performs processing (response processing) according to the voice input, and returns the result to the telephone 10. The response processing executed by the automatic voice telephone 30 is controlled by the server 20.

[0020] Each information processing device such as the telephone 10, the server 20, and the automatic voice telephone 30 may be configured, for example, by a computer 91 (see FIG. 8) equipped with an arithmetic unit and a storage device. The basic hardware configuration of the computer 91 and the basic functional configuration of the computer 91 realized by the basic hardware configuration will be described later. Note that for each of the telephone 10, the server 20, and the automatic voice telephone 30, descriptions that overlap with the basic hardware configuration of the computer 91 and the basic functional configuration of the computer will be omitted.

[0021] <2-2 Server configuration> Fig. 2 is a block diagram showing an example of the configuration of the server 20 shown in Fig. 1. As shown in Fig. 2, the server 20 performs the functions of a communication unit 201, a storage unit 202, and a control unit 203.

[0022] The communication unit 201 performs processing for the server 20 to communicate with external devices (for example, the telephone 10, the automated voice telephone 30).

[0023] The storage unit 202 is realized by the memory 25 and the storage 26, and stores data and programs used by the server 20. The programs include application programs such as a web browser application. The storage unit 202 stores, for example, an app 2021, a user information database 2022, a voice guidance database 2023, an inquiry information database 2024, and a system log database 2025.

[0024] The app 2021 is application software for executing and managing IVR services. The user information database 2022 stores various information about users (hereinafter referred to as user information). The voice guidance database 2023 stores various information about voice guidance provided by the automated voice telephone 30 (hereinafter referred to as voice guidance information). The inquiry information database 2024 stores various information about inquiries from users (hereinafter referred to as inquiry information). The system log database 2025 stores logs of the system 1. The logs include, for example, the details of each process of the server 20 and the automated voice telephone 30, error information of the system 1, etc. Details of each database will be described later.

[0025] It should be noted that some or all of the above-mentioned databases may be stored in, for example, the automated voice telephone 30. Also, for example, the storage unit 202 may store a FAQ (Frequently Asked Questions) database. The FAQ database stores questions frequently asked by users and their answers. There are no particular restrictions on the criteria for determining whether a question is frequently asked by users, and the determination may be based on, for example, the experience, knowledge, etc. of the service representative.

[0026] The control unit 203 is realized by the processor 29 reading a program stored in the storage unit 202 and executing instructions included in the program. The control unit 203 controls the operation of the server 20. The control unit 203 operates in accordance with the read program to fulfill the functions of a reception control module 2031, a transmission control module 2032, a presentation control module 2033, and a response processing module 2034.

[0027] The reception control module 2031 controls the process by which the server 20 receives signals from external devices in accordance with a communication protocol. The transmission control module 2032 controls the process by which the server 20 transmits signals to external devices in accordance with a communication protocol. The presentation control module 2033 controls the process of presenting various types of information to the user.

[0028] The response processing module 2034 controls the overall response processing of the automatic voice telephone 30. In relation to one embodiment of the present disclosure, the response processing module 2034 executes the following processes.

[0029] The response processing module 2034 acquires audio stream data indicating a response tone from the called party (specifically, the telephone 10) when a call is made from the automatic voice telephone 30.

[0030] Various cases are conceivable for "making a call from the automated voice telephone 30," but examples of such cases include outbound calls. Outbound calls include reminders (e.g., reservation confirmations, payment due date notifications, etc.), event announcements, questionnaire surveys, emergency contacts such as disaster information, and introductions of new products and services.

[0031] The voice stream data is a digital data flow of voice that has temporal continuity. The voice stream data is expressed as digital data including elements such as sampling rate, bit depth, number of channels, and data format (codec). There are no particular limitations on the method of acquiring the voice stream data. The response processing module 2034 may acquire the voice stream data from the automated voice telephone 30 that has received a response tone from the called party, for example, via TCP (Transmission Control Protocol) communication.

[0032] The response processing module 2034 buffers the acquired audio stream data until a certain amount of audio stream data has been accumulated. There is no particular limit to the amount of audio stream data that the response processing module 2034 accumulates in one buffering. As an example, if the sampling rate is 8 kHz, the response processing module 2034 sets the buffer size of the audio stream data in one buffering to 800. In this example, the response processing module 2034 will execute the subsequent processing every 100 ms.

[0033] The response processing module 2034 extracts features from a certain amount of acquired audio stream data every time the certain amount of audio stream data is accumulated. For example, if the sampling rate of the audio stream data is set to 8 kHz and the buffer is set to 800, the response processing module 2034 extracts features every 100 ms.

[0034] In this embodiment, the response processing module 2034 performs FFT (Fast Fourier Transform) on a certain amount of audio stream data and extracts a frequency spectrum as a feature from the complex array that is the FFT output. For example, if the sampling rate of the audio stream data is set to 8 kHz, the domain of the horizontal axis (frequency axis) when the frequency spectrum is graphed will be 0 to 4 kHz according to the Nyquist frequency principle.

[0035] The response processing module 2034 may extract, as a feature, for example, a change in volume over time or the phase of the audio waveform. The volume in "change in volume over time" specifically refers to any of RMS (Root Mean Square) energy, the average of the absolute value of the amplitude, or the peak amplitude. The phase of the audio waveform is, for example, the phase of the frequency component extracted from the complex array that is the output of the FFT.

[0036] The response processing module 2034 calculates an index for determining whether the response sound on the called party is a beep sound of an answering machine or not, based on the extracted feature amount.

[0037] In this embodiment, the response processing module 2034 sparsifies the extracted frequency spectrum as preprocessing for calculating the index. There are no particular limitations on the sparsification method, and for example, threshold processing, quantization, compressed sensing, sparse coding, L1 regularization, or other methods may be adopted. By sparsifying the frequency spectrum, it is possible to achieve noise removal and data volume compression. Note that sparsifying the frequency spectrum is not a required process.

[0038] In this embodiment, the response processing module 2034 normalizes the extracted frequency spectrum and then calculates the similarity between the normalized frequency spectrum and the beep model. The response processing module 2034 uses the calculated similarity and the peak frequency corresponding to the maximum value of the normalized frequency spectrum (hereinafter referred to as the normalized peak frequency) as indices.

[0039] Specifically, for example, the response processing module 2034 generates a frequency spectrum model based on the normalized peak frequency obtained by normalizing the sparsified frequency spectrum. This model becomes the beep model. The "maximum value of the normalized frequency spectrum" is specifically the largest amplitude value among the amplitudes of the frequency components that make up the normalized frequency spectrum.

[0040] There are no particular limitations on the type and generation method of the beep model. In this embodiment, the beep model is assumed to be a normal distribution. The response processing module 2034, for example, sets the peak frequency as the mean μ of the normal distribution, determines the standard deviation σ in accordance with a predetermined standard (which can be set arbitrarily), and substitutes these values ​​into a probability density function (PDF) of the normal distribution to calculate the beep model.

[0041] In this embodiment, the response processing module 2034 calculates the KL (Kullback-Leibler) divergence as an index for evaluating the similarity between the normalized frequency spectrum (specifically, the amplitude spectrum of the frequency components) and the beep model. That is, in this embodiment, the KL divergence is an index for determining whether the answering sound on the receiving side is a beep sound from an answering machine. The reason why the KL divergence can be used as an index is that when a sparsified frequency spectrum is normalized, the normalized frequency spectrum can be treated as probability distribution data. The smaller the KL divergence value, the higher the similarity, i.e., the more similar the normalized frequency spectrum and the beep model are.

[0042] The response processing module 2034, for example, regards the normalized frequency spectrum and the beep model as two discrete probability distributions, and calculates the KL divergence using the following general formula (1) when the two discrete probability distributions are given. The response processing module 2034 calculates the KL divergence for each fixed amount of audio stream data.

number

[0043] The response processing module 2034 may calculate an index other than the KL divergence as an index for evaluating the similarity between the normalized frequency spectrum and the beep model. Examples of indices other than the KL divergence include total variation distance, Hellinger distance, squared distance, and cosine similarity. Furthermore, for example, the response processing module 2034 may calculate an index other than the similarity described above as an index for determining whether the response sound on the receiving side is a beep sound from an answering machine.

[0044] The response processing module 2034 determines whether or not a plurality of indicators calculated for each fixed amount of audio stream data satisfy a predetermined condition. If the plurality of indicators calculated for each fixed amount of audio stream data satisfy the predetermined condition, the response processing module 2034 determines that the response sound from the called party is a beep sound from an answering machine.

[0045] The predetermined condition may be any condition that can determine whether the answering sound from the receiving side is a beep from an answering machine. In this embodiment, the predetermined condition is composed of a first condition and a second condition. Specifically, the first condition is that a KL divergence that takes a value equal to or less than a predetermined threshold (which can be set arbitrarily) is calculated a certain number of times (which can be set arbitrarily). Furthermore, the second condition is that the peak frequencies used as the basis for calculating multiple KL divergences that satisfy the first condition are all the same. In other words, the predetermined condition is satisfied when both the first condition and the second condition are satisfied. However, the predetermined condition is not satisfied when the first condition is not satisfied, or when the first condition is satisfied but the second condition is not satisfied.

[0046] Here, the response processing module 2034 may, for example, update a predetermined threshold serving as a criterion for determining whether the first condition is satisfied. Specifically, for example, the response processing module 2034 may assign a label (e.g., "1" for satisfied, "0" for not satisfied) to the determination result of whether the first condition is satisfied. For example, the response processing module 2034 may associate a label with audio data (e.g., audio stream data) on which the determination result is based, and store the audio data in a separate storage area in the storage unit 202. For example, the response processing module 2034 may read a certain amount of audio data stored in the storage area, input the data to a trained machine learning model, and cause the machine learning model to output an inferred value (expressed as a label) of the determination result. The machine learning model may, for example, be a neural network or a generative AI model. It is assumed that the output process of this inferred value is executed as a batch process, for example, when a certain amount of audio data has been accumulated. The response processing module 2034 may update the predetermined threshold by, for example, multiplying the difference between the determination result of its own module and the inference value of the machine learning model by a learning coefficient and adding the result to the predetermined threshold. The response processing module 2034 performs this update process using, for example, the following equation (2).

number

[0047] The response processing module 2034 controls the response processing of the automatic voice telephone 30 in response to a response from the called party according to the determination result of whether or not a plurality of indicators calculated for each fixed amount of voice stream data satisfy a predetermined condition. In this embodiment, the response processing of the automatic voice telephone 30 in response to a response from the called party will be described in detail later.

[0048] [3 Data Structure] 3 to 6 are diagrams showing the data structures of the databases held by the system 1. Note that FIGS. 3 to 6 are merely examples and do not exclude data that is not listed. Furthermore, even data that is listed in the same database may be stored in separate storage areas within the database.

[0049] 3 is a diagram showing the data structure of the user information database 2022. The various pieces of information stored in the user information database 2022 may be entered by a service representative, for example, or the server 20 may extract the relevant information from user utterance data. The user information database 2022 has columns for name, age, sex, date of birth, contact information, and usage history, for example, with a user ID as a key.

[0050] The item "User ID" stores, for example, an identifier for uniquely identifying a user. The item "Name" stores, for example, the user's name (first and last name). The item "Age" stores, for example, the user's age. The item "Gender" stores, for example, the user's gender. The item "Date of Birth" stores, for example, the user's date of birth. The item "Contact Information" is an item that stores, for example, the user's contact information (e.g., phone number, email address, etc.). The item "Service Usage History" stores, for example, the dates on which the user previously used an IVR service (usage history).

[0051] 4 is a diagram showing the data structure of the voice guidance database 2023. Various information stored in the voice guidance database 2023 may be input in advance by, for example, a service representative. The voice guidance database 2023 has columns for type, guidance content, options, and transition destination, with, for example, a guidance ID as a key.

[0052] The item "guidance ID" stores, for example, identification information (identifier) ​​for uniquely identifying the type of voice guidance.

[0053] The item "type" stores, for example, the type of voice guidance. In the example of FIG. 4, the types of voice guidance include guidance transition, operator connection, information provision, and end, but types other than these may also be stored in the item "type." Guidance transition is voice guidance that informs the user that they will be guided to the next voice guidance. Operator connection is voice guidance that guides the user to connect to an operator. Information provision is voice guidance that guides the user to provide information corresponding to the content of the user's utterance. End is voice guidance that guides the user to end the call with the user.

[0054] The item "Guidance Content" stores, for example, guidance content for each type of audio guidance. Specifically, for example, the item "Guidance Content" stores reference information (path) of an audio file indicating the guidance content for each type of audio guidance. The reference information is, for example, a file path, a URL, etc. The audio file is stored in a separate storage area in the storage unit 202, for example, a file system or cloud storage. There are no particular limitations on the format of the audio file, and it can be, for example, MP3, WAV, or AAC.

[0055] The item "options" stores, for example, options for each type of audio guidance. The options are a means for selecting the next action when audio guidance corresponding to the reference information stored in the item "guidance content" is being played back.

[0056] In addition, the user information database 2022 may store, for example, at least one of the user's account information, transaction history with the user, or contract information of a contract regarding the use of IVR services concluded with the user.

[0057] In the example of Fig. 4, for example, when the voice guidance for the guidance transition corresponding to reference information 1, "If you have any questions about XX, please press 1," is being played, the push number "1," which is an option, is stored in the item "options" linked to the guidance transition. Also, for example, when the voice guidance for the questionnaire survey corresponding to reference information 4, "This call is for a questionnaire survey about XX," is being played, the beep sound that is the response from the telephone 10 is regarded as an option and is stored in the item "options" linked to the questionnaire survey.

[0058] The item "Transition destination" stores, for example, a transition destination for each type of voice guidance. The transition destination defines the next action that the system 1 takes when the user selects an option stored in the item "Options."

[0059] In the example of FIG. 4 , for example, a guidance ID "G0005" is stored in the "Transition Destination" item associated with the guidance transition. That is, when the voice guidance for the guidance transition corresponding to reference information 1, "Press 1 for inquiries about XX," is being played back, if the user presses the push button number "1," which is an option, the system transitions to the type of voice guidance identified by the guidance ID "G0005." Also, for example, a "Transition Destination" item associated with a questionnaire survey stores "Play / Disconnect Voice Message for Answering Machine." That is, when the voice guidance for the questionnaire survey corresponding to reference information 4, "This call is for a questionnaire survey about XX," is being played back, if the telephone 10 responds with a beep, the voice message for answering machine is played back, and then the call between the telephone 10 and the automated voice telephone 30 is disconnected.

[0060] 5 is a diagram showing the data structure of the inquiry information database 2024. The various information stored in the inquiry information database 2024 may be input by, for example, a service representative or a person who responds to an inquiry from a user. The service representative and the person who responds may be the same person or different people. The inquiry information database 2024 has, for example, an inquiry ID as a key and columns for user ID, date and time, inquiry content, responder ID, and response status.

[0061] The item "Inquiry ID" stores, for example, identification information (identifier) ​​for uniquely identifying the content of the inquiry. The item "User ID" stores, for example, identification information (identifier) ​​for uniquely identifying the user who made the inquiry. The identification information stored in the item "User ID" of the inquiry information database 2024 is, for example, the same as the identification information stored in the item "User ID" of the user information database 2022. The item "Date and Time" stores, for example, the date and time when an inquiry was made from a user. The item "Inquiry Content" stores, for example, the content of the inquiry (the content of the inquiry from the user). The item "Responder ID" stores, for example, identification information (identifier) ​​for uniquely identifying the responder. The item "Response Status" stores, for example, the response status of the responder. The response status includes, for example, the date and time when the responder responded to the inquiry, the response content of the responder, the user's reaction, the progress after the response, etc.

[0062] 6 is a diagram showing the data structure of the system log database 2025. The various information stored in the system log database 2025 may be entered by, for example, a service representative or an administrator of the system 1, or may be automatically acquired by the server 20. The service representative and the administrator may be the same person or different people. The system log database 2025 has columns for date and time, processing content, and error information, with an event ID as a key, for example.

[0063] The item "Event ID" stores, for example, identification information (identifier) ​​for uniquely identifying an event that has occurred in system 1. The event includes, for example, the operating status of the server 20 and the automated voice telephone 30, the communication status within system 1, error information, etc. The item "Date and Time" stores, for example, the date and time when the event occurred. The item "Processing Content" stores, for example, the content of the processing performed by at least one of the server 20 and the automated voice telephone 30. The content of this processing includes, for example, the call history with the user, performance data of system 1 (e.g., response time of system 1, processing speed, etc.), etc. The item "Error Information" stores, for example, an outline of an error that has occurred in system 1. In other words, the items "Processing Content" and "Error Information" both store the content of an event that has occurred in system 1. Note that the item "Error Information" may store, for example, details of an error that has occurred in system 1.

[0064] [4 actions] With reference to Figure 7, an example of the operation of the server 20 when controlling the automatic voice telephone 30 in accordance with the determination result of whether the answering sound from the called party is a beep sound of an answering machine or not will be described. Figure 7 is a flowchart showing an example of the operation of the server 20 when controlling the automatic voice telephone 30 in accordance with the above-mentioned determination result. In this example of operation, it is assumed that a call is being made from the automatic voice telephone 30 to the telephone 10.

[0065] 7, the server 20 acquires voice stream data indicating a response tone from the receiving side (specifically, the telephone 10). Specifically, for example, the response processing module 2034 acquires the voice stream data from the automatic voice telephone 30 that has received the response tone from the receiving side via TCP communication. The response processing module 2034 buffers the acquired voice stream data until a certain amount of voice stream data is accumulated, for example.

[0066] In step S12, every time a certain amount of acquired audio stream data is accumulated, the server 20 extracts a frequency spectrum from the certain amount of audio stream data. Specifically, for example, the response processing module 2034 performs FFT on the certain amount of audio stream data and extracts a frequency spectrum from the complex number array that is the output of the FFT.

[0067] In step S13, the server 20 calculates the KL divergence and the normalized peak frequency based on the extracted frequency spectrum.

[0068] Specifically, for example, the response processing module 2034 sparsifies the extracted frequency spectrum as preprocessing for calculating the KL divergence. The response processing module 2034 generates a beep model based on a normalized peak frequency obtained by normalizing the sparsified frequency spectrum. The response processing module 2034 calculates the KL divergence by substituting the normalized frequency spectrum and the beep model into the above-mentioned general formula (1). The response processing module 2034 calculates the KL divergence and the normalized peak frequency for each fixed amount of audio stream data.

[0069] In step S14, the server 20 determines whether the KL divergences and normalized peak frequencies calculated for each fixed amount of audio stream data satisfy a predetermined condition. If the KL divergences and normalized peak frequencies satisfy the predetermined condition, the server 20 determines that the answering sound from the called party is a beep from an answering machine.

[0070] Specifically, for example, the response processing module 2034 determines whether or not multiple KL divergences calculated for each certain amount of audio stream data satisfy a first condition (KL divergences that take values ​​equal to or less than a predetermined threshold are calculated a certain number of times in succession). If the first condition is satisfied, the response processing module 2034 determines, for example, whether or not multiple normalized peak frequencies calculated for each certain amount of audio stream data satisfy a second condition (the normalized peak frequencies that served as the basis for calculating the multiple KL divergences that satisfy the first condition are all the same). If both the first and second conditions are satisfied, the response processing module 2034 determines, for example, that the response sound from the called party is a beep from an answering machine.

[0071] On the other hand, if the first condition is not satisfied, or if the first condition is satisfied but the second condition is not satisfied, the response processing module 2034 may continue the processing of step S14. Also, for example, if the first condition is not satisfied even after the processing of step S14 is continued for a predetermined time (which can be set arbitrarily), or if the first condition is satisfied but the second condition is not satisfied, the response processing module 2034 may determine that the response sound from the receiving side is the user's speech voice.

[0072] In step S15, the server 20 controls the response process of the automatic voice telephone 30 in response to a response from the receiving party based on the result of determining whether or not the multiple KL divergences and multiple normalized peak frequencies calculated for each certain amount of voice stream data satisfy predetermined conditions.

[0073] There are various variations in the response process of the automatic voice telephone 30 in response to a response from the called party. Specifically, for example, when the response process module 2034 determines that the response sound from the called party is a beep sound from an answering machine, it may read out an audio file for the answering machine from a separate storage area in the storage unit 202 and send it to the automatic voice telephone 30. For example, the response process module 2034 may cause the automatic voice telephone 30 to play the sent audio file for the answering machine and then disconnect the call with the telephone 10.

[0074] In this case, the response processing module 2034 may cause the automated voice telephone 30 to play, for example, a special announcement message as an audio file for the answering machine. The special announcement message may include, for example, a message indicating a high level of urgency or importance, a message requesting a third party to leave a message, or a message indicating that the call will be made again. The special announcement message may include, for example, "This is an automated voice message from the XX service. It appears your call has been connected to the answering machine. We will contact you again later," thereby helping the caller of the automated voice telephone 30 (e.g., a service representative) to clearly recognize the current connection status (that the call is on the answering machine). The content of this message may be selected appropriately from the audio guidance database 2023 in the storage unit 202 depending on the purpose of the call, the level of urgency, etc.

[0075] That is, when it is determined that the response sound from the called party is a beep sound from an answering machine (when the determination result indicates that multiple indicators satisfy predetermined conditions), the response processing module 2034 may cause the automated voice telephone 30 to play a predetermined sound (e.g., a special announcement message) that is different from the standard response sound. The standard response sound is, for example, the response sound that the automated voice telephone 30 plays when the response sound from the called party is the user's spoken voice.

[0076] In this way, by having the automated voice telephone 30 play a predetermined sound such as a special announcement message, the caller of the automated voice telephone 30 can not only detect the beep, but also obtain information to more appropriately determine how to respond after the predetermined sound is played.

[0077] Furthermore, for example, when the response processing module 2034 determines that the response sound from the receiving side is the user's speech, the response processing module 2034 may perform speech recognition on the response sound to acquire language information. In this process, the response processing module 2034 may combine speech recognition and natural language processing. For example, the response processing module 2034 may control the response processing of the automated voice telephone 30 according to the acquired language information.

[0078] More specifically, if the acquired language information indicates an inquiry requiring connection to an operator, the response processing module 2034 may, for example, read out an audio file for operator connection guidance from a separate storage area in the storage unit 202, transmit the audio file to the automated voice telephone 30, and cause the automated voice telephone 30 to play the audio file. Alternatively, if the acquired language information indicates a request for an explanation of a product, the response processing module 2034 may, for example, read out an audio file for product explanation from a separate storage area in the storage unit 202, transmit the audio file to the automated voice telephone 30, and cause the automated voice telephone 30 to play the audio file.

[0079] Furthermore, for example, the response processing module 2034 may cause the automated voice telephone 30 to set a flag in the user data registered in a telephone directory (not shown) according to the determination result. The telephone directory may be stored in the automated voice telephone 30, for example.

[0080] More specifically, for example, when the response processing module 2034 determines that the response sound from the called party is a beep from an answering machine, it may cause the automated voice telephone 30 to set a flag indicating "contacted, not in" in the data of the corresponding user. Alternatively, for example, when the response processing module 2034 determines that the response sound from the called party is a user's spoken voice, it may cause the automated voice telephone 30 to set a flag indicating the content according to the voice recognition result of the response sound (e.g., "contacted, operator connection requested," "contacted, product explanation requested," etc.) in the data of the corresponding user.

[0081] The telephone directory may be stored in, for example, the storage unit 202. In this case, the response processing module 2034 may, for example, set a flag on the data of a user registered in the telephone directory.

[0082] [5 Summary] As described above, in this embodiment, the response processing module 2034 buffers acquired audio stream data until a certain amount of audio stream data is accumulated. The response processing module 2034 performs an FFT on the certain amount of audio stream data and extracts a frequency spectrum from the complex array that is the FFT output. The response processing module 2034 sparsifies the extracted frequency spectrum. The response processing module 2034 generates a beep model based on normalized peak frequencies obtained by normalizing the sparsified frequency spectrum. The response processing module 2034 calculates KL divergence based on the normalized frequency spectrum and the generated beep model. The response processing module 2034 determines whether multiple KL divergences and multiple normalized peak frequencies calculated for each certain amount of audio stream data satisfy predetermined conditions. If the predetermined conditions are satisfied, the response processing module 2034 determines that the response sound from the called party is a beep sound from an answering machine. The response processing module 2034 controls the response processing of the automatic voice telephone 30 in response to a response from the receiving party, depending on the result of determining whether or not the multiple KL divergences and multiple normalized peak frequencies calculated for each certain amount of voice stream data satisfy predetermined conditions.

[0083] As a result, when a call is made from the automatic voice telephone 30 and the automatic voice telephone 30 receives a response tone from the receiving party (specifically, the telephone 10), the server 20 can automatically control the response process of the automatic voice telephone 30 in response to the response tone according to the manner of the response tone. Therefore, the server 20 can instantly control the response process of the automatic voice telephone 30 in response to a response from the receiving party. In other words, the automatic voice telephone 30 can instantly execute the response process in response to a response from the receiving party.

[0084] [6. Modifications] <6-1 First Modification: Determination Process Using Other Indicators> In this embodiment, an example has been described in which KL divergence and normalized peak frequency are used as indices for determining whether the answering sound from the called party is a beep sound of an answering machine. However, the server 20 may use other indices to determine whether the answering sound from the called party is a beep sound of an answering machine. Below, several examples of determination processes using other indices will be described.

[0085] First, the server 20 may, for example, normalize the extracted frequency spectrum, and use the maximum value of the normalized frequency spectrum (hereinafter referred to as the normalized maximum value) and the normalized peak frequency as indicators for determining whether the response sound from the called party is a beep sound from an answering machine.

[0086] In this modified example, the response processing module 2034 may, for example, employ the same first and second conditions as in the present embodiment as the predetermined conditions. That is, the response processing module 2034 may, for example, employ a first condition that a normalized maximum value that is equal to or less than a predetermined threshold is calculated a certain number of times in succession. The response processing module 2034 may, for example, employ a second condition that the normalized peak frequencies corresponding to the plurality of normalized maximum values ​​that satisfy the first condition are all identical.

[0087] The response processing module 2034 may, for example, determine whether multiple normalized maximum values ​​and multiple normalized peak frequencies calculated for each certain amount of audio stream data satisfy a first condition and a second condition. If both the first condition and the second condition are satisfied, the response processing module 2034 may, for example, determine that the predetermined condition is satisfied and that the response sound from the called party is a beep from an answering machine. On the other hand, if the first condition is not satisfied, or if the first condition is satisfied but the second condition is not satisfied, the response processing module 2034 may, for example, determine that the predetermined condition is not satisfied and that the response sound from the called party is a user's speech.

[0088] Next, the server 20 may calculate, for example, a maximum volume and a peak frequency based on the extracted frequency spectrum. The server 20 may use, for example, a first differential value obtained by differentiating the calculated maximum volume and a second differential value obtained by differentiating the calculated peak frequency as indicators for determining whether the answering sound from the called party is a beep from an answering machine.

[0089] Specifically, for example, the response processing module 2034 may square the amplitudes of all frequency components in the extracted frequency spectrum, sum them, and take the square root of the sum. The response processing module 2034 may set the square root of the obtained sum as the maximum volume. Furthermore, for example, the response processing module 2034 may compare the amplitudes of all frequency components in the extracted frequency spectrum and set the value with the largest amplitude as the maximum volume.

[0090] The response processing module 2034 may, for example, determine the frequency corresponding to the largest amplitude among the amplitudes of all frequency components in the extracted frequency spectrum as the peak frequency.

[0091] The response processing module 2034 may, for example, calculate a first derivative value by differentiating the obtained maximum volume. The first derivative value indicates, for example, the rate of change over time of the response sound from the receiving side. The response processing module 2034 may, for example, calculate a second derivative value by differentiating the obtained peak frequency. The second derivative value indicates, for example, the rate of change over time of the peak frequency. The second derivative value is the first derivative value corresponding to the maximum frequency of the extracted multiple frequency spectra.

[0092] In this modification, the response processing module 2034 may, for example, analyze a time series pattern of the first differential value calculated for each of the extracted frequency spectrums and compare the analysis result with a time series pattern of the differential value of the maximum volume of the beep sound of the answering machine. The response processing module 2034 may, for example, analyze a time series pattern of the second differential value calculated for each of the extracted frequency spectrums and compare the analysis result with a time series pattern of the differential value of the peak frequency of the beep sound of the answering machine. The time series patterns of the differential values ​​of the maximum volume and peak frequency of the beep sound of the answering machine may be stored in advance in the storage unit 202, for example.

[0093] The response processing module 2034 may determine, for example, whether or not the time series pattern of the first differential value obtained by analysis matches the time series pattern of the differential value of the maximum volume of the beep sound of the answering machine at a certain rate (which can be set arbitrarily) or more (third condition).The response processing module 2034 may determine, for example, whether or not the time series pattern of the second differential value obtained by analysis matches the time series pattern of the differential value of the peak frequency of the beep sound of the answering machine at a certain rate (which can be set arbitrarily) or more (fourth condition).

[0094] For example, if the third condition and the fourth condition are satisfied, the response processing module 2034 may determine that the predetermined condition is satisfied and that the response sound from the called party is a beep from an answering machine. On the other hand, if at least one of the third condition and the fourth condition is not satisfied, the response processing module 2034 may determine that the predetermined condition is not satisfied and that the response sound from the called party is the user's speech.

[0095] <6-2 Second Modification: Determining the Response Type> In the present embodiment, an example has been described in which the server 20 continues the determination process of whether a beep is sounded if the plurality of indicators do not satisfy the predetermined conditions. However, for example, if the determination result indicates that the plurality of indicators do not satisfy the predetermined conditions, the server 20 may determine whether the response from the called party is a human speech or an automated voice response by analyzing the acquired audio stream data (determination of response type). The automated voice response includes, for example, an answering machine response message (a pre-recorded message), a beep that prompts the answering machine to start recording, a mechanically generated voice, etc. The server 20 may, for example, control the response process of the automated voice telephone 30 in response to the response from the called party according to the determination result.

[0096] Specifically, for example, the response processing module 2034 may analyze the acquired audio stream data for features such as voice activity detection (VAD), the length of silent periods, the periodicity of speech patterns, and the energy distribution of specific frequency bands, in addition to changes in frequency spectrum and volume over time.

[0097] These feature values ​​tend to detect, for example, human speech that begins with a relatively short response time and has an irregular speech pattern or diverse frequency components. Also, for example, answering machine response messages are followed by a relatively long, standardized message with the repetition of a specific speaker's voice quality or intonation pattern. Furthermore, messages often end with a distinctive tone or a short silent interval.

[0098] The response processing module 2034 may determine the response type by analyzing these features in a composite manner using, for example, a predefined rule-based decision logic or a trained machine learning model, such as a support vector machine, a neural network, or a random forest.

[0099] The response processing module 2034 may, for example, store the determination result in the system log database 2025. The response processing module 2034 may, for example, refer to the voice guidance database 2023, select a voice message corresponding to the determination result, and have the automated voice telephone 30 play it back.

[0100] <6-3 Third Modification: Personalizing Voice Messages> In the present embodiment, an example has been described in which the server 20 causes the automatic voice telephone 30 to play back a voice message that has been generated in advance by referring to the voice guidance database 2023. However, the server 20 may, for example, dynamically personalize the voice message that the automatic voice telephone 30 plays back for the recipient.

[0101] That is, the server 20 may acquire information about the receiving party when a call is made from the automated voice telephone 30, for example. The information about the receiving party includes, for example, whether the response of the receiving party is a speech by the receiving party or an automated voice response (response type), and various information about the receiving party stored in the user information database 2022 (e.g., the receiving party's attributes, past usage history, language setting, etc.). The server 20 may acquire the response type by, for example, executing the determination process of the second modified example. For example, the server 20 may cause the automated voice telephone 30 to play a predetermined voice corresponding to the receiving party when a call is made from the automated voice telephone 30, based on the determination result of whether or not a plurality of indicators satisfy predetermined conditions and the acquired information about the receiving party.

[0102] Specifically, for example, the voice guidance database 2023 may store a plurality of voice clips or text templates for voice synthesis in association with purposes and conditions (e.g., response type, customer segment, campaign information, etc.). The response processing module 2034 may generate a personalized voice message by selecting and combining optimal voice clips or templates according to priority, for example, based on the current call situation (response type, callee database information, etc.). This priority logic can be preset by the service representative. Examples of personalized voice messages include, "Dear Mr. / Ms. XX, thank you for your continued patronage. Today, I'd like to inform you about XX," or, in the case of an answering machine, "Dear Mr. / Ms. XX, this is XX, your contact. I'll contact you again later."

[0103] For example, the response processing module 2034 may input the acquired information about the receiving party to an AI model and cause the AI ​​model to output voice data as a personalized voice message. The AI ​​model may be, for example, a generative AI model such as a large-scale language model.

[0104] The response processing module 2034 may, for example, transmit the generated voice data (personalized voice message) to the automated voice telephone 30 and cause the automated voice telephone 30 to play back the voice data.

[0105] <6-4 Fourth Variation: Customizing Voice Messages> When the automatic voice telephone 30 leaves a voice message on the answering machine of the called party, the server 20 may select an optimum voice message according to the situation of the called party, for example, and have the automatic voice telephone 30 play it back, instead of using a standard voice message.

[0106] Specifically, for example, the response processing module 2034 may dynamically generate a voice message using a template whose content corresponds to the call recipient's customer segment, the time period when the call is made (e.g., during business hours, at night, on a holiday, etc.), past response patterns, etc. The response processing module 2034 may, for example, send the generated voice message to the automated voice telephone 30 and cause the automated voice telephone 30 to play back the voice data.

[0107] More specifically, the response processing module 2034 may perform control such as selecting a more polite and specific message for an important customer's answering machine, and avoiding less urgent messages at night.

[0108] The customer segments may be, for example, new customers, important customers, specific service users, etc., and may be stored in the user information database 2022. The past response patterns may be, for example, the time periods when calls are usually answered, the frequency of calls being left on an answering machine, etc. The system log database 2025 may store, for example, past response patterns of callers or similar segments. The voice guidance database 2023 may store, for example, voice message templates.

[0109] <6-5 Fifth Modification: Improving the efficiency of operator work> For example, if the server 20 determines that the call recipient is an answering machine, the server 20 may automate post-processing that requires manual intervention by an operator. The determination that the call recipient is an answering machine is made, for example, by detecting a beep according to the present embodiment or by determining the response type according to the second modification.

[0110] Specifically, for example, the response processing module 2034 may automate the following post-processing: By automating the following post-processing, the operator is freed from repetitive tasks and can concentrate on higher value-added tasks. Automatic recording of call result: The result of the call is automatically recorded as "answering machine connection" in the system log database 2025 or in a linked CRM (customer relationship management) system (not shown). · Contact status update: The contact status of the caller stored in the system log database 2025 is automatically updated to "Answering machine (absent)" or the like. · Automatic task generation: If necessary, follow-up tasks for the operator (e.g., calling back later, preparing to send information, etc.) are automatically generated within the CRM system. Notification: When certain conditions are met (e.g., when an important customer receives voicemail multiple times in a row), the operator in charge will be automatically notified (e.g., by email, in-system notification, etc.).

[0111] <6-6 Sixth Variation: Automatic Scheduling> For example, when the server 20 determines that the called party has an answering machine or that the called party has not answered, the server 20 may automatically set and schedule an optimal call-back time. The server 20 may execute the process according to the sixth modified example in cooperation with the process of determining the response type according to the second modified example.

[0112] Specifically, for example, the response processing module 2034 may schedule based on the following information: Past response history: Past call dates and times and connection results (answer by the callee, answering machine, no answer, busy, etc.) for the called number stored in the system log database 2025. Response pattern analysis results: Results of analyzing trends in response rates on specific days of the week and in specific time periods. For example, if analysis results show that human response rates are high in the mornings on weekdays based on past data, those time periods are prioritized as candidates for recall calls. The analysis results may be stored, for example, in the user information database 2022. · Recall policy: Follows a policy that is set in advance, such as an upper limit on the number of recalls, minimum / maximum values ​​for the recall interval, etc. The recall policy may be stored in the voice guidance database 2023, for example.

[0113] For example, based on this information, the response processing module 2034 may predict the date and time when the recipient is most likely to respond using a rule-based algorithm or a trained machine learning model, and register this as a re-call task in the dialer function of the automated voice telephone 30 or an associated system.

[0114] <6-7 Variation 7: Call Pattern Prediction> For example, the server 20 may analyze call data stored in various databases using a trained machine learning model (hereinafter, a prediction model) to predict response patterns for individual customers (call recipients) or customer segments.

[0115] The call data includes, for example, the date and time of the call, various information about the customer (including attribute information), the response type, the call duration, past call results, etc. The customer attribute information includes, for example, the age, sex, region, contracted service details stored in the user information database 2022, and past inquiry history stored in the inquiry information database 2024. The past call results include, for example, the most recent call result, the sequence of response types for the past n calls, etc.

[0116] Specifically, for example, the response processing module 2034 may read necessary call data from various databases, input the data to a prediction model, and output a predicted response pattern (hereinafter, a predicted response pattern) from the prediction model. The predicted response pattern may include, for example, the probability of a human response in a specific time period, the probability of being connected to an answering machine, etc.

[0117] The response processing module 2034 may automatically generate a "call plan" consisting of a combination of a call list and a time period that is predicted to have the highest response rate based on the output predicted response pattern, for example. This is expected to improve the connection efficiency and productivity of the entire call center of the IVR service.

[0118] <6-8 Eighth Variation: Answering Machine Pattern Learning> The server 20 may, for example, acquire various data related to answering machines and analyze the various data from multiple angles using a rule-based algorithm or a trained machine learning model. The various data may be aggregated, for example, from the inquiry information database 2024, the system log database 2025, etc., and may be compiled and analyzed periodically or in real time. The server 20 may, for example, automatically generate a dashboard that visualizes the analysis results in real time, for example, via the app 2021, and present the dashboard to the service representative.

[0119] Examples of analysis items and KPIs (key performance indicators) that may be displayed on the dashboard mentioned above are as follows: · Voicemail Connection Rate: The percentage of voicemail connections to all outgoing calls (by time of day, day of the week, campaign, etc.). Beep detection accuracy: The accuracy rate and recall rate of beep detection by the server 20 (e.g., comparison with manual confirmation data). · Recall success rate: The rate at which a human responds when retrying a call after connecting to voicemail. Message effectiveness measurement: Callback rate and subsequent conversion rate when leaving a specific voicemail message (e.g., personalized voice message for variant 3, customized voice message for variant 4, etc.). Response type distribution: Percentage of human responses, answering machine messages, and beep sounds, and their trends.

[0120] By viewing this information in real time on a dashboard, service representatives can make quick, data-based decisions, such as immediately adjusting campaign strategies, optimizing operator deployment, and improving IVR scenarios.

[0121] [7 Basic Computer Hardware Configuration] 8 is a block diagram showing the basic hardware configuration of a computer 91. The computer 91 includes at least a processor 901, a main memory device 902, an auxiliary memory device 903, and a communication IF 991 (interface), which are electrically connected to one another by a communication bus 921.

[0122] The processor 901 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

[0123] The main memory device 902 is used to temporarily store programs, data to be processed by the programs, etc. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0124] The auxiliary storage device 903 is a storage device for saving data and programs, such as a flash memory, a hard disk drive (HDD), a magneto-optical disk, a CD-ROM, a DVD-ROM, or a semiconductor memory.

[0125] The communication IF 991 is an interface for inputting and outputting signals for communicating with other computers via a network using wired or wireless communication standards.

[0126] The network is composed of the Internet, a LAN, various mobile communication systems constructed by wireless base stations, etc. For example, the network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks (e.g., Wi-Fi (registered trademark)) that can connect to the Internet via a predetermined access point. In the case of a wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), and Bluetooth (registered trademark). In the case of a wired connection, the network also includes a direct connection using a USB (Universal Serial Bus) cable, etc.

[0127] The computer 91 can be virtually realized by distributing all or part of each hardware configuration across multiple computers 91 and interconnecting them via a network. In this way, the computer 91 is a concept that includes not only a computer 91 housed in a single housing or case, but also a virtualized computer system.

[0128] [8 Basic Functional Configuration of a Computer] A description will now be given of the functional configuration of a computer realized by the basic hardware configuration (FIG. 8) of the computer 91. The computer comprises at least the functional units of a control unit, a storage unit, and a communication unit.

[0129] The functional units of the computer 91 can also be realized by distributing all or part of the functional units to multiple computers 91 interconnected via a network. The computer 91 is a concept that includes not only a single computer 91 but also a virtualized computer system.

[0130] The control unit is realized by the processor 901 reading out various programs stored in the auxiliary storage device 903, expanding them in the main storage device 902, and executing processing in accordance with the programs. The control unit can realize functional units that perform various types of information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0131] The storage unit is realized by a main storage device 902 and an auxiliary storage device 903. The storage unit stores data, various programs, and various databases. Furthermore, the processor 901 can allocate a storage area corresponding to the storage unit in the main storage device 902 or the auxiliary storage device 903 in accordance with the programs. Furthermore, the control unit can cause the processor 901 to execute processes for adding, updating, and deleting data stored in the storage unit in accordance with the various programs.

[0132] A database refers to a relational database, which manages data sets called masters and tables in a tabular format structurally defined by rows and columns, by relating them to each other. In a database, a table is called a table, a master, a column in a table is called a column, and a row in a table is called a record. In a relational database, relationships between tables and masters can be set and associated.

[0133] Typically, each table and each master has a column set as a primary key to uniquely identify a record, but setting a primary key to a column is not essential. The control unit can cause the processor 901 to add, delete, or update records in specific tables and masters stored in the storage unit according to various programs.

[0134] Furthermore, by storing data, various programs, and various databases in the storage unit, it can be considered that the information processing device and information processing system according to the present disclosure have been manufactured.

[0135] Note that the databases and masters in this disclosure may include any data structure in which information is structurally defined (such as a list, dictionary, associative array, or object). The data structure also includes data that can be considered as a data structure by combining data with functions, classes, methods, etc. written in any programming language.

[0136] The communication unit is realized by the communication IF 991. The communication unit realizes the function of communicating with other computers 91 via a network. The communication unit can receive information transmitted from other computers 91 and input the information to the control unit. The control unit can cause the processor 901 to execute information processing on the received information in accordance with various programs. In addition, the communication unit can transmit information output from the control unit to other computers 91.

[0137] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that implements the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, and ROMs.

[0138] Furthermore, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Java (registered trademark), JavaScript, and TypeScript.

[0139] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or the storage medium.

[0140] The functions performed by the components described herein may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (Central Processing Units), conventional circuits, and / or combinations thereof, programmed to perform the described functions. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. A processor may also be a programmed processor that executes a program stored in a memory.

[0141] In this specification, a circuitry, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0142] If the hardware is a processor considered to be a type of circuitry, the circuitry, means, or unit is a combination of the hardware and the software used to configure the hardware and / or processor.

[0143] Although several embodiments of the present disclosure have been described above, these embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are intended to be included in the scope of the inventions and their equivalents as defined in the claims, as well as in the scope and spirit of the inventions.

[0144] [9. Appendix] The matters described in the above embodiments will be supplemented below.

[0145] <Appendix 1> A program to be executed by a computer having a processor and a memory, the program causing the processor to execute the following steps: acquiring audio stream data indicating an answering sound from the receiving party when a call is made from an automated voice telephone; extracting features from the acquired audio stream data each time a certain amount of the audio stream data is accumulated; calculating an index for determining whether the answering sound is a beep sound from an answering machine based on the extracted features; determining whether multiple indexes calculated for each certain amount of audio stream data satisfy predetermined conditions; and controlling the answering process of the automated voice telephone in response to the answer from the receiving party depending on the determination result.

[0146] <Appendix 2> The program according to claim 1, wherein in the determining step, it is determined that the response sound is a beep sound if a plurality of indicators satisfy a predetermined condition.

[0147] <Appendix 3> A program described in (Appendix 1) or (Appendix 2), wherein in the extracting step, the frequency spectrum of a certain amount of audio stream data is extracted as a feature, and in the calculating step, the extracted frequency spectrum is normalized, and the similarity between the normalized frequency spectrum and a frequency spectrum model generated based on the peak frequency corresponding to the maximum value of the normalized frequency spectrum is used as an index.

[0148] <Appendix 4> In the extracting step, the frequency spectrum of a certain amount of audio stream data is extracted as a feature, and the calculating step includes a step of normalizing the extracted frequency spectrum and generating a frequency spectrum model based on a peak frequency corresponding to the maximum value of the normalized frequency spectrum, and a step of calculating the similarity between the normalized frequency spectrum and the generated model, and using the calculated similarity and peak frequency as indices. (Appendix 1) or (Appendix 2) The program described in the above.

[0149] <Appendix 5> A program described in (Appendix 1) or (Appendix 2), wherein in the extracting step, the frequency spectrum of a certain amount of audio stream data is extracted as a feature, and in the calculating step, the maximum volume and peak frequency are calculated based on the extracted frequency spectrum, and a first differential value obtained by differentiating the calculated maximum volume and a second differential value obtained by differentiating the calculated peak frequency are used as indices.

[0150] <Appendix 6> In the control step, if the determination result indicates that the response sound is a beep sound, the program causes the automated voice telephone to play a predetermined sound different from the standard response sound (Appendix 2).

[0151] <Appendix 7> If the control step determines that the multiple indicators do not satisfy the specified conditions, the program analyzes the acquired voice stream data to determine whether the response from the receiving party is a human speech or an automated voice response, and controls the response processing of an automated voice telephone in response to the response from the receiving party depending on the determination result.

[0152] <Appendix 8> A program described in any of (Appendix 1) to (Appendix 7), which further causes the processor to execute a step of acquiring information about the receiving party, and in the control step, causes the automatic voice telephone to play a predetermined voice corresponding to the receiving party when a call is made from the automatic voice telephone based on the judgment result and the acquired information about the receiving party.

[0153] <Appendix 9> An information processing device comprising a control unit and a storage unit, wherein the control unit executes all steps in the program according to any one of (Supplementary Note 1) to (Supplementary Note 8).

[0154] <Appendix 10> A method executed by a computer having a processor and a memory, wherein the processor executes all steps in the program described in any one of (Appendix 1) to (Appendix 8).

[0155] <Appendix 11> A system comprising means for executing all steps in the program described in any one of (Appendix 1) to (Appendix 8). [Explanation of symbols]

[0156] 1. System 10…Telephone 20...Server 22...Communication IF 23...Input / output interface 25…Memory 26…Storage 29...Processor 30...Automatic voice telephone

Claims

1. A program to be executed by a computer having a processor and a memory, The program causes the processor to: acquiring voice stream data indicating a response sound from a called party when a call is made from the automated voice telephone; extracting features from the acquired audio stream data every time the acquired audio stream data reaches a certain amount; calculating an index for determining whether the answering sound is a beep sound of an answering machine based on the extracted feature amount; determining whether the plurality of indicators calculated for each of the predetermined amount of the audio stream data satisfy a predetermined condition; a step of controlling a response process of the automatic voice telephone in response to a response from the receiving side according to the determination result; Execute In the extracting step, a frequency spectrum of the certain amount of the audio stream data is extracted as the feature amount; In the calculating step, the extracted frequency spectrum is normalized, and the index is a similarity between a model of the frequency spectrum generated based on a peak frequency corresponding to a maximum value of the normalized frequency spectrum and the normalized frequency spectrum.

2. 2. The program according to claim 1, wherein, in the determining step, it is determined that the response sound is the beep sound if a plurality of the indicators satisfy the predetermined condition.

3. 3. The program according to claim 2, wherein, in the controlling step, if the determination result indicates that the response sound is the beep sound, the automatic voice telephone is caused to reproduce a predetermined sound different from a standard response sound.

4. In the controlling step, if the determination result indicates that the plurality of indicators do not satisfy the predetermined condition, determining whether the response from the receiving party is a human speech or an automated voice response by analyzing the acquired voice stream data; 2. The program according to claim 1, wherein the program controls a response process of the automatic voice telephone in response to a response from the called party according to the result of the determination.

5. further causing the processor to perform the step of obtaining information about the called party; The program according to claim 1, wherein in the control step, the automatic voice telephone is caused to play a predetermined voice corresponding to the recipient when a call is made from the automatic voice telephone based on the judgment result and the acquired information about the recipient.

6. A program to be executed by a computer having a processor and a memory, The program causes the processor to: acquiring voice stream data indicating a response sound from a called party when a call is made from the automated voice telephone; extracting features from the acquired audio stream data every time the acquired audio stream data reaches a certain amount; calculating an index for determining whether the answering sound is a beep sound of an answering machine based on the extracted feature amount; determining whether the plurality of indicators calculated for each of the predetermined amount of the audio stream data satisfy a predetermined condition; a step of controlling a response process of the automatic voice telephone in response to a response from the receiving side according to the determination result; Execute In the extracting step, a frequency spectrum of the certain amount of the audio stream data is extracted as the feature amount; The calculating step normalizing the extracted frequency spectrum and generating a model of the frequency spectrum based on a peak frequency corresponding to a maximum value of the normalized frequency spectrum; calculating a similarity between the normalized frequency spectrum and the generated model, and setting the calculated similarity and the peak frequency as the index; Programs including.

7. A program to be executed by a computer having a processor and a memory, The program causes the processor to: acquiring voice stream data indicating a response sound from a called party when a call is made from the automated voice telephone; extracting features from the acquired audio stream data every time the acquired audio stream data reaches a certain amount; calculating an index for determining whether the answering sound is a beep sound of an answering machine based on the extracted feature amount; determining whether the plurality of indicators calculated for each of the predetermined amount of the audio stream data satisfy a predetermined condition; a step of controlling a response process of the automatic voice telephone in response to a response from the receiving side according to the determination result; Execute In the extracting step, a frequency spectrum of the certain amount of the audio stream data is extracted as the feature amount; In the calculating step, a program is provided which calculates a maximum volume and a peak frequency based on the extracted frequency spectrum, and uses a first derivative value obtained by differentiating the calculated maximum volume and a second derivative value obtained by differentiating the calculated peak frequency as the indexes.

8. 8. An information processing apparatus comprising: a control unit; and a storage unit, wherein the control unit executes all steps of the program according to claim 1.

9. A method implemented on a computer having a processor and a memory, wherein the processor executes all the steps of the program of any one of claims 1 to 7.

10. A system comprising means for executing all steps in the program according to any one of claims 1 to 7.

Images