system

Abstract

Improve the system for analyzing recorded dialogue audio. [Solution] System 1 comprises a mobile terminal 10 and an information processing device 20 that can communicate with each other. The mobile terminal 10 outputs an audio message requesting consent regarding the acquisition of personal information. When it detects an utterance indicating the customer's consent, it starts recording and transmits the first audio data generated by the recording to the information processing device 20. The information processing device 20 stores the first audio data received from the mobile terminal 10.

Description

Technical Field

[0001] This disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a dialogue analysis system that records dialogue data based on voice data obtained by recording dialogue content, extracts dialogues that match conditions specified by a user from the dialogue data, and displays them in a list.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventionally, in an analysis system that analyzes voices during business negotiations in a vehicle dealership using machine learning, recording is performed on the salesperson's terminal device (for example, a laptop PC (Personal Computer)), so the analysis target is limited to voices during business negotiations. However, since important information may also be included in voices in scenes other than business negotiations (for example, conversations during test drives and conversations around display vehicles), it is desirable to also analyze voices in scenes other than during business negotiations.

[0005] An object of this disclosure is to improve an analysis system for voices that record dialogues.

Means for Solving the Problems

[0006] A system according to an embodiment of this disclosure is a system including a portable terminal and an information processing device that can communicate with each other, wherein the portable terminal outputs the voice of a message asking for consent regarding the acquisition of personal information, When a customer's utterance indicating consent is detected, recording will begin. The first audio data generated by the recording is transmitted to the information processing device. The information processing device stores the first audio data received from the mobile terminal. [Effects of the Invention]

[0007] According to one embodiment, a system for analyzing recorded dialogue audio can be improved. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the system configuration according to one embodiment. [Figure 2] Figure 1 is a block diagram showing an example of the configuration of a mobile device. [Figure 3] Figure 1 is a block diagram showing an example of the configuration of an information processing device. [Figure 4] Figure 1 is a flowchart illustrating an example of how a mobile device operates. [Modes for carrying out the invention]

[0009] Hereinafter, one embodiment of this disclosure will be described with reference to the drawings.

[0010] (Summary of the embodiment) Figure 1 shows an example configuration of System 1 according to one embodiment. System 1 analyzes recordings of conversations in business negotiations regarding the sale of various goods and services, including automobiles and related products, as well as related services. Hereinafter, the person receiving the goods and services will be referred to as the "customer," and any person on the sales side providing the goods and services will be referred to as the "salesperson." The following explanation will focus on an example of System 1 being used in automobile sales, but the use of System 1 is arbitrary.

[0011] System 1 comprises a mobile terminal 10 and an information processing device 20. The mobile terminal 10 and the information processing device 20 are connected to each other so as to be able to communicate with each other via a network N. Network N is any communication path including, for example, an intranet, a mobile communication network, the Internet, or any combination thereof. Alternatively, the mobile terminal 10 and the information processing device 20 may communicate directly via wireless LAN (Local Area Network) or Bluetooth®.

[0012] The mobile terminal 10 is a portable computer capable of inputting and outputting voice, held by the salesperson. The mobile terminal 10 can be, for example, an electronic catalog tablet, a smartphone, or a small microphone, but the type of mobile terminal 10 is arbitrary. Figure 1 shows an example where System 1 has one mobile terminal 10, but the number of mobile terminals 10 is arbitrary.

[0013] The information processing device 20 is a computer that stores the audio recorded by the mobile terminal 10. The information processing device 20 is, for example, a PC operated by a salesperson, but the type of information processing device 20 is arbitrary. For example, the information processing device 20 may be a computer installed in a data center, etc. The information processing device 20 may be, for example, a server belonging to a cloud computing system or other computing system. Figure 1 shows an example where system 1 has one information processing device 20, but the number of information processing devices 20 is arbitrary.

[0014] First, an overview of this embodiment will be described, and details will be described later. System 1 comprises a mobile terminal 10 and an information processing device 20 that can communicate with each other. The mobile terminal 10 first outputs an audio message requesting consent regarding the acquisition of personal information. When the mobile terminal 10 detects an utterance indicating the customer's consent, it starts recording. The mobile terminal 10 transmits the first audio data generated by the recording to the information processing device 20. The information processing device 20 stores the first audio data received from the mobile terminal 10.

[0015] In this way, in System 1, for example, voice data recorded by a mobile terminal 10 such as a tablet terminal for an electronic catalog and a small microphone is uploaded to the information processing device 20. Therefore, according to System 1, even without carrying the information processing device 20 itself, voices in scenes other than during business negotiations can also be targeted for analysis. Also, since the detection of a customer's speech indicating consent regarding the acquisition of personal information is a condition for starting recording, it is possible to obtain consent through a simpler procedure compared to, for example, a case where a customer signs a written consent. Therefore, according to the present disclosure, in terms of also targeting voices in scenes other than during business negotiations for analysis, an audio analysis system that records conversations can be improved.

[0016] Next, each component of System 1 will be described in detail.

[0017] (Configuration of Mobile Terminal 10) FIG. 2 is a block diagram showing a configuration example of the mobile terminal 10 in FIG. 1. The mobile terminal 10 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, and an output unit 15.

[0018] The control unit 11 includes at least one processor, at least one dedicated circuit, or a combination thereof. The processor is a general-purpose processor such as a CPU (central processing unit) or GPU (graphics processing unit), or a dedicated processor specialized for specific processing. The dedicated circuit is, for example, an FPGA (field-programmable gate array) or an ASIC (application specific integrated circuit). The control unit 11 executes processes related to the operation of the mobile terminal 10 while controlling each part of the mobile terminal 10.

[0019] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The semiconductor memory is, for example, a RAM (random access memory) or a ROM (read only memory). The RAM is, for example, a SRAM (static random access memory) or a DRAM (dynamic random access memory). The ROM is, for example, an EEPROM (electrically erasable programmable read only memory). The storage unit 12 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores data used for the operation of the mobile terminal 10 and data obtained by the operation of the mobile terminal 10.

[0020] The communication unit 13 includes at least one interface for external communication. The communication interface may be either a wired communication or a wireless communication interface. In the case of wired communication, the communication interface is, for example, a LAN interface, a USB (Universal Serial Bus). In the case of wireless communication, the communication interface is, for example, an interface corresponding to a mobile communication standard such as LTE (Long Term Evolution), 4G (4th generation), or 5G (5th generation), or an interface corresponding to a short-range wireless communication such as Bluetooth (registered trademark). The communication unit 13 receives data used for the operation of the mobile terminal 10 and transmits data obtained by the operation of the mobile terminal 10.

[0021] The input unit 14 includes at least one input interface. The input interface is equipped with a sound sensor (microphone) that accepts voice input. The input unit 14 may also be, for example, a physical key, a capacitive key, a pointing device, or a touchscreen integrated with a display. Alternatively, the input interface may be, for example, a camera that accepts gesture input. The input unit 14 accepts operations to input data used for the operation of the mobile terminal 10. Instead of being integrated into the mobile terminal 10, the input unit 14 may be connected to the mobile terminal 10 as an external input device. Any connection method can be used, for example, USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface), or Bluetooth (registered trademark).

[0022] The output unit 15 includes at least one output interface. The output interface is equipped with a speaker that outputs information as sound. Alternatively, the output interface may be, for example, a display that outputs information as video, or a speaker that outputs information as sound. The display may be, for example, an LCD (liquid crystal display) or an organic EL (electroluminescence) display. The output unit 15 may also display and output data obtained from the operation of the mobile terminal 10. Instead of being equipped in the mobile terminal 10, the output unit 15 may be connected to the mobile terminal 10 as an external output device. Any connection method can be used, for example, USB, HDMI (registered trademark), or Bluetooth (registered trademark).

[0023] The functions of the mobile terminal 10 are realized by executing the program according to this embodiment on a processor corresponding to the control unit 11. In other words, the functions of the mobile terminal 10 are realized by software. The program causes the computer to perform the operations of the mobile terminal 10, thereby causing the computer to function as the mobile terminal 10. That is, the computer functions as the mobile terminal 10 by performing the operations of the mobile terminal 10 according to the program.

[0024] In this embodiment, the program can be recorded on a computer-readable recording medium. The computer-readable recording medium includes non-temporary computer-readable media, such as magnetic recording devices, optical discs, magneto-optical recording media, or semiconductor memory. The program can be distributed, for example, by selling, transferring, or lending portable recording media such as DVDs (digital versatile discs) or CD-ROMs (compact disc read-only memory) on which the program is recorded. Alternatively, the program may be distributed by storing it on the storage of an external server and transmitting it from the external server to other computers. The program may also be provided as a program product.

[0025] Some or all of the functions of the mobile terminal 10 may be implemented by a dedicated circuit corresponding to the control unit 11. In other words, some or all of the functions of the mobile terminal 10 may be implemented by hardware.

[0026] (Configuration of the information processing device 20) Figure 3 is a block diagram showing an example configuration of the information processing device 20 shown in Figure 1. The information processing device 20 comprises a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, and an output unit 25. The functions and configurations of the control unit 21, storage unit 22, communication unit 23, input unit 24, and output unit 25 of the information processing device 20 are the same as those of the control unit 11, storage unit 12, communication unit 13, input unit 14, and output unit 15 of the mobile terminal 10, so a detailed explanation is omitted.

[0027] (Example of operation) Figure 4 is a flowchart illustrating an example of the operation of the mobile terminal 10 in Figure 1. The operation of the mobile terminal 10 described with reference to Figure 4 may correspond to one of the control methods for the mobile terminal 10 or system 1. The operation of each step in Figure 4 may be performed based on control by the control unit 11 of the mobile terminal 10. The following processes may be performed, for example, in situations other than business negotiations (e.g., conversations during test drives and around display vehicles).

[0028] In step S1, the control unit 11 of the mobile terminal 10 outputs a voice message requesting consent regarding the acquisition of personal information.

[0029] Specifically, the control unit 11 may output an audio message from the speaker of the output unit 15, such as, "Do you consent to the collection of your personal information? If you consent, please answer 'yes'. If you do not consent, please answer 'no'."

[0030] In step S2, the control unit 11 determines whether or not the customer has given consent. If consent has been given (YES in step S2), the control unit 11 proceeds to step S3; otherwise (NO in step S2), it terminates the flowchart process.

[0031] Specifically, the control unit 11 may determine that an utterance of consent was made (YES in step S2) if, within a certain period of time after the message is output aloud in step S1, an utterance indicating consent, such as "yes" or "I agree," is detected by the sound sensor of the input unit 14. The control unit 11 may determine that an utterance of consent was not made (NO in step S2) if, within a certain period of time after the message is output aloud in step S1, an utterance indicating consent is not detected. Alternatively, the control unit 11 may determine that an utterance of consent was not made (NO in step S2) if, within a certain period of time after the message is output aloud in step S1, an utterance indicating consent is not detected.

[0032] Furthermore, the mobile terminal 10 may pre-register characteristic information of the salesperson's voice, and in step S2, it may selectively detect voices other than the salesperson's from the input voice based on that characteristic information. The mobile terminal 10 may determine that the customer has given consent (YES in step S2) if it detects an utterance indicating consent from a voice other than the salesperson's that has been detected. In this way, by detecting the presence or absence of an utterance indicating consent using voices that exclude the pre-registered voice of the salesperson, the system 1 can prevent a salesperson from impersonating a customer and giving consent, thereby preventing recordings from being made against the customer's will.

[0033] In step S3, the control unit 11 starts recording.

[0034] Specifically, the control unit 11 sequentially stores the sound detected by the sound sensor of the input unit 14 as first sound data in the storage unit 12.

[0035] In step S4, the control unit 11 transmits the first audio data generated by the recording to the information processing device 20.

[0036] Specifically, the control unit 11 transmits the first audio data to the information processing device 20 via the network N using the communication unit 13. Alternatively, the control unit 11 may generate one or more files from the series of audio data from the start to the end of the recording, and transmit one or more of these files to the information processing device 20 as the first audio data. Alternatively, the control unit 11 may transmit the first audio data to the information processing device 20 in streaming format in parallel with the recording. The control unit 11 may also transmit audio data indicating the customer's consent to the acquisition of personal information to the information processing device 20. After completing the process in step S4, the control unit 11 terminates the process described in the flowchart.

[0037] When the information processing device 20 receives the first voice data from the mobile terminal 10, it stores the first voice data in the storage unit 12 along with related information such as the date and time, customer identification information, and location. The information processing device 20 may also store voice data indicating the customer's consent to the acquisition of personal information in the storage unit 12, in association with the first voice data.

[0038] In this manner, the mobile terminal 10 (a tablet device for electronic catalogs, a small microphone, etc.) outputs a voice message to the customer requesting their consent regarding the acquisition of personal information. When the mobile terminal 10 receives the customer's voice indicating consent, it uploads the recorded voice data to the information processing device 20 (a salesperson's PC, etc.). Therefore, according to this embodiment, since recording is done using a portable mobile terminal 10, voices from situations other than during business negotiations can also be included in the analysis. Furthermore, since the detection of a customer's utterance indicating consent regarding the acquisition of personal information is the condition for starting recording, it is possible to obtain consent through a simpler procedure compared to, for example, an example where the customer signs a written consent form. Therefore, according to this disclosure, it is possible to improve the voice analysis system of recorded conversations by including voices from situations other than during business negotiations in the analysis.

[0039] In the example shown in Figure 4, recording begins only after the customer's consent is confirmed (YES in step S2, step S3). However, the mobile terminal 10 may start recording after the audio message requesting consent is output, but before any utterance indicating consent or disagreement by the customer is detected. In this case, if an utterance indicating consent by the customer is detected, the mobile terminal 10 may send the audio data related to the recording that started before the detection of the consent utterance to the information processing device as first audio data. In this way, the system 1 can record conversations prior to the utterance of consent by starting recording after the audio message requesting consent is output, regardless of whether the customer consents or not, and then sending the audio data to the information processing device when an utterance of consent is detected.

[0040] On the other hand, if the mobile terminal 10 starts recording after outputting an audio message requesting consent, regardless of whether the customer consents or not, and if an utterance indicating disapproval is detected, the mobile terminal 10 may discard the audio data related to the recording that was started before the utterance indicating disapproval was detected. The mobile terminal 10 may also discard the audio data related to the recording that was started earlier if no utterance of consent is detected within a certain period of time after outputting an audio message requesting consent. In this way, the system 1 can respect the customer's wishes regarding the recording by discarding the audio data of the recording that was started earlier if the customer indicates disapproval, or if the customer does not indicate consent within a certain period of time after outputting an audio message requesting consent.

[0041] Furthermore, not only the mobile terminal 10 but also the information processing device 20 may record conversations with customers. Specifically, the information processing device 20 may acquire second audio data generated by recording conversations with customers. In this case, the information processing device 20 may store the first audio data received from the mobile terminal and the acquired second audio data in association with the same customer. The information processing device 20 may also store audio data indicating the customer's consent to the acquisition of personal information, in association with the same customer. In this way, since system 1 stores the first and second audio data recorded for the same customer in association with the customer, it can effectively utilize the audio data for that customer.

[0042] This disclosure is not limited to the embodiments described above. For example, multiple blocks shown in the block diagram may be combined, or a single block may be divided. Multiple steps shown in the flowchart may be performed in parallel or in a different order, depending on the processing capacity of the device performing each step, or as necessary, instead of being performed in chronological order as described. Other modifications are possible without departing from the spirit of this disclosure.

[0043] Furthermore, for example, the configuration and operation of the mobile terminal 10 and the information processing device 20 may be distributed across multiple computers capable of communicating with each other. Also, for example, some or all of the components of the information processing device 20 may be provided on other devices such as a cloud server or network storage connected to the network N.

[0044] Some embodiments of the present disclosure are described below. However, it should be noted that the embodiments of the present disclosure are not limited to these. [Note 1] A system comprising mobile terminals and information processing devices capable of communicating with each other, The aforementioned mobile terminal is It outputs an audio message requesting consent regarding the collection of personal information. When a customer's utterance indicating consent is detected, recording will begin. The first audio data generated by the recording is transmitted to the information processing device. The information processing device stores the first audio data received from the mobile terminal. system. [Note 2] The aforementioned mobile terminal is Based on the voice characteristics information of pre-registered salespeople, the system selectively detects voices other than those of the aforementioned salespeople from the input audio. If a voice other than the salesperson detected indicates consent, the system will recognize that a voice indicating consent from the customer has been detected and begin recording. The system described in Appendix 1. [Note 3] The aforementioned mobile terminal is Recording begins after the audio message requesting consent is output, but before any utterance indicating consent or disagreement by the customer is detected. When an utterance indicating the customer's consent is detected, the audio data relating to the recording that started before the detection of the utterance indicating consent is transmitted to the information processing device as the first audio data. The system described in Appendix 1 or 2. [Note 4] The aforementioned mobile terminal is If a utterance indicating the customer's non-consent is detected, and if no utterance indicating the customer's consent is detected within a certain period of time after the audio message requesting consent is output, the audio data related to the initiated recording will be discarded. The system described in Appendix 3. [Note 5] The aforementioned information processing device is A second audio data is obtained, generated from a recording of the business negotiation with the aforementioned customer. For the same customer, the first voice data received from the mobile terminal and the second voice data acquired are stored in association with that customer. A system described in any one of the items 1 to 4 of the appendix. [Explanation of Symbols]

[0045] 1: System 10: Mobile devices 11: Control Unit 12: Storage part 13: Communications Department 14: Input section 15: Output section 20: Information Processing Devices 21: Control Unit 22: Storage part 23: Communications Department 24: Input section 25: Output section N: Network

Claims

1. A system comprising mobile terminals and information processing devices capable of communicating with each other, The aforementioned mobile terminal is It outputs an audio message requesting consent regarding the collection of personal information. When a customer's utterance indicating consent is detected, recording will begin. The first audio data generated by the recording is transmitted to the information processing device. The information processing device stores the first audio data received from the mobile terminal. system.

2. The aforementioned mobile terminal is Based on the voice characteristics information of pre-registered salespeople, the system selectively detects voices other than those of the aforementioned salespeople from the input audio. If a voice other than the salesperson detected indicates consent, the system will recognize that a voice indicating consent from the customer has been detected and begin recording. The system according to claim 1.

3. The aforementioned mobile terminal is Recording begins after the audio message requesting consent is output, but before any utterance indicating consent or disagreement by the customer is detected. When an utterance indicating the customer's consent is detected, the audio data relating to the recording that started before the detection of the utterance indicating consent is transmitted to the information processing device as the first audio data. The system according to claim 1.

4. The aforementioned mobile terminal is If a utterance indicating the customer's non-consent is detected, and if no utterance indicating the customer's consent is detected within a certain period of time after the audio message requesting consent is output, the audio data related to the initiated recording will be discarded. The system according to claim 3.

5. The aforementioned information processing device is A second audio data is obtained, generated from a recording of the business negotiation with the aforementioned customer. For the same customer, the first voice data received from the mobile terminal and the second voice data acquired are stored in association with that customer. The system according to any one of claims 1 to 4.

Images