method

Abstract

Improve the techniques for analyzing dialogue content. [Solution] A terminal device 20 comprising a control unit 200, a recording unit 201, and an input unit 202 performs an operation in which the control unit 200 obtains the customer's consent to record customer interactions via the input unit 202, starts recording after consent has been obtained, and terminates recording when predetermined conditions are met.

Description

Technical Field

[0001] This disclosure relates to a method.

Background Art

[0002] Conventionally, technologies for analyzing conversation content are known. For example, Patent Document 1 discloses a conversation analysis system that records conversation data based on voice data obtained by recording conversation content, extracts conversations that match conditions specified by a user from the conversation data, and displays them in a list.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventionally, store staff have recorded voices during customer service and utilized the recorded voices for purposes such as creating customer reports. However, due to concentrating on customer service, the staff may forget to perform the recording end operation.

[0005] In view of such circumstances, an object of the present disclosure is to improve the technology for analyzing conversation content.

Means for Solving the Problems

[0006] A method according to an embodiment of the present disclosure is a method executed by a terminal device including a control unit, a photographing unit, and an input unit, where the control unit acquires consent from a customer regarding recording of customer service via the input unit, starts recording after the consent is acquired, detects a staff member from an image of the photographing unit, The recording will terminate when the specified conditions are met. Performs an action that includes this. [Effects of the Invention]

[0007] According to one embodiment of this disclosure, the technology for analyzing dialogue content is improved. [Brief explanation of the drawing]

[0008] [Figure 1] This block diagram shows a schematic configuration of a system according to one embodiment of the present disclosure. [Figure 2] This is a flowchart showing the operation of a terminal device according to one embodiment of this disclosure. [Modes for carrying out the invention]

[0009] The embodiments of this disclosure will be described below with reference to the drawings.

[0010] (Summary of this embodiment) Referring to Figure 1, an overview of System 1 according to an embodiment of this disclosure will be described. In this embodiment, System 1 comprises an information processing device 10 and a terminal device 20. The information processing device 10 and the terminal device 20 are connected to each other via a network 30 such as the Internet or mobile communication.

[0011] In this embodiment, the information processing device 10 includes one server device or multiple computers that can communicate with each other.

[0012] In this embodiment, the terminal device 20 is a computer such as a laptop computer, tablet, or smartphone. The terminal device 20 is used, for example, by store staff. The terminal device 20 is capable of recording the voices of customers and staff.

[0013] First, an overview of this embodiment will be described, and details will be described later. The method according to this embodiment is performed by a terminal device 20 comprising a control unit 200, a shooting unit 201, and an input unit 202. The control unit 200 obtains the customer's consent to record customer interactions via the input unit 202. After obtaining consent, the control unit 200 starts recording. The control unit 200 detects staff from the image taken by the shooting unit 201. The control unit 200 terminates recording when predetermined conditions are met.

[0014] According to this embodiment, recording automatically terminates when predetermined conditions are met. This ensures that recording ends reliably without requiring staff to manually terminate the recording.

[0015] (Configuration of the information processing device 10) As shown in Figure 1, the information processing device 10 comprises a control unit 100, a communication unit 101, and a storage unit 102.

[0016] The control unit 100 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The processor is a general-purpose processor such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process, but is not limited to these. The programmable circuit is an FPGA (Field-Programmable Gate Array), but is not limited to this. The dedicated circuit is an ASIC (Application Specific Integrated Circuit), but is not limited to this. The control unit 100 performs various processes related to the operation of the information processing device 10 and controls each part of the information processing device 10.

[0017] The communication unit 101 includes at least one communication interface for connecting to the network 30. The communication interface corresponds to, for example, a mobile communication standard such as 4G (4th generation) or 5G (5th generation), or a wired LAN (Local Area Network) communication standard or a wireless LAN communication standard, but is not limited thereto and may correspond to any communication standard.

[0018] The storage unit 102 includes one or more memories. Each memory included in the storage unit 102 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 102 stores any information used for the operation of the information processing device 10. The storage unit 102 may store, for example, system programs, application programs, and embedded software. The storage unit 102 may store any data related to customer correspondence such as consultations. The information stored in the storage unit 102 can be updated, for example, based on information acquired from the network 30 via the communication unit 101.

[0019] (Configuration of the terminal device 20) As shown in FIG. 1, the terminal device 20 includes a control unit 200, a photographing unit 201, an input unit 202, a display unit 203, a communication unit 204, and a storage unit 205.

[0020] The control unit 200 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The processor is, for example, a general-purpose processor such as a CPU or a GPU, or a dedicated processor specialized for a specific process, but is not limited thereto. The programmable circuit is, for example, an FPGA, but is not limited thereto. The dedicated circuit is, for example, an ASIC, but is not limited thereto. The control unit 200 executes various processes related to the operation of the terminal device 20 and controls each part of the terminal device 20.

[0021] The imaging unit 201 includes an optional imaging module capable of imaging the area around the terminal device 20. The imaging module includes one or more cameras. Each camera is positioned appropriately on the terminal device 20 so as to be able to image the area around the terminal device 20. In this embodiment, the imaging unit 201 includes an in-camera capable of imaging subjects on the user side of the terminal device 20 (e.g., staff). The imaging unit 201 may further include an out-camera capable of imaging subjects on the opposite side of the user (e.g., customers).

[0022] The input unit 202 includes one or more input interfaces. The input interfaces include microphones for receiving voice input from customers and staff. The input interfaces may include, for example, physical keys, capacitive keys, pointing devices, or touchscreens integrated with the display of the display unit 203. The input unit 202 accepts operations for inputting information used to operate the terminal device 20. Instead of being installed in the terminal device 20, the input unit 202 may be connected to the terminal device 20 as an external input device. Any connection method can be used, such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), or Bluetooth (Bluetooth).

[0023] The display unit 203 includes one or more display interfaces. A display interface is, for example, a display that displays information as an image. The display is, for example, an LCD or an organic EL display. The display unit 203 displays information obtained through the operation of the terminal device 20. Instead of being installed in the terminal device 20, the display unit 203 may be connected to the terminal device 20 as an external display device. Any connection method can be used, such as USB, HDMI®, or Bluetooth®.

[0024] The communication unit 204 includes at least one communication interface connected to the network 30. The communication interface supports, for example, a mobile communication standard such as 4G or 5G, or a wired LAN communication standard or a wireless LAN communication standard, but is not limited to these and may support any communication standard.

[0025] The storage unit 205 includes one or more memories. Each memory included in the storage unit 205 may function as, for example, a main memory, an auxiliary memory, or a cache memory. The storage unit 205 stores any information used for the operation of the terminal device 20. The storage unit 205 may store, for example, system programs, application programs, and embedded software. The storage unit 205 may also store any data related to customer interactions, such as business negotiations. The information stored in the storage unit 205 may be updated, for example, based on information obtained from the network 30 via the communication unit 204.

[0026] (Operation flow of terminal device 20) Referring to Figure 2, the operation of the terminal device 20 according to this embodiment will be described. In the following, communication between the information processing device 10 and the terminal device 20 is performed via the communication units 101 and 204 and the network 30.

[0027] S101: The control unit 200 of the terminal device 20 obtains the customer's consent to record customer interactions via the input unit 202.

[0028] In this embodiment, the control unit 200 obtains consent by detecting consent phrases that suggest customer consent to recording from the utterance input to the input unit 202 (e.g., microphone). The control unit 200 may also detect consent phrases by comparing phrases stored in the memory unit 102 or 205 with the utterance content. Natural language processing such as morphological analysis, syntactic analysis, semantic analysis, contextual analysis, and anaphora analysis, as well as a learning model pre-trained by machine learning, may be used for the comparison. The learning model may be trained, for example, to take the utterance content as input and output the result of comparing the phrase stored in the memory unit 205 with the utterance content. The features of the learning model may be specific words or phrases, such as "recording" or "consent".

[0029] The consent phrase may include a phrase indicating consent to recording customer interactions, such as, "I consent to recording this customer interaction." The consent phrase may also include a staff member's question regarding consent and the customer's response to that question, such as, "Do you consent to recording this customer interaction?", "Yes." The consent phrase is not limited to the examples above and may include any phrase.

[0030] The control unit 200 may display the question on the display unit 203 to prompt the staff member to utter the question. This ensures that even if the staff member forgets to ask the question or forgets what the question was, they can still reliably ask the consent-related question and obtain consent.

[0031] The control unit 200 may obtain consent by acquiring the customer's signature entered into the input unit 202 (e.g., a touchscreen). Alternatively, the control unit 200 may obtain consent by displaying a screen requesting consent on the display unit 203 of the terminal device 20 and accepting the customer's selection of consent via the input unit 202 (e.g., selection of a button indicating consent).

[0032] S102: The control unit 200 starts recording after obtaining consent.

[0033] Specifically, the control unit 200 records the voices of staff and customers input into the microphone of the input unit 202.

[0034] S103: The control unit 200 detects staff from the image captured by the imaging unit 201.

[0035] The image may be captured, for example, by the in-camera of the imaging unit 201. Alternatively, the control unit 200 may detect customers from the image of the imaging unit 201. The image may be captured, for example, by the out-camera of the imaging unit 201. The control unit 200 may detect staff or customers from the image using any object detection technique such as YOLO (You Only Look Once) and CNN (Convolutional Neural Network).

[0036] S104: The control unit 200 determines whether a predetermined condition is met. If the predetermined condition is met (S104-YES), the process proceeds to S105. If the predetermined condition is not met (S104-NO), the process terminates.

[0037] In this embodiment, the predetermined conditions include a first condition in which an ending phrase indicating the end of customer service is detected from the utterance input to the input unit 202, and the staff member has disappeared from the image captured by the camera unit 201 (e.g., the front camera). If the control unit 200 detects a customer from the image in S103, the first condition may alternatively be a condition in which an ending phrase is detected from the utterance input to the input unit 202, and the customer has disappeared from the image captured by the camera unit 201 (e.g., the rear camera). The control unit 200 may use any object detection technique such as YOLO and CNN to detect the disappearance of the staff member or customer from the image.

[0038] The closing phrase may be pre-stored in the memory unit 102 or 205. The control unit 200 may detect the closing phrase by comparing the phrase stored in the memory unit 102 or 205 with the utterance. For the comparison, natural language processing such as morphological analysis, syntactic analysis, semantic analysis, contextual analysis, and anaphora analysis, and a learning model pre-trained by machine learning may be used. The learning model may be trained, for example, to take the utterance as input and output the result of comparing the phrase stored in the memory unit 102 or 205 with the utterance. The features of the learning model may be specific words or phrases, for example, expressions of gratitude such as "Thank you for today," or farewell greetings such as "I look forward to seeing you again."

[0039] The closing phrase may include phrases that staff are likely to say at the end of a customer interaction, such as expressions of gratitude like "Thank you for today," or farewell greetings like "We look forward to seeing you again." By setting such a phrase as the closing phrase, recording will be reliably terminated even if the staff forgets to end the recording. The closing phrase is not limited to the examples above and may include any phrase.

[0040] The control unit 200 may display the closing phrase on the display unit 203 to prompt the staff member to utter the closing phrase. This ensures that even if the staff member forgets to say the closing phrase or forgets its content, they can still reliably say the closing phrase and perform the closing operation.

[0041] The predetermined conditions may include, in addition to or instead of the first condition, a second condition that a certain period of time has elapsed during which no speech from staff or customers is detected via the input unit 202 and no staff is detected in the image captured by the imaging unit 201. The certain period of time is, for example, 10 seconds. The process may proceed to S105 if the first and second conditions, or only the second condition, are met.

[0042] S105: The control unit 200 terminates recording.

[0043] S106: The control unit 200 transmits the generated recording data to the information processing device 10. The process then terminates.

[0044] The information processing device 10 stores the recorded data in the storage unit 102.

[0045] While this disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art may make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions, etc., included in each component or step can be rearranged in a logically consistent manner, and multiple components or steps can be combined into one or divided into two.

[0046] For example, in the above-described embodiment, it is also possible to have an embodiment in which the configuration and operation of the information processing device 10 or terminal device 20 are distributed among multiple computers that can communicate with each other.

[0047] In the above-described embodiment, the control unit 200 may interrupt the recording before ending it. In the above-described embodiment, the first condition was used as a condition for ending the recording, but the first condition may also be used as a condition for interrupting the recording. Specifically, the control unit 200 may interrupt the recording when the first condition is met. In this case, the predetermined condition may include the second condition described above. [Explanation of Symbols]

[0048] 1 System, 10 Information Processing Device, 100 Control Unit, 101 Communication Unit, 102 Storage Unit, 20 Terminal Device, 200 Control Unit, 201 Imaging Unit, 202 Input Unit, 203 Display Unit, 204 Communication Unit, 205 Storage Unit

Claims

1. A method performed by a terminal device comprising a control unit, an imaging unit, and an input unit, The control unit, Obtaining customer consent for recording customer interactions via the aforementioned input unit, After obtaining the aforementioned consent, recording will begin. To detect staff from the images taken by the aforementioned camera unit, The recording will terminate when the specified conditions are met. A method for performing an action that includes the following.

2. A method according to claim 1, wherein the predetermined conditions include a first condition in which an ending phrase indicating the end of the customer interaction is detected from the utterance input to the input unit, and the staff member has disappeared from the image of the camera unit.

3. A method according to claim 2, wherein the predetermined condition further includes a second condition that a certain period of time has elapsed in which the speech of the staff and the customer is not detected via the input unit and the staff is not detected in the image of the imaging unit.

4. The method according to claim 1, further comprising interrupting recording when a termination phrase indicating the end of the customer interaction is detected from the utterance input to the input unit, and a first condition is met in which the staff member has disappeared from the image of the camera unit, The method wherein the predetermined conditions include a second condition that a certain period of time has elapsed in which the speech of the staff and the customer is not detected via the input unit and the staff is not detected in the image of the imaging unit.

5. A method according to any one of claims 1 to 4, wherein the operation further comprises transmitting the generated recording data to an information processing device.

Images