method

Abstract

Improve the technology for analyzing dialogue content. [Solution] A method to be performed by an information processing device 1 comprising a control unit 10, a shooting unit 11, and an input unit 12, wherein the control unit 10 performs an operation including obtaining customer consent regarding the recording of customer service via the input unit 12, starting recording after consent has been obtained, detecting staff from the image of the shooting unit 11, and interrupting the recording when predetermined conditions are met, the predetermined conditions include a first condition that the staff has disappeared from the image of the shooting unit 11.

Description

Technical Field

[0001] This disclosure relates to a method.

Background Art

[0002] Conventionally, technologies for analyzing conversation content are known. For example, Patent Document 1 discloses a conversation analysis system that records conversation data based on voice data obtained by recording conversation content, extracts conversations that match conditions specified by a user from the conversation data, and displays them in a list.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventionally, store staff record voices during customer service such as business negotiations and utilize the recorded voices for purposes such as creating customer reports. The staff may temporarily leave their seats for errands such as preparing estimates and contract documents. It is desirable that the recording be interrupted while the staff is away. However, the staff may forget to interrupt the recording due to concentrating on customer service.

[0005] In view of such circumstances, an object of the present disclosure is to improve the technology for analyzing conversation content.

Means for Solving the Problems

[0006] A method according to an embodiment of the present disclosure is a method executed by an information processing apparatus including a control unit, a photographing unit, and an input unit, The control unit, obtains consent of a customer regarding recording of customer service via the input unit, After obtaining the aforementioned consent, recording will begin. To detect staff from the images taken by the aforementioned camera unit, If the specified conditions are met, the recording will be interrupted, Perform an action that includes, The aforementioned predetermined conditions include a first condition that the staff member has disappeared from the image of the camera unit. [Effects of the Invention]

[0007] According to one embodiment of this disclosure, the technology for analyzing dialogue content is improved. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the schematic configuration of an information processing device according to one embodiment of the present disclosure. [Figure 2] This is a flowchart showing the operation of an information processing device according to one embodiment of this disclosure. [Modes for carrying out the invention]

[0009] The embodiments of this disclosure will be described below with reference to the drawings.

[0010] (Summary of this embodiment) Referring to Figure 1, an overview of the information processing device 1 according to the embodiment of this disclosure will be described. In this embodiment, the information processing device 1 is a computer such as a laptop computer or a tablet. The information processing device 1 is used, for example, by store staff. The information processing device 1 is capable of recording the voices of customers and staff.

[0011] First, an overview of this embodiment will be described, and details will be described later. The method according to this embodiment is performed by an information processing device 1 comprising a control unit 10, a shooting unit 11, and an input unit 12. The control unit 10 obtains the customer's consent to record customer interactions via the input unit 12. After obtaining consent, the control unit 10 starts recording. The control unit 10 detects staff from the image taken by the shooting unit 11. The control unit 10 interrupts recording when predetermined conditions are met. The predetermined conditions include a first condition that the staff disappears from the image taken by the shooting unit 11.

[0012] According to this embodiment, recording is automatically interrupted when predetermined conditions are met. This ensures that recording is reliably interrupted without the need for staff to manually interrupt the recording.

[0013] (Configuration of Information Processing Device 1) As shown in Figure 1, the information processing device 1 comprises a control unit 10, an imaging unit 11, an input unit 12, a display unit 13, a communication unit 14, and a storage unit 15.

[0014] The control unit 10 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The processor is a general-purpose processor such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process, but is not limited to these. The programmable circuit is an FPGA (Field-Programmable Gate Array), but is not limited to this. The dedicated circuit is an ASIC (Application Specific Integrated Circuit), but is not limited to this. The control unit 10 executes various processes related to the operation of the information processing device 1 and controls each part of the information processing device 1.

[0015] The imaging unit 11 includes an optional imaging module capable of imaging the area around the information processing device 1. The imaging module includes one or more cameras. Each camera is positioned appropriately on the information processing device 1 so as to be able to image the area around the information processing device 1. In this embodiment, the imaging unit 11 includes an in-camera capable of imaging subjects on the user side of the information processing device 1 (e.g., staff). The imaging unit 11 may further include an out-camera capable of imaging subjects on the opposite side of the user (e.g., customers).

[0016] The input unit 12 includes one or more input interfaces. The input interfaces include microphones for receiving voice input from customers and staff. The input interfaces may include, for example, physical keys, capacitive keys, pointing devices, or touchscreens integrated with the display of the display unit 13. The input unit 12 accepts operations to input information used for the operation of the information processing device 1. Instead of being provided in the information processing device 1, the input unit 12 may be connected to the information processing device 1 as an external input device. Any connection method such as USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface), or Bluetooth (registered trademark) can be used.

[0017] The display unit 13 includes one or more display interfaces. The display interface is, for example, a display that displays information as an image. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display. The display unit 13 displays information obtained through the operation of the information processing device 1. Instead of being provided on the information processing device 1, the display unit 13 may be connected to the information processing device 1 as an external display device. Any connection method such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used.

[0018] The communication unit 14 includes at least one communication interface for connecting to a network. The communication interface may conform to, for example, a mobile communication standard such as 4G (4th generation) or 5G (5th generation), or a wired LAN (Local Area Network) communication standard or a wireless LAN communication standard, but is not limited thereto and may conform to any communication standard.

[0019] The storage unit 15 includes one or more memories. Each memory included in the storage unit 15 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 15 stores any information used for the operation of the information processing apparatus 1. The storage unit 15 may store, for example, a system program, an application program, and embedded software. In the present embodiment, the storage unit 15 may store any data related to customer support such as consultations. The information stored in the storage unit 15 may be updated based on, for example, information acquired from a network via the communication unit 14.

[0020] (Operation flow of the information processing apparatus 1) Referring to FIG. 2, the operation of the information processing apparatus 1 according to the present embodiment will be described. Hereinafter, communication between the respective units of the information processing apparatus is performed via the communication unit 14.

[0021] S101: The control unit 10 of the information processing apparatus 1 acquires the consent of the customer regarding the recording of customer support via the input unit 12.

[0022] In this embodiment, the control unit 10 obtains consent by detecting consent phrases that suggest customer consent to recording from the utterance input to the input unit 12 (e.g., microphone). The control unit 10 may also detect consent phrases by comparing phrases stored in the memory unit 15 with phrases included in the utterance. For the comparison, natural language processing such as morphological analysis, syntactic analysis, semantic analysis, contextual analysis, and anaphora analysis, and a learning model pre-trained by machine learning may be used. The learning model may be trained to take the utterance as input and output the result of comparing phrases stored in the memory unit 15 with phrases included in the utterance. The features of the learning model may be specific words or phrases, such as "recording" or "consent".

[0023] The consent phrase may include a phrase indicating consent to recording customer interactions, such as, "I consent to recording this customer interaction." The consent phrase may also include a staff member's question regarding consent and the customer's response to that question, such as, "Do you consent to recording this customer interaction?", "Yes." The consent phrase is not limited to the examples above and may include any phrase.

[0024] The control unit 10 may display the question on the display unit 13 to prompt the staff member to utter the question. This ensures that even if the staff member forgets to ask the question or forgets what the question was, they can still reliably ask the consent-related question and obtain consent.

[0025] The control unit 10 may obtain consent by acquiring the customer's signature entered into the input unit 12 (for example, a touchscreen). Alternatively, the control unit 10 may obtain consent by displaying a screen requesting consent on the display unit 13 of the information processing device 1 and accepting the customer's selection of consent via the input unit 12 (for example, selecting a button indicating consent).

[0026] S102: The control unit 10 starts recording after obtaining consent.

[0027] Specifically, the control unit 10 records the voices of staff and customers input into the microphone of the input unit 12.

[0028] S103: The control unit 10 detects the staff from the image taken by the imaging unit 11.

[0029] The image may be captured, for example, by the in-camera of the imaging unit 11. Alternatively, the control unit 10 may detect customers from the image taken by the imaging unit 11. The image may be captured, for example, by the out-camera of the imaging unit 11. The control unit 10 may detect staff or customers from the image using any object detection technique such as YOLO (You Only Look Once) and CNN (Convolutional Neural Network).

[0030] S104: The control unit 10 determines whether a predetermined condition is met. If the predetermined condition is met (S104-YES), the process proceeds to S105. If the predetermined condition is not met, the process terminates.

[0031] In this embodiment, the predetermined conditions include a first condition that a staff member has disappeared from the image captured by the camera unit 11 (e.g., the front camera). If the control unit 10 detects a customer from the image in S103, the first condition may alternatively be that the customer has disappeared from the image captured by the camera unit 11 (e.g., the rear camera). This allows recording to be interrupted even if a customer leaves their seat. The control unit 10 may use any object detection technique such as YOLO and CNN to determine if a staff member or customer has disappeared from the image.

[0032] The predetermined conditions may include, in addition to or instead of the first condition, a second condition in which a phrase indicating a customer has left their seat is detected from the utterance entered into the input unit 12. The process may proceed to S105 if the first and second conditions, or only the second condition, are met. If the predetermined conditions include only the second condition, S103 may not be executed.

[0033] The phrase for leaving one's seat may be stored in advance in the memory unit 15. The control unit 10 may detect the phrase for leaving one's seat by comparing the phrase stored in the memory unit 15 with the phrase included in the utterance. For the comparison, natural language processing such as morphological analysis, syntactic analysis, semantic analysis, contextual analysis, and anaphora analysis, and a learning model that has been trained in advance by machine learning may be used. The learning model may be trained, for example, to take the utterance as input and output the result of comparing the phrase stored in the memory unit 15 with the phrase included in the utterance. The features of the learning model may be specific words or phrases, for example, "I will be leaving my seat."

[0034] The "away phrase" may include phrases that are likely to be spoken by staff or customers when they leave their seats, such as "I'm sorry, but I'll be away from my seat," or "I'm going to look around the store." By setting such phrases as "away phrases," the interruption will be reliably executed even if the staff forgets to perform the interruption operation. The interruption phrase is not limited to the examples above and may include any phrase.

[0035] S105: The control unit 10 interrupts the recording. The process then terminates.

[0036] The control unit 10 may resume recording after interrupting it if it detects a staff member or customer from the image captured by the camera unit 11 (for example, the front camera or rear camera).

[0037] After interrupting recording, the control unit 10 may enable it to acquire audio from the input unit 12 in the background. While acquiring audio in the background, the control unit 10 does not perform recording. Furthermore, the control unit 10 may resume recording if it detects a feedback phrase from the utterances input to the input unit 12 that suggests a staff member or customer has returned.

[0038] The return phrase may include phrases that are likely to be spoken by staff or customers upon their return, such as "I'm back." Setting such a phrase as the return phrase ensures that the system is restarted even if staff forget to restart it. The return phrase is not limited to the example above and may include any phrase.

[0039] While this disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art may make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions, etc., included in each component or step can be rearranged in a logically consistent manner, and multiple components or steps can be combined into one or divided into two.

[0040] For example, in the embodiment described above, it is also possible to distribute the configuration and operation of the information processing device 1 across multiple computers that can communicate with each other. For example, the configuration and operation of the information processing device 1 may be distributed across a server device and one or more terminal devices. [Explanation of Symbols]

[0041] 1 Information processing device, 10 Control unit, 11 Imaging unit, 12 Input unit, 13 Display unit, 14 Communication unit, 15 Storage unit

Claims

1. A method performed by an information processing device comprising a control unit, an imaging unit, and an input unit, The control unit, Obtaining customer consent for recording customer interactions via the aforementioned input unit, After obtaining the aforementioned consent, recording will begin. To detect staff from the images taken by the aforementioned camera unit, If the specified conditions are met, the recording will be interrupted, Perform an action that includes, A method wherein the predetermined conditions include a first condition that the staff member has disappeared from the image of the camera unit.

2. A method according to claim 1, wherein the operation further includes resuming recording if the staff member is detected from the image of the camera unit after the recording has been interrupted.

3. A method according to claim 1, wherein the predetermined condition further includes a second condition that a phrase indicating the customer has left their seat is detected from the utterance input to the input unit.

4. The method according to claim 3, wherein the operation is After interrupting recording, the control unit makes it possible to acquire audio from the input unit in the background. If a return phrase indicating that the customer has returned is detected from the utterance input to the input unit, the recording will be resumed. Methods that further include this.

5. A method according to claim 1, wherein obtaining the consent includes detecting a consent phrase that suggests the consent from an utterance entered into the input unit.

Images