Voice data analysis apparatus, voice data analysis method, and computer program

Abstract

To provide a voice data analysis apparatus configured to accurately analyze voice of a store clerk while reducing the influence of noise.SOLUTION: A voice data analysis apparatus 1 includes: a conversation sound data acquisition unit (device attachment / detachment unit 13, control unit 23 and communication unit 27) for acquiring conversation sound data generated by recording voice of conversation; a conversation text generation unit (control unit 23) which generates conversation text data which is text data of the content of conversation included in the conversation sound data; and a conversation content analysis unit (control unit 23) which analyzes, using the conversation text data, the content of conversation to obtain an analytical result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a voice data analysis device and a voice data analysis method. [Background technology]

[0002] A sales management device has been proposed that records the voice (sales talk) of a store clerk (or salesperson) when serving customers and manages product sales results (Patent Document 1). This sales management device records the voices of store clerks using microphones installed in the store and determines whether the sales talk contains specific keywords, thereby managing the correlation with sales results. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-200851 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned sales management device has a problem in that since the voice of the store clerk is recorded using a sound-collecting microphone installed in the store, noise such as background music is also recorded, which can easily reduce the accuracy of voice analysis.

[0005] Therefore, it is desirable for one aspect of the present disclosure to provide a voice data analysis device and a voice data analysis method that can accurately analyze the voice of a store clerk while reducing the effects of noise. [Means for solving the problem]

[0006] A voice data analysis device according to one aspect of the present disclosure includes a conversational voice data acquisition unit, a conversational text generation unit, and a conversation content analysis unit. The conversation sound data acquisition unit is configured to acquire conversation sound data in which the voice of a conversation is recorded. The conversation text generation unit is configured to generate conversation text data. The conversation text data is text data of a conversation content included in the conversation sound data. The conversation content analysis unit is configured to analyze the conversation content using the conversation text data to obtain an analysis result.

[0007] This voice data analysis device acquires conversational voice data in which the voices of conversations are recorded and analyzes the conversational voice data, so that the conversational voice data can be analyzed while suppressing the effects of noise. Therefore, the voice data analysis device can accurately analyze the voices of store clerks' conversations while reducing the effects of noise.

[0008] In the voice data analysis device of the present disclosure, the conversational voice data acquisition unit may be configured to acquire the conversational voice data from a recording device. The recording device may be configured to be portable by the speaker. The recording device may also be configured to record the voice of the speaker to record the conversational voice data.

[0009] This voice data analysis device can record the conversation of a speaker while the speaker is carrying the recording device, thereby generating conversation sound data corresponding to the content of the speaker's conversation while suppressing the influence of noise by the recording device.

[0010] In the voice data analysis device of the present disclosure, the conversation sound data acquisition unit may be configured to removably attach the recording device, and may be configured to acquire the conversation sound data in response to the recording device being attached.

[0011] According to this voice data analysis device, by attaching a recording device to the conversation sound data acquisition unit, conversation sound data can be easily acquired from the recording device. This simplifies the task of acquiring conversation sound data by using the voice data analysis device.

[0012] In the voice data analysis device of the present disclosure, the conversation sound data acquisition unit may be configured to charge the attached recording device. By attaching the recording device to the conversation sound data acquisition unit, this voice data analysis device can simultaneously achieve the acquisition of conversation sound data and the charging of the recording device.

[0013] In the voice data analysis device of the present disclosure, the conversation sound data acquisition unit may be configured to acquire conversation sound data from the recording device via wireless communication. In this voice data analysis device, even if the conversation sound data acquisition unit and the recording device are not directly connected and are located apart from each other, the conversation sound data acquisition unit can acquire the conversation sound data from the recording device via wireless communication. This simplifies the task of acquiring conversation sound data from the recording device by using the voice data analysis device.

[0014] In the voice data analysis device of the present disclosure, the conversation sound data acquisition unit may be configured to acquire the conversation sound data in response to receiving a data transmission command from the recording device. The voice data analysis device can start acquiring the conversation sound data based on the timing of receiving the data transmission command.

[0015] In the voice data analysis device of the present disclosure, the recording device may be configured to record conversational voice data or to record voice data. The voice data may include conversational voice data and background voice data. The conversational voice data acquisition unit may include a voice data acquisition unit and a conversational voice data extraction unit. The voice data acquisition unit may be configured to acquire conversational voice data or voice data. The conversational voice data extraction unit may be configured to extract the conversational voice data from the voice data.

[0016] Thereby, when the conversation sound data acquisition unit acquires voice data, it can acquire the conversation sound data by extracting the conversation sound data from the voice data. The background sound data is information that records sounds that are different from conversation and occur in the background of a speaker.

[0017] Furthermore, when the conversation sound data acquisition unit acquires the conversation sound data, the conversation text generation unit may generate the conversation text data using the conversation sound data. When the conversation sound data acquisition unit acquires the voice data, the conversation text generation unit may generate the conversation text data using the conversation sound data extracted from the voice data.

[0018] A voice data analysis method according to another aspect of the present disclosure may include acquiring conversation sound data in which a conversation sound is recorded, generating conversation text data, and analyzing the content of the conversation using the conversation text data to generate analysis information. The conversation text data is text data of the conversation content included in the conversation sound data.

[0019] This voice data analysis method acquires conversational voice data in which the voices of conversations are recorded and analyzes the conversational voice data, so that the conversational voice data can be analyzed while suppressing the influence of noise. Therefore, the voice data analysis method can accurately analyze the voices of store clerks' conversations while reducing the influence of noise.

[0020] In the above-mentioned audio data analysis method, obtaining conversation sound data may comprise obtaining conversation sound data or audio data, and extracting the conversation sound data from the audio data. The audio data includes conversation sound data and background sound data.

[0021] As a result, in the voice data analysis method, when voice data is acquired, conversation sound data can be acquired by extracting the conversation sound data from the voice data. The background sound data is information that records sounds that are different from conversation and occur in the background of a speaker.

[0022] A computer program in another aspect of the present disclosure may be a computer program for causing a computer to function as at least one of the conversation sound data acquisition unit, the conversation text generation unit, and the conversation content analysis unit in the above-mentioned voice data analysis device.

[0023] The above voice data analysis method may be executed by a computer. A voice data analysis method to which the same technical idea as the above voice data analysis device is applied may be provided. A computer program having instructions for causing a computer to execute these voice data analysis methods may be provided. The computer program may be recorded on a non-transitory computer-readable recording medium. [Brief description of the drawings]

[0024] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a voice data analysis device. [Diagram 2] 2A to 2E are explanatory diagrams of the recording device, with FIG. 2A being a plan view, FIG. 2B being a front view, FIG. 2C being a right side view, FIG. 2D being a rear view, and FIG. 2E being a bottom view. [Diagram 3] FIG. 2 is an explanatory diagram showing an example of a mounting state of a recording device. [Figure 4] 4A to 4F are explanatory diagrams of the equipment attachment / detachment section, where FIG. 4A is a plan view, FIG. 4B is a front view, FIG. 4C is a right side view, FIG. 4D is a rear view, FIG. 4E is a bottom view, and FIG. 4F is an end view of the IVF-IVF cross section of FIG. 4A. [Diagram 5] 13 is a flowchart showing the processing contents of a voice data analysis process. [Figure 6] 1 is an explanatory diagram in which the upper area typically shows speech periods and noise periods in a speech data waveform, and the lower area typically shows text data generated from the speech data waveform in the speech period. [Figure 7] 13 is a flowchart showing the processing contents of an analysis process. [Figure 8] FIG. 2 is an explanatory diagram showing an example of conversation text data TD1 to which a time stamp is added. [Figure 9] An explanatory diagram showing an example of matching data in which seating start time, seating end time, and service genre are recorded. [Figure 10] FIG. 11 is an explanatory diagram showing an example of a period determination result obtained by the analysis process. [Figure 11]FIG. 11 is an explanatory diagram showing an example of analysis information generated by a conversation content analysis. [Figure 12] FIG. 13 is an explanatory diagram showing an example of a display screen showing performance data for each product salesperson. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] Exemplary embodiments of the present disclosure will now be described with reference to the drawings. It goes without saying that the present disclosure is in no way limited to the following embodiments, and various forms can be adopted as long as they fall within the technical scope of the present disclosure.

[0026] [1. First embodiment] [1-1. Overall composition] The voice data analysis device 1 of the first embodiment is a system for analyzing voices uttered by product salespeople (hereinafter also referred to as users). The voice data analysis device 1 is configured to analyze voices uttered by product salespeople during sales activities and support sales activities through conversation. An example of sales activities is when a product salesperson calls out to people passing by a mobile phone sales area in a shopping mall and explains the product to them in order to sell them mobile phones.

[0027] As shown in FIG. 1, the voice data analysis device 1 includes a recording device 11, a device attachment / detachment unit 13, a server device 21, a display device 41, and a mobile terminal device 43. Note that the voice data analysis device 1 is provided with a plurality of recording devices 11 corresponding to the number of product sales staff, but only one recording device 11 is shown in Fig. 1. The number of recording devices 11 corresponds to the number of product sales staff. Similarly, the voice data analysis device 1 is provided with a plurality of mobile terminal devices 43 corresponding to the number of product sales staff, but only one mobile terminal device 43 is shown in Fig. 1. The number of mobile terminal devices 43 corresponds to the number of product sales staff.

[0028] [1-2. Recording equipment] The recording device 11 is a portable device that is provided separately from the device attachment / detachment section 13 and the server device 21.

[0029] As shown in Figures 1 and 2, the recording device 11 includes a microphone 11a, an operation unit 11b, a control unit 11c, a memory unit 11d, a charging terminal 11e, a first communication terminal 11f, a clip unit 11g, and a housing 11h.

[0030] The microphone 11a, the operation unit 11b, the charging terminal 11e, and the first communication terminal 11f are each provided with a part exposed to the outside of the housing 11h. The clip unit 11g is provided on the outside of the housing 11h. The control unit 11c and the storage unit 11d are provided inside the housing 11h.

[0031] The recording device 11 is configured so that it can be worn by a salesperson in a state in which the voice of the salesperson can be recorded. For example, the recording device 11 is configured so that it can be worn around the collar or breast pocket of the salesperson by using a clip part 11g. The recording device 11 is equipped with a rechargeable / dischargeable battery (not shown). The recording device 11 is configured to operate using power stored in the battery.

[0032] The microphone 11a converts the user's voice into an electrical audio signal and outputs it to the control unit 11c. The operation unit 11b has one or more mechanical switches that can receive recording instructions and stop instructions from the user. The control unit 11c is configured to comprehensively control each unit of the recording device 11.

[0033] When a recording instruction is input from the user through the operation unit 11b, the control unit 11c converts the user's voice signal input from the microphone 11a from analog voice data to digital voice data and records it in the storage unit 11d until a stop instruction is input. The control unit 11c performs A / D conversion (analog-to-digital conversion) of the voice data and stores the converted voice data DA1 (digital data) in the storage unit 11d. The voice data DA1 may be, for example, digital data in binary format (bin format).

[0034] The storage unit 11d is an electrically rewritable semiconductor memory, such as a flash memory. Hereinafter, the audio data recorded in the storage unit 11d is also referred to as recorded data.

[0035] The charging terminal 11e is configured so as to be directly connectable to the device attachment / detachment unit 13 (more specifically, the power supply terminal 13a, which will be described later). The charging terminal 11e is configured so as to receive charging power from the connected device attachment / detachment unit 13. The charging power is used to charge the battery (not shown) of the recording device 11.

[0036] The first communication terminal 11f is configured to be directly connected to the device attachment / detachment unit 13 (more specifically, the second communication terminal 13b described later) so as to be able to perform wired communication. The first communication terminal 11f is configured to configure a wired communication path together with the device attachment / detachment unit 13 to which it is connected.

[0037] The control unit 11c is configured to transmit and receive various information to and from the server device 21 by wired communication via the first communication terminal 11f and the device attachment / detachment unit 13. The control unit 11c is configured to transmit the recording data stored in the storage unit 11d to the server device 21 in response to a request from the server device 21.

[0038] As shown in Fig. 2, clip portion 11g includes a movable portion 112 and a fixed connecting portion 113. In Fig. 2, the directions in each drawing are represented as U for the upward direction, D for the downward direction, R for the rightward direction, L for the leftward direction, Fr for the frontward direction, and Rr for the rearward direction.

[0039] The movable part 112 is linear in side view. The movable part 112 extends from the end on the front side of the housing 11h toward the rear side to a position beyond the center of the housing 11h in the longitudinal direction (in other words, the direction from the front to the rear). When the salesperson is not wearing the recording device 11, the movable part 112 is inclined from the front to the rear with respect to the housing 11h in side view. Specifically, one end 112a (end on the front side) of the movable part 112 in the longitudinal direction is separated from the housing 11h, and the other end 112b (end on the rear side) of the movable part 112 in the longitudinal direction is in contact with the housing 11h.

[0040] The fixed connecting part 113 is configured to be substantially U-shaped in side view. In this embodiment, the fixed connecting part 113 includes a first connecting part connected to the housing 11h and a second connecting part connected to the movable part 112. The first connecting part of the fixed connecting part 113 is fixed to the housing 11h in a state where it is embedded inside the housing 11h. The second connecting part of the fixed connecting part 113 is configured as a hinge part 113a. The movable part 112 is configured to be rotatable with respect to the fixed connecting part 113 and the housing 11h with the hinge part 113a as a rotation axis. That is, by pushing one end part 112a of the movable part 112 in the longitudinal direction toward the housing 11h side (the lower side of FIG. 2C), the other end part 112b of the movable part 112 in the longitudinal direction moves away from the housing 11h. With the other end 112b separated from the housing 11h, the cloth of the clothing is inserted into the insertion space between the movable part 112 and the housing 11h, and the force applied to the one end 112a is released, so that the other end 112b of the movable part 112 is biased toward the housing 11h, and the cloth of the clothing is clamped between the movable part 112 and the housing 11h. For example, as shown in Fig. 3, the recording device 11 can be attached to the collar of a salesperson by using the clip part 11g.

[0041] [1-3.Equipment attachment / detachment section] 1 and 4, the device attachment / detachment unit 13 is connected to the server device 21 via a wired cable 14. The device attachment / detachment unit 13 is configured so that the recording device 11 can be detachably attached thereto. The device attachment / detachment unit 13 includes a power supply terminal 13a and a second communication terminal 13b.

[0042] The power supply terminal 13a is configured to receive charging power supplied from the server device 21 via the wired cable 14. The power supply terminal 13a is configured to output the charging power supplied from the server device 21 to the recording device 11 by being directly connected to the charging terminal 11e of the recording device 11. The power supply terminal 13a, together with the connected charging terminal 11e, constitutes a part of a charging path for charging the recording device 11 by the server device 21. This charging path is a path for sending charging power from the server device 21 to the recording device 11.

[0043] The second communication terminal 13b is configured to be directly connected to the first communication terminal 11f of the recording device 11 for wired communication. The second communication terminal 13b is configured to be electrically connected to the server device 21 via a wired cable 14 for wired communication. The second communication terminal 13b, together with the connected first communication terminal 11f, constitutes a part of the wired communication path between the recording device 11 and the server device 21. This wired communication path is configured to transmit various information transmitted and received between the recording device 11 and the server device 21.

[0044] As shown in Fig. 4, the device attachment / detachment section 13 has a substantially rectangular parallelepiped external shape. As shown in Fig. 4F, the device attachment / detachment section 13 is formed with a rectangular parallelepiped recess 121 for inserting the sound recording device 11 downward from the top surface of the device attachment / detachment section 13. In other words, the sound recording device 11 can be attached to the device attachment / detachment section 13 in an upright state.

[0045] In this embodiment, a rectangular parallelepiped recess 122 is formed behind the recess 121 (on the right side of FIG. 4F). This recess 122 is intended to make it easier to remove the recording device 11 by providing a space for the user's fingers when removing the recording device 11 from the device attachment / detachment section 13. Specifically, the recess 121 and the recess 122 behind it form a stepped recess in the upper surface of the device attachment / detachment section 13 in a side view. Specifically, the device attachment / detachment section 13 includes a first bottom surface 121a forming the bottom surface of the recess 121 and a second bottom surface 122a forming the bottom surface of the recess 122. The second bottom surface 122a is formed at a higher position than the first bottom surface 121a.

[0046] The device attachment / detachment section 13 is provided with a contact terminal 121b shown in FIG. 4A and FIG. 4F on the first bottom surface 121a. The contact terminal 121b is provided with the above-mentioned power supply terminal 13a and the second communication terminal 13b. In response to the sound recording device 11 being attached to the device attachment / detachment section 13, the power supply terminal 13a is connected to the charging terminal 11e of the sound recording device 11, and the second communication terminal 13b is connected to the first communication terminal 11f of the sound recording device 11. In other words, in response to the sound recording device 11 being attached to the device attachment / detachment section 13, the sound recording device 11 can be charged and communication between the sound recording device 11 and the device attachment / detachment section 13 becomes possible. By making communication possible in this way, for example, sound data DA1 can be moved from the sound recording device 11 to the server device 21 via the device attachment / detachment section 13.

[0047] The device attachment / detachment section 13 has a wall section 124 on the front side. The sound recording device 11 is attached to the device attachment / detachment section 13 with the wall section 124 sandwiched between the housing 11h of the sound recording device 11 and the clip section 11g.

[0048] 4B and 4F, a groove 124a extending in the up-down direction is formed on the outer surface of the wall 124 at the center in the left-right direction of the wall 124. The clip 11g fits into the groove 124a when the sound recording device 11 is attached to the device attachment / detachment section 13. The groove 124a is provided to guide the user how to attach the sound recording device 11 to the device attachment / detachment section 13.

[0049] [1-4. Portable terminal device] The mobile terminal device 43 is an electronic device terminal that can be carried by a product salesperson. The mobile terminal device 43 is configured to be able to communicate with the server device 21 (more specifically, the communication unit 27) via wireless communication. The mobile terminal device 43 may be configured using, for example, a smartphone, a tablet terminal, or the like.

[0050] Application software for performing various operations related to the voice data DA1 is installed in the mobile terminal device 43. This application software has at least a voice recording function and a voice data output function. The voice recording function is a function for recording the user's voice as digital voice data DA1. The voice data output function is a function for outputting the recorded voice data DA1 to another device.

[0051] Although not shown, the portable terminal device 43 includes a microphone, an operation unit, a control unit, and a storage unit. The microphone, operation unit, control unit, and storage unit of the portable terminal device 43 have the same functions as the microphone, operation unit, control unit, and storage unit of the recording device 11, respectively. The operation unit of the portable terminal device 43 may be, for example, a touch panel provided on the screen display unit.

[0052] When the application software is started by a user operation, the mobile terminal device 43 displays a voice data operation screen including a recording start button, a recording stop button, and a data transmission button on the screen display unit. When a user inputs a recording instruction through the recording stop button, the mobile terminal device 43 converts the user's voice signal input from the microphone from analog voice data to digital voice data and records it in the storage unit as voice data DA1 until the user inputs a stop instruction through the recording stop button.

[0053] The mobile terminal device 43 is configured to output a data transmission command Sa to the server device 21 (more specifically, the communication unit 27). The mobile terminal device 43 outputs the data transmission command Sa by wireless communication. The mobile terminal device 43 outputs the data transmission command Sa to the server device 21 in response to a user's operation via a data transmission button. The data transmission command Sa is a command for notifying the server device 21 in advance when transmitting the voice data DA1 from the mobile terminal device 43 to the server device 21. After outputting the data transmission command Sa, the mobile terminal device 43 starts transmitting the voice data DA1 to the server device 21 in response to receiving a transmission permission from the server device 21.

[0054] [1-5. Server equipment] 1, the server device 21 includes a control unit 23, a storage unit 25, and a communication unit 27. In detail, the server device 21 is configured by one or more computers. The control unit 23 includes a CPU 23a. The CPU 23a realizes various functions by executing processes in accordance with a computer program stored in the storage unit 25. In the following, the processes executed by the CPU 23a may be described as processes executed by the control unit 23 or the server device 21.

[0055] The storage unit 25 stores computer programs executed by the CPU 23a and various data. The storage unit 25 includes one or more hard disk drives (HDD) and / or solid state drives (SSD). The communication unit 27 is configured to be able to communicate with the recording device 11 via the wired cable 14 and the device attachment / detachment unit 13. The communication unit 27 is configured to be able to communicate with the mobile terminal device 43 via wireless communication. Examples of wireless communication include mobile data communication, wireless LAN (WiFi (registered trademark), etc.), and short-range wireless communication (Bluetooth (registered trademark) communication, infrared communication, etc.).

[0056] Control unit 23 communicates with recording device 11 via communication unit 27, receives audio data from recording device 11, and executes various processes based on the audio data. Alternatively, control unit 23 communicates with mobile terminal device 43 via communication unit 27, receives audio data from mobile terminal device 43, and executes various processes based on the audio data.

[0057] The voice data includes at least conversational voice data and background voice data. The conversational voice data includes voice data uttered by a product salesperson to a customer during sales activities. The background voice data includes voice data other than the voice data of the product salesperson during sales activities (such as background music and surrounding noise).

[0058] For example, the control unit 23 executes a process of extracting conversation sound data from the received voice data. The control unit 23 also executes a process of generating text data of the conversation contents included in the conversation sound data (hereinafter also referred to as conversation text data). The control unit 23 also executes a process of determining "calling periods, standing negotiation periods, sitting periods, other periods" in the conversation sound data based on the conversation text data. The control unit 23 also executes a process of analyzing the conversation contents of each of the "calling periods, standing negotiation periods, sitting periods, other periods" by using the conversation text data for each of the "calling periods, standing negotiation periods, sitting periods, other periods" to generate analysis information.

[0059] The control unit 23 executes each process for each of the recording devices 11, thereby generating analysis information on the content of the conversation for each of the salespeople. [1-6.Display device] The display device 41 is connected to the server device 21 via a wired cable or wireless communication. The display device 41 is configured to display various information based on a display command from the server device 21. The display device 41 may be configured using, for example, a liquid crystal display, an organic EL display, or the like.

[0060] [1-7. Audio data analysis processing] Next, the voice data analysis process executed by the server device 21 will be described. The server device 21 executes the voice data analysis process shown in FIG. 5 in response to the start-up of the server device 21 itself.

[0061] When the voice data analysis process is started, the server device 21 (more specifically, the control unit 23) judges whether or not a predetermined acquisition condition is satisfied (S100 (S represents a step)), and if the judgment is affirmative, the process proceeds to S110, and if the judgment is negative, the same step is repeatedly executed. The acquisition condition is set as a condition for judging whether or not the server device 21 is in a state in which it can acquire the voice data DA1 from an external device.

[0062] The acquisition condition for the voice data DA1 is set to be satisfied at least when the recording device 11 is attached to the device attachment / detachment unit 13. Also, the acquisition condition for the voice data DA1 is set to be satisfied in response to receiving a data transmission command Sa from the mobile terminal device 43. In other words, the server device 21 is configured so that the acquisition condition for the voice data DA1 is satisfied in response to the recording device 11 being attached to the device attachment / detachment unit 13 or in response to receiving a data transmission command Sa from the mobile terminal device 43.

[0063] Before mounting the recording device 11 on the device mounting section 13, the recording device 11 is mounted around the collar or breast pocket of the salesperson to record audio data including the voice uttered by the salesperson during sales activities. As described above, in the recording device 11, the control section 11c performs A / D conversion (analog-to-digital conversion) of the audio data, and stores the converted audio data DA1 (digital data) in the storage section 11d.

[0064] The audio data DA1 includes conversational audio data DA2 and background audio data DA3, as shown typically in the upper region of Fig. 6. The conversational audio data DA2 includes audio data including the voice uttered by a product salesperson during sales activities. In Fig. 6, the conversational audio data DA2 is data for sections described as Conversation A, Conversation B, and Conversation C. The background audio data DA3 includes audio data that is not the audio data of a product salesperson during sales activities. In Fig. 6, the background audio data DA3 is data for a section described as noise.

[0065] Furthermore, before the portable terminal device 43 transmits the data transmission command Sa, the portable terminal device 43 is placed near the salesperson to record voice data including the voice uttered by the salesperson during sales activities. The portable terminal device 43 may record voice while being worn by the salesperson, for example, by being placed in the salesperson's breast pocket or hung by a neck strap. Like the recording device 11, the portable terminal device 43 performs A / D conversion (analog-digital conversion) on the voice data and stores the converted voice data DA1 (digital data).

[0066] When the control unit 23 makes a positive determination in S100, it acquires voice data DA1 from the recording device 11 (or the mobile terminal device 43) and stores (saves) the voice data DA1 in the storage unit 25 (S110). The control unit 23 converts the acquired voice data DA1 into a predetermined data format (for example, WAV format, etc.) and stores it in the storage unit 25. The voice data DA1 acquired at this time may be the voice data DA1 recorded by the recording device 11 (or the mobile terminal device 43) on the same day, or may be all of the voice data DA1 stored in the recording device 11 (or the mobile terminal device 43) that has not yet been acquired by the server device 21.

[0067] The control unit 23 performs a process of extracting the conversation sound data DA2 from the voice data DA1 (S120). At this time, the control unit 23 extracts the conversation sound data DA2 using a known VAD (Voice Activity Detection) technique.

[0068] The control unit 23 performs a process of generating conversation text data TD1 corresponding to the conversation included in the conversation sound data DA2 (S130). At this time, the control unit 23 generates the conversation text data TD1 based on the conversation sound data DA2 using a known STT (Speech To Text) technique. The conversation text data TD1 is generated as text data corresponding to the conversations A, B, and C of the voice data DA1, as shown typically in the lower region of FIG.

[0069] The control unit 23 performs a process of adding a time stamp (time information) to the conversation text data TD1 (S140). For example, for each of the conversations A, B, and C in the conversation text data TD1 shown in Fig. 6, the control unit 23 adds the start time of the corresponding section in the conversation sound data DA2 as a time stamp.

[0070] 8 shows an example of conversation text data TD1 with a timestamp. It shows the date and time (timestamp) and the main text (text content) of each of the conversation text data TD1 from number 1 to number 50. In this example, the conversation text data TD1 is based on the speech of a product salesperson recorded by the recording device 11 (or the mobile terminal device 43), and shows the conversation text data TD1 from "2023-06-08 10:43:44" to "2023-06-08_10:57:34" with some parts omitted.

[0071] The control unit 23 performs an analysis process of the conversation sound data DA2 based on the conversation text data TD1 (S150). As shown in Fig. 7, in the analysis process, the control unit 23 performs a period determination process using the conversation text data TD1 and a machine learning model (S210). The period determination process is a process of determining whether a determination target period in the conversation sound data DA2 corresponds to a "calling period, standing negotiation period, sitting period, other period." In other words, the period determination process is a process of determining a period in the recording period of the conversation sound data DA2 that corresponds to each of the "calling period, standing negotiation period, sitting period, other period."

[0072] The "calling period, standing negotiation period, sitting period, and other periods" are periods obtained by subdividing the sales activity period according to the customer service state. The calling period is a period corresponding to a calling state in which a salesperson calls out to people passing by. The standing negotiation period is a period corresponding to a standing negotiation state in which a person who was about to pass by stops in response to a call or other such action, and the salesperson has a conversation with the person who stopped. The sitting period is a period corresponding to a seated negotiation state in which a person who has stopped sits down in a chair in the negotiation area, and the salesperson has a conversation with the seated person.

[0073] Other periods are periods that do not fall into any of the "calling periods, standing negotiation periods, or sitting periods." Other periods include, for example, periods related to business-related conversations between product salespeople, and periods related to special "standing negotiations" that start not with a "calling" but with a "standing negotiation." Sales activities usually start with a "calling," but there are rare cases where they start with a "standing negotiation." Such irregular "standing negotiations" are classified as "other periods."

[0074] The machine learning model used in S210 is a model configured to output the period of a business negotiation corresponding to the conversation text data TD1 in response to the input of the conversation text data TD1. In other words, this machine learning model is a model for determining the period of a business negotiation corresponding to the input conversation text data TD1.

[0075] The machine learning model used in S210 is a model obtained by so-called "supervised learning." As supervising data for learning (in other words, training data), supervising data is used in which a given conversation sentence (text data) is associated with the period of the negotiation in which the conversation sentence was spoken (a calling period, a standing negotiation period, a seated period, or any other period).

[0076] The training data may be prepared, for example, for 100 days, for 50 salespeople, with 8 hours of speech recorded per day. In the learning process using this training data, a machine learning model may be constructed taking into account the characteristics of the speech sentences, the context, and the consistency of the transition order of each period. The machine learning model does not need to be limited to a form in which a model is constructed once and used as is indefinitely. The machine learning model may be configured to learn using new training data containing newly recorded conversation sentences (text data) after a predetermined period of time has passed, and the learning results may be updated. The machine learning model may be updated at any interval, such as once a month, every few months, or once a year.

[0077] When an unnatural period determination is found in the period determination result in S210, the control unit 23 performs a first correction process to correct the unnatural period determination (S220). For example, as described above, sales activities are usually started with a "call", but in rare cases, they may start with a "standing negotiation" without passing through a "call". Even for such irregular "standing negotiation", in the case of conversation text data TD1 including a phrase used in a "standing negotiation", the machine learning model determines the "standing negotiation period", making it difficult to distinguish it from a normal "standing negotiation period". Therefore, instead of determining based only on the conversation text data TD1, the determination result is corrected by also paying attention to the transition of the period of the sales activity. For example, for a period determined to be a "standing negotiation period" from the conversation text data TD1, in the case of a sales activity that started from a "standing negotiation period" without passing through a "call period", the period determination result is corrected by correcting the classification of the "standing negotiation period" to "other periods".

[0078] The control unit 23 performs a second correction process for correcting the period determination result in S210 using the comparison data (S230). As the comparison data, for example, a timestamp (time information) of the customer service record recorded in the mobile terminal device 43 used by the product salesperson may be used. The mobile terminal device 43 is equipped with a dedicated app for recording sales activities, and the product salesperson inputs the customer service record using the dedicated app after serving a customer. When inputting this customer service record, a timestamp including the seating start time and the seating end time is automatically recorded in the storage area of ​​the dedicated app of the mobile terminal device 43.

[0079] An example of the matching data is shown in Fig. 9. This matching data has a sitting start time of "2023-06-08 10:45:45", a sitting end time of "2023-06-08 10:55:15", and a service category of "sitting".

[0080] Control unit 23 acquires a timestamp of the customer service record from a dedicated application of mobile terminal device 43 at a predetermined time (for example, when executing S230 or S110). Control unit 23 uses the customer service time period during which the customer was seated, identified based on the timestamp of the customer service record, as matching data, and determines whether the period determination result in S210 corresponding to the seated time period of the matching data is a "seating period". If the period determination result in S210 is a "seating period", control unit 23 maintains the determination result, and if the period determination result in S210 is different from a "seating period", corrects the period determination result by correcting the period determination result to a "seating period".

[0081] FIG. 10 shows an example of a period determination result obtained in the analysis process (more specifically, the processes of S210 to S230). In Fig. 10, the results of period determination for each of the conversation text data TD1 numbers 1 to 50 are entered in the "Phase" column of the "Period Data" column. In addition, the start and end points of a series of sales activities for one customer (or one group of customers) are entered in the "Start / End" column of the "Period Data" column. Specifically, "Start" is entered in the conversation text data TD1 number 2, which corresponds to the start of the series of sales activities, and "End" is entered in the conversation text data TD1 number 49, which corresponds to the end.

[0082] Furthermore, in FIG. 10, there are provided the following items: "Salesperson input," "Standing negotiation count," and "Sitting count." The "salesperson input" is information based on the above-mentioned verification data. The conversation text data TD1 for number 6 contains "start sitting down," and the conversation text data TD1 for number 49 contains "end sitting down."

[0083] "Standing negotiation count" is bit data for counting the number of standing negotiations performed by a product salesperson. "1" is set to the first conversation text data TD1 (conversation text data TD1 numbered 3 in FIG. 10) among a series of conversation text data TD1 determined to be "standing negotiations" (conversation text data TD1 numbered 3 to 5 in FIG. 10).

[0084] Fig. 10 shows a part of the conversation text data TD1 (and the period determination results) on the analysis target day, and there is conversation text data TD1 (and the period determination results) that is not shown in Fig. 10. The sum of the number of times "1" is set in the "standing negotiation count" on the analysis target day represents the standing negotiation performance of the product salesperson on that day.

[0085] "Sitting Count" is bit data for counting the number of times a salesperson sat down during negotiations. Among a series of conversation text data determined to be "sitting down" (conversation text data numbered 6 to 49 in FIG. 10), the first conversation text data (conversation text data numbered 6 in FIG. 10) is set to "1." As with "Standing Negotiation Count," the total number of times "1" is set in "Sitting Count" on the day being analyzed represents the number of seated negotiations performed by the salesperson on that day.

[0086] The control unit 23 performs a content analysis process based on the conversation sound data DA2 and the conversation text data TD1 (S240). First, as a content analysis process, the control unit 23 performs a conversation content analysis regarding "voice volume, etc." based on the conversation sound data DA2, and also performs a conversation content analysis regarding "category, greetings, nicknames, empathy, etc." based on the conversation text data TD1.

[0087] FIG. 11 shows an example of analysis information generated by the conversation content analysis. FIG. 11 shows examples of analysis information generated by conversation content analysis of the conversation sound data DA2 from number 1 to number 50 and the conversation text data TD1, including examples of the items "category, greeting, nickname, voice volume, and empathy."

[0088] "Category" is analysis information that indicates whether or not the conversation content contains terms related to categories such as "fees, terminal, radio waves, data capacity," and if so, the category name (in FIG. 11, "fees") is entered in the conversation text data TD1 (in FIG. 11, conversation text data TD1 numbered 3).

[0089] "Greetings" is analysis information that indicates whether greeting terms such as "hello" are included in the conversation content, and if they are included, "1" is set in the conversation text data TD1 (in Figure 11, conversation text data TD1 number 2).

[0090] "Nickname" is analysis information that indicates whether or not nickname terms such as "Dad" and "Mom" are included in the conversation content. If they are included, "1" is set in the conversation text data TD1 (in FIG. 11, the conversation text data TD1 numbered 2).

[0091] "Voice volume" is analysis information that expresses the volume of the salesperson's voice as a numerical value, and a numerical value is recorded for each conversation text data TD1. "Sympathy" is analysis information that indicates whether or not a term that evokes sympathy from customers is included in the conversation content, and if it is included, "1" is set in the conversation text data TD1 (conversation text data TD1 with numbers 2 and 4 in FIG. 11). Examples of terms that evoke sympathy from customers include "please take it" and "are you interested?".

[0092] Furthermore, as a continuation of the content analysis process, the control unit 23 creates performance data for each product salesperson in which the analysis information generated by the above-mentioned conversation content analysis is associated with each period (calling out, standing while negotiating, sitting down).

[0093] An example of a display screen showing the performance data for each product salesperson is shown in Fig. 12. This display screen is displayed on the display device 41 or the mobile terminal device 43, for example. Figure 12 shows the performance data for three salespeople (clerks C, D, and J). Each item is marked as follows: "that number" if the result is a number, "◎" for extremely good results, "○" for good results, "R" for sufficient results, "△" for insufficient results, and (blank) if analysis is not possible. Note that if the analysis period (standing negotiations, sitting, etc.) has not been reached, "-" or "0" is entered for each item for that period (in Figure 12, each item is for the standing negotiation period for clerk C).

[0094] Furthermore, as a continuation of the content analysis process (S240), the control unit 23 creates detailed performance data for each item for one product salesperson. [1-8.Effects] According to the embodiment described above in detail, the following effects can be obtained.

[0095] (1a) In the voice data analysis device 1, the control unit 23 extracts conversation sound data DA2 from the acquired voice data DA1 and analyzes it (S120), so that the voice data DA1 can be analyzed while suppressing the influence of background sound data DA3 (S150). Therefore, the voice data analysis device 1 can accurately analyze the voice of the store clerk while reducing the influence of noise.

[0096] (1b) In the voice data analysis device 1, the recording device 11 is configured so that it can be carried by the salesperson (in other words, the speaker) using the clip unit 11g, and the conversation of the salesperson can be recorded while the recording device 11 is carried by the salesperson. This allows the recording device 11 to generate voice data corresponding to the content of the conversation of the salesperson while suppressing the effects of noise.

[0097] (1c) In the voice data analysis device 1, the server device 21 can easily acquire the voice data DA1 from the recording device 11 by attaching the recording device 11 to the device attachment / detachment unit 13. This allows a user of the voice data analysis device 1 to move the voice data DA1 from the recording device 11 to the server device 21 by attaching the recording device 11 to the device attachment / detachment unit 13 without performing complicated operations. Therefore, by using the voice data analysis device 1, the task of acquiring the voice data DA1 is simplified.

[0098] (1d) In the voice data analysis device 1, the device attachment / detachment unit 13 is configured to charge the attached recording device 11. Therefore, by attaching the recording device 11 to the device attachment / detachment unit 13, the voice data analysis device 1 can simultaneously achieve the operation of acquiring the voice data DA1 and the operation of charging the recording device 11.

[0099] (1e) In the voice data analysis device 1, the control unit 23 and the communication unit 27 are configured to acquire voice data DA1 from the mobile terminal device 43 via wireless communication. Even if the server device 21 (more specifically, the control unit 23 and the communication unit 27) and the mobile terminal device 43 are not directly connected and are located apart from each other, the voice data analysis device 1 can allow the server device 21 to acquire the voice data DA1 from the mobile terminal device 43 via wireless communication. This simplifies the task of acquiring the voice data DA1 from the mobile terminal device 43 by using the voice data analysis device 1.

[0100] In the voice data analysis device 1, the server device 21 (more specifically, the control unit 23 and the communication unit 27) acquires the voice data DA1 (S110) in response to receiving a data transmission command Sa from the mobile terminal device 43 (positive determination in S100). The voice data analysis device 1 can start the operation of acquiring the voice data DA1 based on the timing of receiving the data transmission command Sa.

[0101] [1-9. Correspondence of terms] In this embodiment, the device attachment / detachment unit 13, the control unit 23, and the communication unit 27 correspond to an example of a conversation sound data acquisition unit or an example of a voice data acquisition unit in this disclosure. The control unit 23 corresponds to an example of a conversation sound data extraction unit in this disclosure. The control unit 23 corresponds to an example of a conversation text generation unit in this disclosure. The control unit 23 corresponds to an example of a conversation content analysis unit in this disclosure. The recording device 11 and the mobile terminal device 43 each correspond to an example of a recording device in this disclosure.

[0102] 2. Other embodiments Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments and can be implemented in various modified forms.

[0103] (2a) In the above embodiment, the voice data analysis device 1 is described as being configured to analyze the voice of a salesperson (in other words, the voice data DA1) recorded using either the recording device 11 or the mobile terminal device 43, but the voice data analysis device of the present disclosure is not limited to this configuration. For example, the voice data analysis device of the present disclosure may be configured such that the only external device that transmits the voice data DA1 to the server device 21 is the recording device 11, or the only external device is the mobile terminal device 43.

[0104] (2b) In the above embodiment, the voice data DA1 is temporarily recorded in the recording device 11 or the mobile terminal device 43, but the voice data analysis device of the present disclosure is not limited to this configuration. For example, the voice data collected by the recording device 11 or the mobile terminal device 43 may be immediately transmitted to the server device 21.

[0105] (2c) In the above embodiment, sales activities (sales activities by mobile phone salespeople at a shopping mall) are given as an example of the target of voice analysis using the voice data analysis device of the present disclosure, but the present disclosure may be applied to a voice data analysis device that targets other sales activities. For example, the present disclosure may be applied to the voice of a salesperson in sales activities for selling learning materials, sales activities for selling automobiles, sales activities for selling home appliances, sales activities for selling cosmetics, sales activities for selling insurance products, sales activities for providing information on joining a sports club, etc., as the target of voice analysis. In addition, the present disclosure may be applied to analyze the voice of a person when the person speaks in a noisy situation, without being limited to sales activities.

[0106] (2d) In the above embodiment, the voice data analysis device 1 of the present disclosure is configured to acquire voice data DA1 (S110) and perform a process of extracting conversation sound data DA2 from the voice data DA1 (S120), but the voice data analysis device of the present disclosure is not limited to such a configuration. For example, if the recording device 11 stores conversation sound data DA2, the voice data analysis device may be configured to acquire the conversation sound data DA2 instead of the voice data DA1. In this case, the process of extracting the conversation sound data DA2 from the voice data DA1 (S120) can be omitted.

[0107] (2e) In the above embodiment, the analysis target using the voice data analysis device 1 of the present disclosure is the sales activities of a product salesperson, but the analysis target is not limited to the sales activities of a product salesperson. For example, the analysis target may be a business negotiation that has been scheduled in advance. Or, the analysis target may be a conversation between a supplier and a person in charge of a business partner.

[0108] (2f) Multiple functions possessed by one component in the above embodiments may be realized by multiple components, or one function possessed by one component may be realized by multiple components. Also, multiple functions possessed by multiple components may be realized by one component, or one function realized by multiple components may be realized by one component. Also, part of the configuration of the above embodiments may be omitted. Also, at least part of the configuration of the above embodiments may be added to or substituted for the configuration of another of the above embodiments. [Explanation of symbols]

[0109] 1...voice data analysis device, 11...recording device, 11a...microphone, 11b...operation unit, 11c...control unit, 11d...memory unit, 11e...charging terminal, 11f...first communication terminal, 11g...clip unit, 11h...housing, 13...device attachment / detachment unit, 13a...power supply terminal, 13b...second communication terminal, 14...wired cable, 21...server device, 23...control unit, 25...memory unit, 27...communication unit, 41...display device, 43...mobile terminal device, 112...movable part, 113...fixed connecting portion, 121...recess, 121a...first bottom surface, 121b...contact terminal, 122...recess, 122a...first bottom surface, 122a...second bottom surface, 124...wall portion, 124a...groove portion.

Claims

1. A conversation sound data acquisition unit configured to acquire conversation sound data in which a conversation sound is recorded; A conversation text generating unit configured to generate conversation text data which is text data of a conversation content included in the conversation sound data; a conversation content analysis unit configured to analyze the conversation content using the conversation text data to obtain an analysis result; A voice data analysis device comprising:

2. 2. The voice data analysis apparatus according to claim 1, the conversation sound data acquisition unit is configured to acquire the conversation sound data from a recording device; The recording device is configured to be portable by the speaker and configured to record the speech sound data by recording the voice of the speaker. Audio data analysis equipment.

3. 3. The voice data analysis device according to claim 2, the conversation sound data acquisition unit is configured to removably mount the recording device, and is configured to acquire the conversation sound data in response to the recording device being mounted. Audio data analysis equipment.

4. 4. The voice data analysis device according to claim 3, The conversation sound data acquisition unit is configured to charge the attached recording device. Audio data analysis equipment.

5. 3. The voice data analysis device according to claim 2, The conversation sound data acquisition unit is configured to acquire the conversation sound data from the recording device via wireless communication. Audio data analysis equipment.

6. 6. The voice data analysis device according to claim 5, The conversation sound data acquisition unit is configured to acquire the conversation sound data in response to receiving a data transmission command from the recording device. Audio data analysis equipment.

7. 7. The voice data analysis device according to claim 6, the recording device is configured to record the conversation sound data or record audio data including the conversation sound data and background sound data; The conversation sound data acquisition unit includes: a voice data acquisition unit configured to acquire the conversation sound data or the voice data; a conversation sound data extraction unit configured to extract the conversation sound data from the voice data; Equipped with Audio data analysis equipment.

8. Obtaining conversation sound data in which the voice of a conversation is recorded; generating conversation text data which is text data of the conversation content included in the conversation sound data; Analyzing the content of the conversation using the conversation text data to generate analysis information; A voice data analysis method comprising:

9. 9. The method for analyzing voice data according to claim 8, The acquiring of the conversation sound data includes: acquiring the conversation sound data or audio data including the conversation sound data and background sound data; extracting the conversation sound data from the voice data; Equipped with Methods for analyzing audio data.

10. A computer program for causing a computer to function as at least one of the conversation sound data acquisition unit, the conversation text generation unit, and the conversation content analysis unit in the voice data analysis device according to any one of claims 1 to 7.

Images