Information processing program, information processing device and information processing method
The information processing device addresses the challenge of noisy audio in electronic documents by extracting and filtering speech data for each speaker, thereby reducing the workload of registration and reproduction.
Patent Information
- Application Number
- JP2023085051
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-24
- Publication Date
- 2024-12-05
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Audio attached to electronic documents often includes noise and sounds from the surrounding environment, increasing the workload of acquiring the desired audio, and removing these noises during registration further complicates the process.
An information processing device extracts speech data for each speaker from linked audio, generates statistical data, and narrows down output candidates based on this data to output only significant speech data when referring to the electronic document.
Reduces the workload associated with registering and reproducing audio data by eliminating unnecessary speech data and focusing on valuable audio segments.
Smart Images

Figure 2024168410000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing program, an information processing device, and an information processing method. [Background technology]
[0002] A document processing method has been proposed in which, when an annotation including audio or the like attached to an arbitrary position on an electronic document instructs the performance of work, work instruction information indicating that the annotation instructs the performance of work is obtained as a work sub-attribute, and the work instruction information is registered for each corresponding annotation by an annotation management server. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2006-004361 A Summary of the Invention [Problem to be solved by the invention]
[0004] The voice attached to the electronic document may contain noise or sounds from the surrounding environment, which may increase the workload of acquiring the desired voice. In addition, removing noise or sounds from the surrounding environment from the voice to be registered when attached to the electronic document may increase the workload at the time of registration and may impose restrictions on the registration environment.
[0005] In one aspect, the present invention aims to reduce the workload associated with registering and playing back voice data attached to electronic documents. [Means for solving the problem]
[0006] In order to achieve the above object, an information processing device as described below is provided. The information processing device has a processing unit that extracts speech data in units of utterances for each speaker from voice data linked to an electronic document, generates statistical data by statistically processing the speech data for each speaker, narrows down output candidates for the speech data when referring to the electronic document based on the statistical data, and enables the speech data that is the output candidate when referring to the electronic document to be output from a sound output unit. Effect of the Invention
[0007] According to one aspect, it is possible to reduce the workload associated with registering and reproducing voice data attached to an electronic document. [Brief description of the drawings]
[0008] [Figure 1] FIG. 1 illustrates an example of an information processing apparatus according to a first embodiment. [Diagram 2] FIG. 13 illustrates an example of a process management system according to a second embodiment. [Diagram 3] FIG. 13 illustrates an example of a hardware configuration of a server according to a second embodiment. [Figure 4] FIG. 11 illustrates an example of functional blocks of a server according to a second embodiment; [Diagram 5] FIG. 11 is a diagram illustrating an example of distribution of production instruction sheets according to the second embodiment. [Figure 6] FIG. 13 is a diagram illustrating an example of a special note registration screen according to the second embodiment. [Figure 7] FIG. 11 is a diagram illustrating an example of a speaker confirmation screen according to the second embodiment. [Figure 8] FIG. 13 is a diagram illustrating an example of a special note confirmation screen according to the second embodiment. [Figure 9] FIG. 11 illustrates an example of a flowchart of a voice accumulation process according to the second embodiment. [Figure 10] FIG. 11 is a diagram illustrating an example of voice accumulation data according to the second embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of utterance accumulation data according to the second embodiment. [Figure 12]FIG. 11 is a diagram illustrating an example of speaker data according to the second embodiment. [Figure 13] FIG. 11 illustrates an example of a flowchart of a statistical data update process according to the second embodiment. [Figure 14] FIG. 11 is a diagram illustrating an example of speaker statistical data according to the second embodiment. [Figure 15] FIG. 13 is a diagram showing a modified example of speaker statistical data according to the second embodiment. [Figure 16] FIG. 11 illustrates an example of a flowchart of speech data confirmation processing according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, the present embodiment will be described with reference to the drawings. Note that each embodiment can be implemented in combination with a plurality of other embodiments within a range that does not cause inconsistency. [First embodiment] First, the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of an information processing device of the first embodiment. The information processing device 10 extracts speech data from voice data 15 linked to an electronic document 14, and enables speech data selected from the extracted speech data according to a predetermined criterion to be output when the electronic document is referred to.
[0010] The electronic document 14 is a document that has been digitized to allow for the addition of special notes, such as a manual, a production instruction sheet, etc. The special notes are recorded in association with the electronic document 14, and can be output in whole or in part together with the electronic document 14 when the electronic document 14 is output. The special notes may be any information recorded in association with the electronic document 14, such as a pre-prepared fixed phrase, a free-form text input (free input), a voice input, etc.
[0011] The information processing device 10 in the first embodiment handles an electronic document 14 to which voice data 15 is linked as a special note. The voice data 15 includes an utterance (message) of a speaker (a person who inputs a special note). The number of utterances included in the voice data 15 is not limited to one, and may be two or more. The number of speakers in the voice data 15 is not limited to one, and may be two or more. The voice data 15 may include noise because recording in a good recording environment is not necessarily guaranteed, and may include speech from a third party who happens to be present in the recording environment.
[0012] The information processing device 10 is a so-called computer, and includes a processing unit 11 that performs information processing. The processing unit 11 controls the information processing device 10 and is capable of executing required processing. The processing unit 11 is, for example, a processor or an arithmetic circuit included in the information processing device 10.
[0013] The processing unit 11 extracts utterance data on an utterance basis from the voice data 15, identifies the speaker for each utterance data, and records the utterance data in the accumulated data 12. For example, the processing unit 11 extracts utterance data 16, 17, and 18 from the voice data 15. The processing unit 11 identifies the speaker 19 and records the utterance data 16 in the accumulated data 12, identifies the speaker 20 and records the utterance data 17 in the accumulated data 12, and identifies the speaker 21 and records the utterance data 18 in the accumulated data 12. The storage unit that records (stores) the accumulated data 12 corresponds to the utterance data storage unit.
[0014] The processing unit 11 calculates the speech rate (occupancy rate) of each speaker in the voice data 15 and the average volume of each speaker, and records them in the statistical data 13. For example, the processing unit 11 calculates the speech rate and average volume of each speaker 19, 20, 21 from the speech data 16, 17, 18 recorded in the accumulated data 12, and records them in the statistical data 13. The storage unit that records (stores) the statistical data 13 corresponds to the statistical data storage unit. The statistical data 13 is not limited to speech data extracted from one electronic document, and may be data obtained by statistically processing speech data extracted from two or more electronic documents.
[0015] The processing unit 11 narrows down the speakers 19, 20, and 21 to the speakers 19 and 21 as significant speakers in the voice data 15 based on the statistical data 13. The processing unit 11 selects the utterance data 16 of the speaker 19 and the utterance data 18 of the speaker 21 as output candidates as significant utterance data as special notes. In other words, the processing unit 11 can eliminate the utterance data 17 of low value in the voice data 15 from the speakers 19, 20, and 21 based on the statistical data 13. For example, the processing unit 11 can determine from the statistical data 13 that utterance data with a high speech rate (e.g., utterance data 16 and 18) is related to utterances that are highly valuable as special notes, and can determine that utterance data with a low speech rate (e.g., utterance data 17) is related to utterances that are less valuable as special notes. In addition, the processing unit 11 can determine from the statistical data 13 that speech data with a high average volume (for example, speech data 16, 18) relates to speech that is valuable as a special note, and can determine that speech data with a low average volume (for example, speech data 17) relates to speech that is less valuable as a special note.
[0016] The processing unit 11 links the speech data 16, 18 selected as output candidates to the electronic document 14, and enables the speech data 16, 18 to be output from a sound output device (such as a speaker) when the electronic document 14 is referenced.
[0017] The voice data 15 may be recorded by a terminal device (not shown) or by the information processing device 10. The electronic document 14 may be referenced by a terminal device (not shown) and the speech data 16, 18 may be reproduced by a sound output device included in the terminal device. The information processing device 10 may be equipped with a display device to enable the electronic document 14 to be displayed and output, and may be equipped with a sound output device to enable the speech data 16, 18 to be output as sound.
[0018] In this way, the information processing device 10 can eliminate the speech data 17 that is not useful as a special note when referring to the electronic document 14, and can output the speech data 16, 18 that are useful as a special note by voice. Such an information processing device 10 can ease the environmental conditions when recording voice as a special note of the electronic document 14. For example, the information processing device 10 can handle voice data that has been recorded in an environment where a third party other than the speaker related to the special note is present. Furthermore, instead of outputting the voice data 15 as a special note by voice when referring to the electronic document 14, the information processing device 10 can eliminate the speech data 17 and output the speech data 16, 18 by voice, thereby reducing the workload of the reader (user) of the electronic document 14.
[0019] Such an information processing device 10 can reduce the workload associated with registering and reproducing voice data attached to an electronic document. Second Embodiment Next, a second embodiment will be described. The second embodiment relates to a process management system that enables the recording and playback of audio that is linked to a production instruction sheet sent from a production management department to a factory (production department) as an electronic document and that is linked to the production instruction sheet as a special note.
[0020] First, the system configuration of the process control system will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of a process control system of the second embodiment. The process control system 50 has a network 30, a server 100 connected to the network 30, and terminals 200(a), 200(b), 200(c), ... The network 30 is, for example, an intranet of a company that has the process control system 50.
[0021] Terminals 200(a), 200(b), 200(c),... are computers capable of registering (inputting) special notes in a production instruction sheet, or computers capable of confirming (outputting) special notes from a production instruction sheet. Terminals 200(a), 200(b), 200(c),... are PCs (Personal Computers) that are installed in a production management department that creates production instructions or in a factory that confirms production instructions and are operated by a person in charge. Alternatively, terminals 200(a), 200(b), 200(c),... are mobile terminals that are carried and operated by a person in charge assigned to the production management department or the factory.
[0022] A terminal 200 that allows voice to be registered as a special note is connected to a sound input device (microphone), and a terminal 200 that allows voice to be confirmed as a special note is connected to a sound output device (speaker).
[0023] The server 100 enables the speech data extracted from the voice data registered as special notes in the production instruction sheet to be confirmed as special notes in the production instruction sheet. The server 100 in the second embodiment is an information processing device corresponding to the information processing device 10 in the first embodiment. The server 100 may have the special notes registration function and confirmation function that the terminal 200 has.
[0024] Next, the hardware configuration of the server 100 will be described with reference to FIG. 3. FIG. 3 is a diagram showing an example of the hardware configuration of the server of the second embodiment. The server 100 is entirely controlled by a processor 101. A memory 102 and a plurality of peripheral devices are connected to the processor 101 via a bus 114. The processor 101 may be a multiprocessor. The processor 101 is, for example, a central processing unit (CPU), a micro processing unit (MPU), or a digital signal processor (DSP). At least a part of the functions realized by the processor 101 executing a program may be realized by an electronic circuit such as an application specific integrated circuit (ASIC) or a programmable logic device (PLD).
[0025] The memory 102 is used as a main storage device of the server 100. The memory 102 temporarily stores at least a part of the OS (Operating System) program and application programs to be executed by the processor 101. The memory 102 also stores various data used for processing by the processor 101. As the memory 102, for example, a volatile semiconductor storage device such as a RAM (Random Access Memory) is used.
[0026] The peripheral devices connected to the bus 114 include a storage device 103, a GPU (Graphics Processing Unit) 104, an input interface 105, and an optical drive device 106. The peripheral devices connected to the bus 114 also include a device connection interface 107 and a network interface 108.
[0027] The storage device 103 electrically or magnetically writes and reads data to and from a built-in recording medium. The storage device 103 is used as an auxiliary storage device for a computer. The storage device 103 stores an OS program, application programs, and various data. Note that, for example, a hard disk drive (HDD) or a solid state drive (SSD) can be used as the storage device 103.
[0028] The GPU 104 is connected to a monitor 31. The GPU 104 displays an image on the screen of the monitor 31 in accordance with an instruction from the processor 101. The monitor 31 may be a display device using an organic EL (Electro Luminescence) display device, a liquid crystal display device, or the like.
[0029] The input interface 105 is connected to the keyboard 32 and the mouse 33. The input interface 105 transmits signals sent from the keyboard 32 and the mouse 33 to the processor 101. Note that the mouse 33 is an example of a pointing device, and other pointing devices can also be used. Examples of other pointing devices include a touch panel, a tablet, a touch pad, a trackball, etc.
[0030] The optical drive device 106 uses a laser beam or the like to read data recorded on the optical disc 34. The optical disc 34 is a portable recording medium on which data is recorded so that it can be read by reflection of light. The optical disc 34 includes a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc Read Only Memory), a CD-R (Recordable) / RW (ReWritable), etc.
[0031] The device connection interface 107 is a communication interface for connecting peripheral devices to the server 100. For example, a memory device 35 or a memory reader / writer 36 can be connected to the device connection interface 107. The memory device 35 is a recording medium equipped with a function for communicating with the device connection interface 107. The memory reader / writer 36 is a device for writing data to the memory card 37 or reading data from the memory card 37. The memory card 37 is a card-type recording medium.
[0032] The network interface 108 is connected to the network 30. The network interface 108 transmits and receives data via the network 30 to and from other computers or communication devices.
[0033] The server 100 can realize the processing function of the second embodiment by the above hardware configuration. The information processing device 10 shown in the first embodiment can also be realized by the same hardware as the server 100 shown in FIG. 3. The terminals 200(a), 200(b), 200(c),... can also be realized by the same hardware as the server 100. The terminals 200(a), 200(b), 200(c),... can be configured to allow a microphone or a speaker to be connected to the device connection interface 107. The processor 101 is an example of the processing unit 11 shown in the first embodiment.
[0034] The server 100 realizes the processing function of the second embodiment by, for example, executing a program recorded in a computer-readable recording medium. The program describing the processing contents to be executed by the server 100 can be recorded in various recording media. For example, the program to be executed by the server 100 can be stored in the storage device 103. The processor 101 loads at least a part of the program in the storage device 103 into the memory 102 and executes the program. The program to be executed by the server 100 can also be recorded in a portable recording medium such as the optical disk 34, the memory device 35, or the memory card 37. The program stored in the portable recording medium becomes executable after being installed in the storage device 103 under the control of the processor 101, for example. The processor 101 can also directly read and execute the program from the portable recording medium.
[0035] Next, the function of the server 100 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of functional blocks of the server of the second embodiment. The server 100 has a processing unit 120 and a storage unit 130. The processing unit 120 can be realized by the processor 101 executing a program describing required processing contents, and includes a voice processing unit 121, a statistical processing unit 122, and an output candidate generating unit 123. The storage unit 130 can be realized by the storage device 103, and can store data including accumulated data 131 and statistical data 132.
[0036] The voice processing unit 121 removes noise from the voice data recorded by the terminal 200, extracts speech data on an utterance-by-utterance basis from the noise-removed voice data, and records the extracted speech data in the stored data 131. Note that the speech data extracted from the speech data may be extracted on a speaker-by-speaker basis. For example, in the former case, one or more pieces of speech data are extracted for one speaker from the voice data, and in the latter case, one piece of speech data is extracted from one speaker.
[0037] The statistical processing unit 122 performs statistical processing on the speech data recorded in the accumulated data 131 for each speaker, and records the processed data in the statistical data 132. The statistical data 132 includes speech rate and average volume. The collection range of the statistical data 132 may be the voice data of one production instruction sheet, or may be the voice data of two or more production instructions. When the collection range of the statistical data is the voice data of two or more production instructions, the collection range can be the voice data of production instructions belonging to a predetermined group. Note that the predetermined group is desirably divided according to the attributes of the production instructions (for example, the recipient of instructions, the source of instructions, the time of creation, etc.). Such grouping can make the statistical data more meaningful.
[0038] The output candidate generation unit 123 generates, as output candidates, utterance data that is significant as an output target from among the utterance data linked to the production instruction sheet as special notes. The output candidate generation unit 123 selects, as significant utterance data, utterance data of a speaker having statistical data according to a predetermined criterion.
[0039] Next, a registration scene and a confirmation scene of special notes linked to a production instruction sheet will be described with reference to a distribution example of a production instruction sheet shown in Fig. 5. Fig. 5 is a diagram showing a distribution example of a production instruction sheet according to the second embodiment.
[0040] Production instructions are created by the production management department after receiving a case ID that can identify the production instructions and the product design drawings from the design department. In addition to basic product information (product name, model, etc.), production instructions also include the reason for creating the production instructions (model change, specification changes such as cost reduction and defect response), start conditions (immediately, as soon as the production line is free, specified date and time, etc.), start deadline, completion deadline, and special notes. Special notes are items that are not bound by the standard form of production instructions, and for example, precautions and restrictions during manufacturing are registered as special notes.
[0041] The production control department distributes the design drawings and production instructions for the product to the factory that produces the product. The production control department also distributes the design drawings for the product, and the design drawings and production instructions for the secondary products to the factories that will be the destinations for the product and that will produce secondary products using the product as a part.
[0042] For example, the design department creates design drawing A for product A, design drawing B for product B, and design drawing C for product C. When factory B produces product B, the production control department distributes production instruction B and design drawing B to factory B. When factory C produces product C, the production control department distributes production instruction C and design drawing C to factory C. When factory C produces an assembly (secondary product) whose components (primary product) are products A and B, the production control department distributes design drawings A and B in addition to production instruction C and design drawing C to factory C.
[0043] Production instructions are distributed as electronic documents, and audio can be added (linked) as special notes. The audio to be added to the production instructions may be recorded in the production management department, or it may be recorded in the corresponding factory. For example, the recording environment for the audio to be added to the production instructions may be good in the production management department, but it may not be easy to make it good in the factory. Therefore, the audio to be added to the production instructions may contain noise, may contain the voice of a third party, and may not be just the intended voice.
[0044] Next, a special note registration screen displayed on the terminal 200 when special notes are registered in the production instruction sheet will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the special note registration screen in the second embodiment.
[0045] The special notes registration screen is displayed when the person in charge selects the special notes registration menu when the production instruction sheet is displayed. The special notes registration screen includes a case ID display, a selection input display for the special notes registration method, a recording operation display, a recording time display, and a confirmation / back operation display.
[0046] The case ID display is information that can identify the production instruction sheet to which the special notes are linked. The selective input display for the method of registering special notes has a UI (User Interface) of a multiple-choice input form, for example, radio buttons, and allows the selection of whether the special notes are to be entered by voice, free input, or fixed phrase. The recording operation display, recording time display, and speaker selection display are displayed when voice input is selected in the selective input display. Note that free input allows the input of text, images, etc. The fixed phrase allows the selection of one of the fixed phrases prepared in advance, or a combination of two or more fixed phrases.
[0047] The recording operation display shows icons for the start, pause, and end of recording operations. The recording time display shows the recording time of the audio recorded by the start, pause, and end of recording operations, as well as an icon for the audio playback operation. The confirm / back operation display shows icons that accept the operation of whether or not to mark the recorded audio as a special note.
[0048] The terminal 200 allows voice data linked to a production instruction sheet to be registered as a special note using such a special note registration screen. The terminal 200 can notify the server 100 of the voice data linked to the production instruction sheet. The server 100 can extract speech data in units of utterances from the acquired voice data, identify the speaker, and generate statistical data 132 for each speaker. The server 100 can narrow down the speech data to be played back from the voice data registered as special notes in the production instruction sheet based on the statistical data 132, and set it as an output candidate. The server 100 can narrow down the speech data by narrowing down the speakers.
[0049] The terminal 200 enables the output candidates generated by the server 100 to be confirmed and changed as necessary. Next, a speaker confirmation screen displayed on the terminal 200 when confirming the output candidates generated by the server 100 will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the speaker confirmation screen of the second embodiment.
[0050] After registering the voice data as a special note on the special note registration screen, the terminal 200 displays a speaker confirmation screen. The speaker confirmation screen includes a case ID display, a statistical category display, a judgment criterion display, a playback target display, a non-playback target display, a speaker change operation display, and a confirmation / back operation display.
[0051] The case ID display is information that can identify the production instruction sheet to which the special notes are linked. The statistical category display specifies the range of statistical data 132 used to narrow down the speakers. For example, the range of statistical data 132 can be all of the accumulated data 131, or a limited portion of the accumulated data 131 for a limited period such as within the past year, or a limited portion of the attributes of the production instruction sheet identified from the case ID. The attributes of the production instruction sheet include the product type and the production factory. The terminal 200 notifies the server 100 of the received statistical category. The server 100 performs statistical processing according to the received statistical category.
[0052] The judgment criteria display is a display for specifying the narrowing judgment criteria of the statistical data 132 used to narrow down the speakers. The judgment criteria display includes a speech rate designation display and a volume designation display. The terminal 200 notifies the server 100 of the accepted statistical category. The server 100 narrows down the speakers according to the accepted speech rate and volume. The speech rate, which is one of the judgment criteria, allows the reference value to be selected from a pull tab. The judgment criteria display shown in the figure uses a speech rate of 80% or more as one of the judgment criteria. Furthermore, the volume, which is also one of the judgment criteria, allows the reference value to be selected from a radio button. The judgment criteria display shown in the figure uses a volume of "medium" or more as one of the judgment criteria.
[0053] The speech rate is the rate at which the total time of speech data for each speaker occupies the recording time of the voice data. The speech rate is one of the rate information related to the rate at which speech data for each speaker occupies the voice data. The speech rate may be calculated by other calculation formulas as long as it measures the importance of the speaker appearing in the voice data on a time scale. The volume is the average volume (dB) of the speech data for each speaker. The volume is one of the volume information related to the volume of the speech data for each speaker. The volume may be a relative evaluation by comparing the voice data and the speech data, such as whether the average volume of the speech data is larger, the same, or smaller than the average volume of the voice data. Such a relative evaluation of the volume can evaluate the volume of the speaker regardless of the recording environment. In addition, such a relative evaluation of the volume can simplify the normalization process of the voice data (for example, correction of the average volume).
[0054] The playback target display and non-playback target display are displays of a list of speakers included in the voice data linked to the production instruction sheet. The playback target display is a display of a list of speakers who meet the narrowing down criteria of the statistical data 132 and are to be played back, and the non-playback target display is a display of a list of speakers who do not meet the narrowing down criteria of the statistical data 132 and are therefore not to be played back.
[0055] The playback target display and non-playback target display each include a radio button, an identification information display, a speech playback icon, and speaker statistics information for each speaker. The playback target display and non-playback target display can switch speakers between the playback target display and non-playback target display by checking the radio button and operating the speaker switching operation display, which is an arrow-shaped icon. The confirm / return operation display displays an icon that accepts an operation to confirm or not confirm the configuration of speakers in the playback target display and non-playback display. If there is a change in the configuration of speakers in the playback target display and non-playback display, the terminal 200 notifies the server 100 of the change.
[0056] The speaker displayed in the playback target display is a speaker who the server 100 determines to meet the criteria, and the non-playback target display is a speaker who the server 100 determines not to meet the criteria. For example, if the operator of the terminal 200 wants to play back the speech of speaker C in the non-playback target display and set it as a special note to be played back, the operator can check the radio button for speaker C and operate the speaker replacement operation display to replace speaker C with the playback target display. In the same way, the operator displayed in the playback target display can be replaced with the non-playback target display. The terminal 200 notifies the server 100 of the speaker replacement result, and the server 100 updates the speech data to be played back, linked to the production instruction sheet, based on the speaker replacement result.
[0057] Next, a special note confirmation screen displayed on the terminal 200 when checking the special note from the terminal 200 will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of the special note confirmation screen of the second embodiment.
[0058] The special notes confirmation screen is displayed when the person in charge selects the special notes confirmation menu when the production instruction sheet is displayed. The special notes confirmation screen includes a case ID display, a selection input display for the special notes confirmation method, a speaker display, a playback operation display, a playback time display, and a back operation display.
[0059] The case ID display is information that can identify the production instruction sheet to which the special notes are linked. The selection input display for the confirmation method of special notes has a UI of a selection-type input form, for example, a radio button, and allows the selection of whether the special notes to be confirmed are to be output as voice, displayed freely, or displayed as a fixed phrase. Items that are registered as special notes are displayed in black, and items that are not registered are displayed in gray, to clearly indicate whether they are registered or not.
[0060] The speaker display displays the speaker that corresponds to the speech data that is linked to the production instruction sheet and is to be played back. The speaker displayed here is the speaker that is displayed in the speaker display on the speaker confirmation screen. The speaker display includes a radio button, an identification information display, a speech playback display, and speaker statistical information for each speaker. The speech playback display displays icons for the start playback and pause operations. The playback time display displays the playback time of the audio that is played back by the start playback and pause operations. The back operation display displays an icon that accepts an operation to end the special notes confirmation. Note that when one speaker has two or more speech data, they may be played back continuously as a series of speech data, or each may be played back individually.
[0061] As a result, the server 100 does not need to request confirmation of all of the voice data when checking the special notes from the terminal 200. Also, the server 100 allows the confirmation of appropriate utterance data without requiring selection of utterance data each time when checking the special notes from the terminal 200.
[0062] Next, the process executed by the server 100 will be described. First, the voice accumulation process executed by the server 100 when the terminal 200 acquires voice data as a special note will be described with reference to Figs. 9 to 12. Fig. 9 is a diagram showing an example of a flowchart of the voice accumulation process of the second embodiment. Fig. 10 is a diagram showing an example of voice accumulation data of the second embodiment. Fig. 11 is a diagram showing an example of utterance accumulation data of the second embodiment. Fig. 12 is a diagram showing an example of speaker data of the second embodiment.
[0063] [Step S11] The voice processing unit 121 (processing unit 120) acquires voice data registered as special notes by the terminal 200. The voice processing unit 121 stores the acquired voice data in the storage unit 130 as voice accumulation data among the accumulated data 131.
[0064] The voice accumulation data 140 shown in FIG. 10 is an example of voice accumulation data. The voice accumulation data 140 includes a case ID, a voice recording time, a volume, and a date for each voice data. The voice processing unit 121 assigns unique identification information (for example, identification information "V-xxxx") to the voice data. For example, the voice data identified by the identification information "V-xxxx" makes it possible to identify a production instruction sheet linked by the case ID "XXXX-xxxx". In addition, the voice data identified by the identification information "V-xxxx" is stored in the storage unit 130 as voice accumulation data together with the voice recording time "2'54", the volume "50", and the date (date of registration as a special note) "y1 / m1 / d1". In addition, the voice processing unit 121 stores the voice data in the storage unit 130 as voice accumulation data every time it acquires the voice data.
[0065] [Step S12] The audio processor 121 (processor 120) removes noise from the acquired audio data. In addition to removing noise, the audio processor 121 may perform normalization processing to correct the average volume to a specified value. The audio processor 121 may also perform filtering processing to make speech easier to hear. The audio processor may also delete silent parts that exceed a specified length.
[0066] [Step S13] The voice processing unit 121 (processing unit 120) extracts speech data in utterance units from the processed voice data. Note that the speech units may be separated by a change of speaker, or may be separated by a silent period by the same speaker lasting for a specified time or longer. The speech units may also be separated by other known methods. The voice processing unit 121 stores the extracted utterance data in the storage unit 130 as utterance accumulated data of the accumulated data 131.
[0067] The speech accumulation data 150 shown in FIG. 11 is an example of speech accumulation data. The speech accumulation data 150 includes, for each piece of speech data, a case ID, a speaker, a speech time, a volume, a playback flag, a date, and a voice ID. The playback flag is information that can identify whether the speech data is to be output (played back), and has an initial value of "0" and becomes "1" when the speech data becomes a speaker candidate (playback candidate) in the special notes. The date is the date of registration as a special note. The voice ID is information that can identify the voice data to be extracted.
[0068] The speech data is assigned unique identification information (for example, identification information "v000001") by the voice processing unit 121. For example, the speech data identified by the identification information "v000001" makes it possible to identify a production instruction sheet linked by the case ID "XXXX-xxxx". Furthermore, the speech data identified by the identification information "v000001" is stored in the memory unit 130 as speech accumulation data together with the speaker "A", the speech time "2'36", the volume "82", the playback flag "0", the date "y1 / m1 / d1", and the voice ID "V-xxxx".
[0069] The speech data extracted from the speech accumulation data 150 using the voice ID "V-xxxx" is the speech data identified by the identification information "v000001", "v000002", and "v000003". The remaining speech data is the speech data extracted before the voice ID "V-xxxx". In this way, the voice processing unit 121 stores the extracted speech data in the memory unit 130 as speech accumulation data each time it extracts the speech data.
[0070] [Step S14] The voice processing unit 121 (processing unit 120) identifies the speaker of the extracted speech data. First, the voice processing unit 121 distinguishes the speaker for each piece of speech data using information that can distinguish the speaker. If there is any identifying information that can identify the speaker for each distinguished speaker, the voice processing unit 121 assigns the identifying information to the distinguished speakers.
[0071] In addition, information that can distinguish a speaker may be based on voice characteristics (e.g., voiceprint, vocal range, intonation, etc.), speaker information included in the utterance (e.g., affiliation, position, name, etc.), or a combination of these.
[0072] The specific information that can identify the speaker can be information that can be displayed instead of the identification information that is added when distinguishing the speaker. For example, the identification information can be information that does not easily recall a specific person, such as A, B, C, D, etc., and the specific information can be information that easily recalls a specific person, such as "Manager of Section XX, Production Management Department."
[0073] Speaker data 152 shown in FIG. 12 is an example of speaker data. Speaker data 152 is a correspondence relationship between identification information that can distinguish a speaker and specific information that can specify a speaker. Examples of identification information that can distinguish a speaker are A, B, C, D, and E. Speaker A is specified as belonging to "Production Management Department," with a job title of "Section Manager" and a name of "Yamada," while speaker E is distinguished from other speakers but is not individually specified. For example, a new speaker is not individually specified, but after registration of special notes or playback, the speaker is specified by the speaker himself or someone other than the speaker and related information is input.
[0074] [Step S15] The statistical processing unit 122 (the processing unit 120) executes a statistical data update process to statistically process the utterance data. In the statistical data update process, the statistical processing unit 122 performs statistical processing on the speaker data, or on the voice data in addition to the speaker data, to generate and update the statistical data 132. The statistical data update process will be described later with reference to FIG. 13.
[0075] [Step S16] The output candidate generation unit 123 (processing unit 120) uses the statistical data 132 to generate speaker candidates for the utterance data to be played back as special notes in the production instruction sheet linked to the acquired voice data.
[0076] [Step S17] The output candidate generation unit 123 (processing unit 120) sets the playback flag "1" to the utterance data related to the speaker candidate among the utterance data to be played back as special notes in the production instruction sheet linked to the acquired voice data. For example, when speaker A in the case ID "XXXX-xxxx" is generated as a speaker candidate in the utterance accumulation data 150, the playback flag "0" in the utterance data "v000001" is updated to "1".
[0077] As a result, the production instruction sheet identified by the case ID "XXXX-xxxx" can play back the speech data "v000001" related to speaker A when playing back audio in the special notes.
[0078] Next, the statistical data update process executed by the server 100 will be described with reference to Fig. 13. The statistical data update process is a process executed by the server 100 in step S15 of the voice accumulation process. Fig. 13 is a diagram showing an example of a flowchart of the statistical data update process of the second embodiment.
[0079] [Step S21] The statistical processing unit 122 (processing unit 120) selects the aggregation range in the statistical processing. The aggregation range may be selected by the user of the terminal 200, or may be determined by default. The aggregation range may be all of the accumulated data 131, or may be data related to production instructions having specific attributes among the accumulated data 131. For example, production instructions having specific attributes may be limited by a case ID, linked to audio data including the speech of a specific speaker, or linked to audio data recorded during a specific period.
[0080] [Step S22] The statistical processing unit 122 (processing unit 120) acquires data belonging to the selected aggregation range from the stored data 131 as data to be aggregated (target data).
[0081] [Step S23] The statistical processing unit 122 (processing unit 120) performs statistical processing on the target data for each speaker. Specifically, the statistical processing unit generates a speech rate and an average volume for each speaker for the target data.
[0082] <Step S24> The statistical processing unit 122 (processing unit 120) stores the generated statistical data as statistical data 132 in the storage unit 130. An example of the statistical data 132 will now be described with reference to Fig. 14. Fig. 14 is a diagram showing an example of speaker statistical data according to the second embodiment. Speaker statistical data 160 is one of the statistical data 132 stored in the storage unit 130.
[0083] The speaker statistical data 160 is statistical data with a collection range of "all" and includes the speech rate and average volume for each speaker. The speech rate is the proportion of speech data related to the speaker in the original voice data from which the data was extracted. The average volume is the average volume of the speech data and is generated in dB units. For example, speaker A has statistical data with a speech rate of "80(%)" and an average volume of "85(dB)". The statistical data for each speaker is used to generate speaker candidates. Such speaker statistical data 160 can generate statistical data related to speakers from a wider collection range, improving the accuracy of the statistical data.
[0084] Next, a modified example of the speaker statistical data 160 will be described with reference to Fig. 15. Fig. 15 is a diagram showing a modified example of the speaker statistical data of the second embodiment. The speaker statistical data 162 is one of the statistical data 132 stored in the storage unit 130.
[0085] The speaker statistical data 162 is statistical data with the aggregation range per production instruction sheet, and includes the speech rate and volume evaluation of each speaker by case ID. The speech rate is the ratio of speech data related to the speaker to the original voice data from which the data was extracted. The volume evaluation is the evaluation level of the average volume of the speech data, and is evaluated at three volume levels, for example, high, medium, and low. For example, speaker A in case ID "XXXX-xxxx" is statistical data with a speech rate of "80(%)" and a volume evaluation of "high". Also, speaker A in case ID "YYYY-yyyy" is statistical data with a speech rate of "5(%)" and a volume evaluation of "low". In other words, speaker A is the main speaker in the special notes of the production instruction sheet identified by the case ID "XXXX-xxxx", but is not the main speaker in the special notes of the production instruction sheet identified by the case ID "YYYY-yyyy". Statistical data for each case ID and speaker can also be used to generate speaker candidates. Such speaker statistical data 162 can generate statistical data related to speakers from a specific aggregation range, and can improve the accuracy of the statistical data in response to environmental changes such as when a speaker is transferred.
[0086] Next, the utterance data confirmation process executed by the server 100 will be described with reference to Fig. 16. The utterance data confirmation process is executed when the terminal 200 displays a speaker confirmation screen. Fig. 16 is a diagram illustrating an example of a flowchart of the utterance data confirmation process of the second embodiment.
[0087] [Step S31] When the terminal 200 displays the speaker confirmation screen, the output candidate generation unit 123 (processing unit 120) notifies the terminal 200 of all speakers related to the production instruction sheet and speakers to be played back (speakers whose playback flags are set to "1" in step S17 of the voice storage process). Upon receiving the notification, the terminal 200 displays the speakers to be played back in the playback target display (see FIG. 7), and displays the speakers not to be played back among all speakers related to the production instruction sheet in the non-playback target display.
[0088] [Step S32] When displaying a speaker confirmation screen from the terminal 200, the output candidate generation unit 123 (processing unit 120) determines whether to end confirmation of the utterance data based on a notification that determines whether confirmation of the configuration of speakers to be displayed for playback and those not to be displayed for playback has been completed. If confirmation of the utterance data has been completed, the output candidate generation unit 123 ends confirmation of the utterance data and proceeds to step S35, and if confirmation of the utterance data has not been completed, the output candidate generation unit 123 proceeds to step S33.
[0089] [Step S33] If there is an utterance playback instruction from the terminal 200, the output candidate generation unit 123 (processing unit 120) proceeds to step S34. If there is no utterance playback instruction, the output candidate generation unit 123 proceeds to step S32 and waits for the utterance playback instruction or completion of confirmation of the utterance data.
[0090] [Step S34] The output candidate generation unit 123 (processing unit 120) transmits, to the terminal 200, utterance data based on the utterance playback instruction. [Step S35] The output candidate generation unit 123 (processing unit 120) judges whether or not there is a notification of a change in the configuration of speakers displayed as playback targets and non-playback targets together with the notification of the end of confirmation of the utterance data, that is, whether or not there is a switch in the output candidates (playback candidates). If there is no switch in the output candidates, the output candidate generation unit 123 ends the utterance data confirmation process, and if there is a switch in the output candidates, the output candidate generation unit 123 proceeds to step S36.
[0091] [Step S36] The output candidate generation unit 123 (processing unit 120) updates the playback flag of the utterance data in accordance with the output candidate replacement notification. The output candidate generation unit 123 sets the playback flag of the utterance data to be played back to "1" and sets the playback flag of the utterance data not to be played back to "0". Note that the server 100 may hold update information of the playback flag based on the output candidate replacement notification, and may subsequently overwrite and update the setting of the playback flag based on the statistical data 132 based on the update information.
[0092] This allows the server 100 to present utterance data to be played back based on the statistical data 132, and then accept corrections by the user of the terminal 200. The server 100 also makes it possible to output appropriate utterance data when checking special notes. Such a server 100 can reduce the workload associated with registering and playing back utterance data linked to a production instruction sheet.
[0093] In addition, the disclosed embodiments are illustrative in all respects and should not be considered as limiting. In addition, the above-mentioned embodiments and each configuration of the modified examples may be combined and applied. The scope of the present invention is indicated by the claims, not the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]
[0094] 10. Information processing device 11 Processing section 12 Accumulated Data 13. Statistics 14 Electronic Documents 15 Audio data 16,17,18 Speech data 19, 20, 21 Speaker
Claims
1. On the computer, extracting speech data from the voice data linked to the electronic document in units of utterances for each speaker, generating statistical data by statistically processing the speech data for each speaker, narrowing down output candidates for the speech data when referring to the electronic document based on the statistical data, and making it possible to output the speech data that has become the output candidates when referring to the electronic document from a sound output unit; An information processing program that executes processing.
2. The process comprises: storing the speech data in a speech data storage unit in association with the electronic document; storing the statistical data in a statistical data storage unit in association with the speaker; The information processing program according to claim 1.
3. The process comprises: adding information capable of identifying whether the utterance data is to be output to the utterance data, and storing the utterance data in the utterance data storage unit; 3. The information processing program according to claim 2.
4. The statistical data is The speech data for each speaker includes ratio information regarding a ratio of the speech data to the audio data, and volume information regarding a volume of the speech data for each speaker.
3. The information processing program according to claim 2.
5. a processing unit that extracts speech data from the voice data linked to the electronic document in units of utterances for each speaker, generates statistical data by statistically processing the speech data for each speaker, narrows down output candidates for the speech data when referring to the electronic document based on the statistical data, and enables the speech data that is the output candidate when referring to the electronic document to be output from a sound output unit; An information processing device having the above configuration.
6. The computer extracting speech data from the voice data linked to the electronic document in units of utterances for each speaker, generating statistical data by statistically processing the speech data for each speaker, narrowing down output candidates for the speech data when referring to the electronic document based on the statistical data, and making it possible to output the speech data that has become the output candidates when referring to the electronic document from a sound output unit; Information processing methods.
Citation Information
Patent Citations
Document processing apparatus, document processing method, and document processing program
JP2006004361A
