A voice collection method and system for multi-factor collection room

Abstract

The present application provides a method and system for voice collection in a multi-factor collection room, which performs primary screening of the beam feedback voice of the microphone array through the location information of the sound source and the beam coverage of the microphone array. The optimal voice collection direction and the optimal voice information are obtained by comparing the sound intensity of the beam feedback voice. The optimal voice information is clarified to reduce the interference of noise and echo on the voice information, and clear voice information is obtained. The voiceprint characteristics of the person being collected and the person collecting are used as the distinction criteria, a prompt tone is set, and the clear voice information is classified and stored.

Description

Technical Field

[0001] The present application relates to the field of sound collection technology, and in particular to a voice collection method and system for a multi-factor collection room. Background Art

[0002] The multi-factor collection room is a place where collectors and collected persons communicate. Every collection conducted in the multi-factor collection room must be recorded in audio and video so that the information of the collected persons can be called up at any time when processing the collected content.

[0003] During the collection process, the voice information of the collector and the person being collected is usually recorded using a full-time recording device. The collector controls the on and off of the microphones and speakers inside and outside the multi-factor collection room. When the collector asks questions or the person being collected answers questions, the recording operation can be started and stopped by controlling the buttons of the recording device. This method realizes the collection of voice information of the collector and the person being collected, but does not record the on-site behavior and facial expressions of the collector and the person being collected.

[0004] In order to collect multiple features such as the voice, behavior, and expression of the person being collected, the sound collection sensor and the image collection sensor are integrated into the same collection device, and the control module controls the start and stop uniformly, so as to realize the comprehensive collection of the characteristics of the person being collected. However, when the sound collection information is stored, the voice information of the collector and the person being collected is not classified and stored. In the case of low sound resolution, it is difficult to distinguish the voice of the collector and the person being collected. Summary of the invention

[0005] The present application provides a voice collection method and system for a multi-factor collection room to solve the problem that the recorded sound resolution is low and it is difficult to distinguish the voice of the collector and the collected person.

[0006] The present application provides a voice collection method for a multi-factor collection room, the steps of which include:

[0007] Collect characteristic information of sound sources in a multi-factor acquisition room.

[0008] The characteristic information includes voice information and position information. The voice position information is used to preliminarily identify the location of the sound source, thereby filtering out some feedback beams of the microphone array whose sound collection effect is obviously affected by the sound source location. The voice information is used to further identify the optimal voice information collection direction.

[0009] The position information of the sound source is extracted from the feature information of the sound source.

[0010] By performing image acquisition on the collector and the person being collected, the position information of the collector and the person being collected is extracted from the image acquisition results to preliminarily confirm the voice collection direction.

[0011] The direction of the sound source is first positioned in combination with the sound source position information and the coverage of the microphone array beam for collecting voice information to obtain a suboptimal voice collection direction.

[0012] A microphone array beam is formed between every two sub-microphones in the microphone array, and each microphone array beam is regarded as feeding back the collected voice signal to the processor of the microphone array through straight-line propagation. The coverage of the beam is limited by the number of sub-microphones, so the coverage is limited. Combined with the position information of the collector and the collected person, it can be preliminarily determined that the voice information fed back by some beams is weak, and the voice information fed back by these beams can be cleared. And the voice collection direction corresponding to the remaining beams is positioned as the suboptimal voice collection direction.

[0013] According to the voice information of the suboptimal voice collection direction, the sound source direction is secondarily positioned to obtain the optimal voice collection direction.

[0014] The sound intensity of the voice information in the suboptimal voice collection direction is compared, a voice waveform is generated according to the voice information in the suboptimal voice collection direction, parameters such as amplitude and vibration are extracted from the voice waveform, and the sound intensity of the voice information in the suboptimal voice collection direction is calculated. The sound intensities of the voice information in the suboptimal voice collection direction are compared one by one, and the voice collection direction corresponding to the voice information with the largest sound intensity is set as the optimal voice collection direction.

[0015] The voice information collected in the optimal voice collection direction is clarified to obtain clarified voice information.

[0016] The voice information in the optimal voice collection direction also contains sound information such as noise and echo. In order to improve the resolution of the collected sound, the noise, echo and other noises are filtered and eliminated. This process is called clarification processing.

[0017] The clarified voice information is classified and stored based on the distinction between the person being collected and the collector.

[0018] It is used to distinguish the voice information of the person being collected and the collector, and classify and save the voices of each person. In addition, the voice information of the person being collected and the collector will still be saved in another storage space in the form of dialogue for future comparison.

[0019] Optionally, the characteristic information of the sound source collected in the multi-factor collection room includes: position information of the person being collected, position information of the collector, voice information of the person being collected, and voice information of the collector.

[0020] The position information of the person being collected and the position information of the collector are used for primary positioning of the direction of the sound source. The voice information of the person being collected and the voice information of the collector are used for secondary positioning of the direction of the sound source. The primary positioning method is to perform primary positioning of the sound source in combination with the position information and the coverage range of the microphone array beam.

[0021] Optionally, the step of performing primary positioning on the direction of the sound source in combination with the position information of the sound source and the coverage of the microphone array beam for collecting voice information to obtain a suboptimal voice collection direction includes:

[0022] A sound source determination area is set according to the position information of the sound source.

[0023] The beam of the microphone array in a suboptimal speech collection direction is screened in combination with the sound source determination area and the coverage of the microphone array beam to obtain a suboptimal speech collection direction.

[0024] The sound source determination area is an area where sound waves emitted by the sound source are more concentrated. By extracting the part where the coverage range of the microphone array beam overlaps with the sound source determination area, a part of the microphone array beam feedback with better sound reception effect can be obtained. The microphone array speech collection direction where the microphone array beam feedback is located is determined to be a suboptimal speech collection direction.

[0025] Optionally, combining the sound source determination area and the coverage of the microphone array beam for collecting voice information, screening the beam of the microphone array in the suboptimal voice collection direction, and obtaining the suboptimal voice collection direction comprises:

[0026] If the coverage of the microphone array beam overlaps with the sound source determination area, the beam is determined to be a suboptimal voice collection beam, and the voice collection direction of the suboptimal voice collection beam is determined to be a suboptimal voice collection direction.

[0027] If the coverage range of the microphone array beam has no overlapping portion with the sound source determination area, the beam is determined to be an interference beam, and the voice information of the interference beam is not received.

[0028] The suboptimal voice collection direction is determined by combining the sound source determination area and the coverage of the microphone array beam, and the voice information in the suboptimal voice collection direction is secondarily positioned. The voices in the remaining collection directions are determined as interference beams and are not received, which reduces the amount of computation for voice data processing and improves the accuracy of voice collection.

[0029] Optionally, the step of performing secondary positioning on the direction of the sound source according to the voice information of the suboptimal voice collection direction to obtain the optimal voice collection direction includes:

[0030] A voice information waveform is generated according to the voice information data in the suboptimal voice collection direction.

[0031] The sound intensity of the voice information in the suboptimal voice collection direction is obtained according to the voice information waveform.

[0032] The sound intensities of the voice information in the suboptimal voice collection directions are compared one by one to obtain the optimal voice collection direction.

[0033] By extracting the amplitude, magnitude and other parameters from the voice information waveform, and then using the amplitude, magnitude and other parameters to calculate the sound intensity of the voice information, the value of the sound intensity is used to reflect the quality of the collected voice information. The larger the value of the sound intensity, the better the quality of the collected voice information.

[0034] Optionally, the step of comparing the sound intensities of the voice information in the suboptimal voice collection directions one by one to obtain the optimal voice collection direction includes:

[0035] The sound intensities of the speech information in the two suboptimal speech collection directions are extracted and compared, and the speech information with stronger sound intensity is marked as Class A speech information, and the speech information with weaker sound intensity is marked as Class B speech information.

[0036] The sound intensity of the A-level voice information is compared with the sound intensity of the voice information in the other suboptimal voice collection directions, and the voice information with the strongest sound is still marked as the A-level voice information.

[0037] The A-level voice information marked after the last sound intensity comparison is set as the optimal voice information, and the optimal voice collection direction is obtained.

[0038] There is only one Class A voice information in the process of comparing one by one, and the voice information with the strongest sound is the Class A voice information. In the comparison process, if the sound intensity of the Class A voice information is less than that of the comparison voice information, the comparison voice information becomes the new Class A voice information. The Class B voice information no longer participates in the sound intensity comparison. The Class A voice information obtained in the last comparison is determined as the optimal voice information.

[0039] Optionally, the step of performing clarity processing on the voice information collected in the optimal voice collection direction to obtain the clear voice information includes:

[0040] The class B voice information is attenuated.

[0041] The optimal voice information collected in the optimal voice collection direction is enhanced to obtain enhanced optimal voice information.

[0042] The enhanced optimal voice information is subjected to noise reduction processing to obtain stimulation-clarified voice information.

[0043] Echo cancellation is performed on the stimulus-clarified voice information to obtain clarified voice information.

[0044] The microphone array weights the speech feedback from each beam to synthesize the final speech information. By attenuating the Class B speech information and reducing the weight of the Class B speech during weighting, the final speech information is synthesized mainly based on the optimal speech information to a great extent, thereby improving the resolution of the sound. Noise reduction processing refers to the optimization processing on the waveform. The waveform is optimized and the burrs on the waveform are removed through software processing means such as filtering to improve the calculation accuracy. The multi-factor acquisition room is a closed space. Sound is prone to echo in the multi-factor acquisition room. Echo cancellation reduces the interference caused by echoes. The speech information that has been clarified by noise reduction, echo cancellation, etc. must eventually be classified and stored for retrieval.

[0045] Optionally, the step of classifying and storing the clarified voice information based on the person being collected and the collector as a distinguishing criterion includes:

[0046] A first prompt tone is set for the voice information of the collector.

[0047] A second prompt tone is set for the voice information of the person being collected.

[0048] The voice information of the collector and the collected person is classified and saved according to the first prompt tone and the second prompt tone.

[0049] The prompt tone is used to distinguish the voice clips of the person being collected and the person collecting the voice clips, so as to facilitate classification and storage. The first prompt tone and the second prompt tone use two completely different sounds to avoid confusion.

[0050] Optionally, the step of classifying and saving the voice information of the collector and the collected person according to the first prompt tone and the second prompt tone includes:

[0051] Set waiting time thresholds for the first prompt tone and the second prompt tone.

[0052] The voice information interval time of the collector and the voice information interval time of the collected person are recorded.

[0053] The first prompt tone or the second prompt tone is set in combination with the voice information interval time and the voiceprint feature.

[0054] The voice information including the first prompt tone is stored in the voice storage space of the collector.

[0055] The voice information including the second prompt tone is stored in the voice storage space of the person being collected.

[0056] The pause time between the speech of the collector or the collected person is used as a sign of determining the end of the narration, and the waiting time threshold is the maximum value of the pause time. If the pause time is greater than the waiting time threshold, it is preliminarily determined that the narration is over, but the voiceprint needs to be further verified to confirm that the pause time is not too long due to too long thinking time.

[0057] After confirming that the prompt tone is added correctly, the voice information of the collector and the collected person can be classified and saved according to the prompt tone.

[0058] Optionally, the step of setting the first prompt tone or the second prompt tone in combination with the voice information interval time and the voiceprint feature includes:

[0059] The voiceprint features of the collector and the collected person are collected and stored in a voice verification database.

[0060] If the voice information interval is greater than the waiting time threshold, the first prompt tone or the second prompt tone is set to perform voiceprint verification.

[0061] If the voiceprint verification result shows that the voiceprint features are consistent, the first prompt tone or the second prompt tone that is set is canceled.

[0062] If the voiceprint verification result shows that the voiceprint features are inconsistent, the first prompt tone or the second prompt tone that is set is retained.

[0063] The voiceprint feature check is to check whether the setting of the prompt tone is reasonable. The setting of the prompt tone is to distinguish the collector from the collected person, so the voice information before and after the prompt tone should have different voiceprint features. Therefore, if the voiceprint features before and after the prompt tone check are consistent, it means that the voice information before and after the prompt tone comes from the same person, and the prompt tone setting should be cancelled. Otherwise, the prompt tone setting should be retained.

[0064] The present application also provides a voice collection system for a multi-factor collection room, comprising: a data collection module, a data processing module and an execution module.

[0065] The data acquisition module is used to collect characteristic information of sound sources in the multi-factor acquisition room.

[0066] The data processing module is used to extract the location information of the sound source from the feature information of the sound source.

[0067] The data processing module is also used to combine the sound source position information and the beam coverage of the microphone array used to collect voice information, perform primary positioning on the direction of the sound source, and obtain a suboptimal voice collection direction.

[0068] The data processing module is also used to perform secondary positioning on the sound source direction according to the voice information of the suboptimal voice collection direction to obtain the optimal voice collection direction.

[0069] The data processing module is also used to perform clarity processing on the voice information collected in the optimal voice collection direction to obtain clear voice information.

[0070] The execution module is used to classify and store the clarified voice information based on the person being collected and the collector as a distinction standard.

[0071] The present application provides a method and system for voice collection in a multi-factor collection room, which performs primary screening of the beam feedback voice of the microphone array through the location information of the sound source and the beam coverage of the microphone array. The optimal voice collection direction and the optimal voice information are obtained by comparing the sound intensity of the beam feedback voice. The optimal voice information is clarified to reduce the interference of noise and echo on the voice information, and clear voice information is obtained. The voiceprint characteristics of the person being collected and the person collecting are used as the distinction criteria, a prompt tone is set, and the clear voice information is classified and stored. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0073] Figure 1 A schematic diagram of the steps of a voice collection method for a multi-factor collection room;

[0074] Figure 2 A schematic diagram of the steps for primary positioning of a sound source;

[0075] Figure 3 A schematic diagram of the steps for filtering suboptimal voice information;

[0076] Figure 4 A schematic diagram of the steps for secondary localization of a sound source;

[0077] Figure 5 A schematic diagram of the steps for comparing the sound intensity of the voice information in the suboptimal voice collection directions one by one to obtain the optimal voice collection direction;

[0078] Figure 6 A schematic diagram of the steps for extracting clear speech information;

[0079] Figure 7 A schematic diagram of the steps of classifying and storing the cleared voice information based on the person being collected and the collector as the distinguishing criteria;

[0080] Figure 8 A schematic diagram of the steps of classifying and saving the voice information of the collector and the collected person according to the first prompt tone and the second prompt tone;

[0081] Fig. 9 A schematic diagram of the steps of setting the first prompt tone or the second prompt tone in combination with the voice information interval time and the voiceprint feature. DETAILED DESCRIPTION

[0082] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0083] The present application provides a method for voice collection in a multi-factor collection room, which is applied to the sound source sound collection in the multi-factor collection room. The sound source position in the multi-factor collection room is located at the geometric center of the multi-factor collection room. The speaker in the multi-factor collection room is parallel to the sound source and is arranged at the window of the multi-factor collection room. The microphone array in the multi-factor collection room is perpendicular to the speaker and is arranged on the top of the multi-factor collection room. Figure 1 The detailed description includes the following steps:

[0084] Collect characteristic information of sound sources in a multi-factor acquisition room.

[0085] The characteristic information includes voice information and position information. The voice position information is used to preliminarily identify the location of the sound source, thereby filtering out some feedback beams of the microphone array whose sound collection effect is obviously affected by the sound source location. The voice information is used to further identify the optimal voice information collection direction.

[0086] The position information of the sound source is extracted from the feature information of the sound source.

[0087] By performing image acquisition on the collector and the person being collected, the position information of the collector and the person being collected is extracted from the image acquisition results to preliminarily confirm the voice collection direction.

[0088] The direction of the sound source is first positioned in combination with the sound source position information and the coverage of the microphone array beam for collecting voice information to obtain a suboptimal voice collection direction.

[0089] A microphone array beam is formed between every two sub-microphones in the microphone array, and each microphone array beam is regarded as feeding back the collected voice signal to the processor of the microphone array through straight-line propagation. The coverage of the beam is limited by the number of sub-microphones, so the coverage is limited. Combined with the position information of the collector and the collected person, it can be preliminarily determined that the voice information fed back by some beams is weak, and the voice information fed back by these beams can be cleared. And the voice collection direction corresponding to the remaining beams is positioned as the suboptimal voice collection direction.

[0090] According to the voice information of the suboptimal voice collection direction, the sound source direction is secondarily positioned to obtain the optimal voice collection direction.

[0091] The sound intensity of the voice information in the suboptimal voice collection direction is compared, a voice waveform is generated according to the voice information in the suboptimal voice collection direction, parameters such as amplitude and vibration are extracted from the voice waveform, and the sound intensity of the voice information in the suboptimal voice collection direction is calculated. The sound intensities of the voice information in the suboptimal voice collection direction are compared one by one, and the voice collection direction corresponding to the voice information with the largest sound intensity is set as the optimal voice collection direction.

[0092] The voice information collected in the optimal voice collection direction is clarified to obtain clarified voice information.

[0093] The voice information in the optimal voice collection direction also contains sound information such as noise and echo. In order to improve the resolution of the collected sound, the noise, echo and other noises are filtered and eliminated. This process is called clarification processing.

[0094] The clarified voice information is classified and stored based on the distinction between the person being collected and the collector.

[0095] It is used to distinguish the voice information of the person being collected and the collector, and classify and save the voices of each person. In addition, the voice information of the person being collected and the collector will still be saved in another storage space in the form of dialogue for future comparison.

[0096] Collecting characteristic information of sound sources in a multi-factor collection room is the basic work of voice collection in this application. The characteristic information of sound sources in a multi-factor collection room includes: location information of the person being collected, location information of the collector, voice information of the person being collected, and voice information of the collector.

[0097] The position information of the person being collected and the position information of the collector are combined with the beam coverage of the microphone array to perform preliminary screening of the beam feedback voice, exclude some poor beam feedback voice signals, and reduce the amount of data processing operations. The voice information of the person being collected and the voice information of the collector can provide voice samples for the voiceprint verification database, and the corresponding sound intensity can be calculated based on the voice information to screen the optimal voice information.

[0098] Let's combine Figure 2 The initial screening of beamfeedback speech is described in detail, and the steps include:

[0099] A sound source determination area is set according to the position information of the sound source.

[0100] The sound source determination area is the area where the sound energy emitted by the sound source is most concentrated. The collection beam of the microphone array should be the best choice to collect the sound information of the sound source determination area. Therefore, the sound source determination area provides a basis for judging the quality of the collected voice information.

[0101] The beam of the microphone array in a suboptimal speech collection direction is screened in combination with the sound source determination area and the coverage of the microphone array beam to obtain a suboptimal speech collection direction.

[0102] The microphone array beam collects the sound emitted by the sound source in a manner that is considered as a straight line collection. Obviously, when the collection beam overlaps with the sound source determination area, the quality of the collected voice information will be better. Therefore, based on whether the sound source determination area and the coverage of the microphone array beam overlap, the suboptimal voice collection direction is screened out and the suboptimal voice information is obtained.

[0103] Combine the following Figure 3 The process of preliminary screening of the voice information collected by the beam of the microphone array is described in detail, and the steps include:

[0104] If the coverage of the microphone array beam overlaps with the sound source determination area, the beam is determined to be a suboptimal voice collection beam, and the voice collection direction of the suboptimal voice collection beam is determined to be a suboptimal voice collection direction.

[0105] If the coverage range of the microphone array beam has no overlapping portion with the sound source determination area, the beam is determined to be an interference beam, and the voice information of the interference beam is not received.

[0106] After filtering out the suboptimal voice information, the voice information collected by the remaining beams can be deleted or not received to achieve the purpose of reducing the amount of data and save storage space. Then the suboptimal voice collection direction can be further filtered to obtain the optimal voice collection direction. This process is the secondary positioning process. Figure 4 The process of secondary positioning is described, and the steps include:

[0107] A voice information waveform is generated according to the voice information data in the suboptimal voice collection direction.

[0108] The voice information waveform is used to extract parameters such as amplitude and oscillation in the waveform, and the parameters are used to calculate the sound intensity of the corresponding voice information, providing a data basis for sound intensity comparison.

[0109] The sound intensity of the voice information in the suboptimal voice collection direction is obtained according to the voice information waveform.

[0110] The sound intensities of the voice information in the suboptimal voice collection directions are compared one by one to obtain the optimal voice collection direction.

[0111] Two voice information obtained from all suboptimal voice collection directions are selected for comparison, and the process is repeated until the voice information with the largest sound intensity is selected and determined as the optimal voice information. The corresponding collection direction is also determined as the optimal voice information collection direction.

[0112] Combine the following Figure 5 The sound intensity comparison process is described in detail, and the steps include:

[0113] The sound intensities of the speech information in the two suboptimal speech collection directions are extracted and compared, and the speech information with stronger sound intensity is marked as Class A speech information, and the speech information with weaker sound intensity is marked as Class B speech information.

[0114] There is only one Class A voice information in the process of sound intensity comparison, and there can be multiple Class B voice information at the same time, and the Class B voices are no longer compared with each other.

[0115] The sound intensity of the A-level voice information is compared with the sound intensity of the voice information in the other suboptimal voice collection directions, and the voice information with the strongest sound is still marked as the A-level voice information.

[0116] If the sound intensity of the Class A voice information is smaller than that of the comparison voice information in the comparison process, the comparison voice information becomes the new Class A voice information, and the Class A voice information before the comparison is determined to be Class B voice information.

[0117] The A-level voice information marked after the last sound intensity comparison is set as the optimal voice information, and the optimal voice collection direction is obtained.

[0118] Since the multi-factor acquisition room is a closed space, it is easy for sound to form echoes during propagation. From the perspective of signal, the voice information collected by the microphone array will have certain signal noise even if it is the best voice information. Therefore, it is necessary to perform clarification processing such as noise reduction and echo processing on the best voice information to obtain clear voice information.

[0119] Combine the following Figure 6 The process of performing the voice information clarification process is described in detail, and the steps include:

[0120] The Class B speech is attenuated.

[0121] The microphone array needs to weight the speech collected by its beam and then synthesize it into final speech information. Since the optimal speech information has been screened out, the Class B speech information should be attenuated to ensure that the optimal speech information obtains the maximum weight, thereby ensuring the quality of the collected speech.

[0122] The optimal voice information collected in the optimal voice collection direction is enhanced to obtain enhanced optimal voice information.

[0123] The enhancement processing of the optimal voice information is also for the purpose of making the optimal voice information obtain the maximum weight in the final voice synthesis process, thereby improving the resolution of the collected voice information.

[0124] The enhanced optimal voice information is subjected to noise reduction processing to obtain secondary clear voice information.

[0125] Echo cancellation is performed on the secondary clear voice information to obtain clear voice information.

[0126] The noise reduction process and the echo cancellation process are both for further improving the quality of the optimal voice information, which is beneficial to improving the resolution of the sound information and obtaining clear voice information.

[0127] After obtaining the clear voice information, the voice information of the person being collected and the collector can be classified and stored for future use. Figure 7 The process of classifying and storing the clear voice information is described in detail, and the steps include:

[0128] A first prompt tone is set for the voice information of the collector.

[0129] A second prompt tone is set for the voice information of the person being collected.

[0130] The first prompt tone is equivalent to the voice tag of the collector, and is used to determine whether the voice information comes from the narration process of the collector. Similarly, the second prompt tone is equivalent to the voice tag of the person being collected, and is used to determine whether the voice information comes from the narration process of the person being collected. Since the collection process is usually asked by the collector, the first prompt tone is usually set in the voice segment.

[0131] The voice information of the collector and the collected person is classified and saved according to the first prompt tone and the second prompt tone.

[0132] Saving the voice segment depends on the settings of the first prompt tone and the second prompt tone. Each time a prompt tone is set, the voice segment preceding the prompt tone is saved in the storage space of the sound source according to the sound source corresponding to the prompt tone.

[0133] Since there may be a long period of thinking during the collection process, resulting in a long interval between narrations, the prompt tone may be set incorrectly, resulting in failure of voice classification and storage. Therefore, the setting of the prompt tone needs further verification. Figure 8 The process of classifying and storing the voice information of the collector and the collected person is further described, and the steps include:

[0134] Set waiting time thresholds for the first prompt tone and the second prompt tone.

[0135] The waiting time threshold is a verification standard set manually, and the narration time interval and the waiting time threshold are compared to preliminarily determine whether to set the prompt tone. However, since the speaker may think for too long, the method of setting the prompt tone needs to be further improved.

[0136] The voice information interval time of the collector and the voice information interval time of the collected person are recorded.

[0137] The voice information interval is used to compare with the waiting time threshold. If the voice information interval is less than the waiting time threshold, the prompt tone is not set until the voice information interval reaches the waiting time threshold, then the prompt tone is set, and the next step is performed to evaluate the accuracy of the prompt tone setting.

[0138] The first prompt tone or the second prompt tone is set in combination with the voice information interval time and the voiceprint feature.

[0139] The accuracy of the prompt tone setting is determined by using the voiceprint features before and after the prompt tone is set. The premise of setting the prompt tone is that the speaker has completed the staged narration and the next voice segment should be another speaker. Therefore, the voiceprint features before and after the prompt tone should be different.

[0140] The voice information including the first prompt tone is stored in the voice storage space of the collector.

[0141] The voice information including the second prompt tone is stored in the voice storage space of the person being collected.

[0142] The voice information of the collector and the collected person is classified and stored according to the first prompt tone and the second prompt tone. It is worth mentioning that the voice information of the collector and the collected person will also be stored in another storage space in the form of a conversation. When the voice information is classified and saved, the voice segment before each prompt tone is only saved once, and no additional classification and saving is performed.

[0143] The voiceprint feature is used to identify whether the speaker before and after the prompt tone is the same person, and is the basis for determining the accuracy of the prompt tone setting. Fig. 9 The process of evaluating the accuracy of the prompt tone setting is described in detail, and the steps include:

[0144] The voiceprint features of the collector and the collected person are collected and stored in a voice verification database.

[0145] The voiceprint feature is used to determine whether the sound source before and after the prompt sound is the same person. The voiceprint feature includes the speaker's timbre, pitch, loudness and other identification features. The voiceprint feature should be extracted and saved to the database before the collection process begins, so that the voice information can be saved synchronously with the collection process after the collection starts.

[0146] If the voice information interval is greater than the waiting time threshold, the first prompt tone or the second prompt tone is set to perform voiceprint verification.

[0147] By combining the determination result of the waiting time threshold with the determination result of the voiceprint test to determine the accuracy of the prompt tone setting behavior, the operation of triggering the prompt tone setting caused by the speaker thinking for too long is avoided, and the accuracy of voice classification storage is ensured.

[0148] If the voiceprint verification result shows that the voiceprint features are consistent, the first prompt tone or the second prompt tone that is set is canceled.

[0149] If the voiceprint verification result shows that the voiceprint features are inconsistent, the first prompt tone or the second prompt tone that is set is retained.

[0150] If the prompt tone is canceled, the next determination cycle is entered, and when the narration interval is greater than the waiting time threshold again, the above determination behavior is repeated. If the prompt tone is retained, the prompt tone and the voice information before the prompt tone are classified and stored.

[0151] The present application also provides a voice collection system for a multi-factor collection room, comprising: a data collection module, a data processing module and an execution module.

[0152] The data acquisition module is used to collect characteristic information of sound sources in the multi-factor acquisition room.

[0153] The data processing module is used to extract the location information of the sound source from the characteristic information of the sound source. The data processing module is also used to perform primary positioning of the direction of the sound source in combination with the sound source location information and the beam coverage of the microphone array used to collect voice information, so as to obtain a suboptimal voice collection direction. The data processing module is also used to perform secondary positioning of the direction of the sound source based on the voice information of the suboptimal voice collection direction, so as to obtain an optimal voice collection direction. The data processing module is also used to perform clarity processing on the voice information collected in the optimal voice collection direction, so as to obtain clear voice information.

[0154] The execution module is used to classify and store the clarified voice information based on the person being collected and the collector as a distinction standard.

[0155] The present application provides a method and system for voice collection in a multi-factor collection room, which performs primary screening of the beam feedback voice of the microphone array through the location information of the sound source and the beam coverage of the microphone array. The optimal voice collection direction and the optimal voice information are obtained by comparing the sound intensity of the beam feedback voice. The optimal voice information is clarified to reduce the interference of noise and echo on the voice information, and clear voice information is obtained. The voiceprint characteristics of the person being collected and the person collecting are used as the distinction criteria, a prompt tone is set, and the clear voice information is classified and stored.

[0156] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A method for voice collection in a multi-factor collection room, applied to sound collection of a sound source in a multi-factor collection room; the sound source in the multi-factor collection room is located at the geometric center of the multi-factor collection room; the speaker in the multi-factor collection room is parallel to the sound source and is arranged at the window of the multi-factor collection room; the microphone array in the multi-factor collection room is perpendicular to the speaker and is arranged at the top of the multi-factor collection room; characterized in that: include: Capturing images of the collector and the person being collected, so as to extract the position information of the collector and the person being collected from the image collection results, and the position information of the collector and the person being collected is used to preliminarily confirm the voice collection direction; Collect characteristic information of sound sources in a multi-factor acquisition room; Extracting the location information of the sound source from the feature information of the sound source; Setting a sound source determination area according to the location information of the sound source; In combination with the sound source determination area and the coverage of the microphone array beam for collecting voice information, the direction of the sound source is first-level positioned to obtain a suboptimal voice collection direction; wherein, if the coverage of the microphone array beam overlaps with the sound source determination area, the microphone array beam is determined to be a suboptimal voice collection beam, and the voice collection direction where the suboptimal voice collection beam is located is determined to be a suboptimal voice collection direction; if the coverage of the microphone array beam does not overlap with the sound source determination area, the microphone array beam is determined to be an interference beam, and the voice information of the interference beam is not received; Generate a voice information waveform according to the voice information data in the suboptimal voice collection direction; According to the voice information waveform, obtaining the sound intensity of the voice information in the suboptimal voice collection direction; Comparing the sound intensities of the voice information in the suboptimal voice collection directions one by one to obtain the optimal voice collection direction; Performing clarity processing on the voice information collected in the optimal voice collection direction to obtain clear voice information; The clarified voice information is classified and stored based on the distinction between the person being collected and the collector.

2. The method for voice collection in a multi-factor collection room according to claim 1, characterized in that: The characteristic information of the sound source in the multi-factor collection room includes: the position information of the person being collected, the position information of the collector, the voice information of the person being collected and the voice information of the collector.

3. The voice collection method for a multi-factor collection room according to claim 1, characterized in that: The step of comparing the sound intensities of the voice information in the suboptimal voice collection directions one by one to obtain the optimal voice collection direction comprises: Extract the sound intensity of the speech information in the two suboptimal speech collection directions for comparison, mark the speech information with stronger sound intensity as Class A speech information, and mark the speech information with weaker sound intensity as Class B speech information; Compare the sound intensity of the A-level voice information with the sound intensity of the voice information in the other suboptimal voice collection directions, and still mark the voice information with the strongest sound as the A-level voice information; The A-level voice information marked after the last sound intensity comparison is set as the optimal voice information, and the optimal voice collection direction is obtained.

4. The method for voice collection in a multi-factor collection room according to claim 3, characterized in that: The steps of performing clarity processing on the voice information collected in the optimal voice collection direction to obtain the clear voice information include: Performing attenuation processing on the Class B speech; Performing enhancement processing on the optimal voice information collected in the optimal voice collection direction to obtain enhanced optimal voice information; Performing noise reduction processing on the enhanced optimal voice information to obtain secondary clear voice information; Echo cancellation is performed on the secondary clear voice information to obtain clear voice information.

5. The voice collection method for a multi-factor collection room according to claim 1, characterized in that: The step of classifying and storing the clear voice information based on the person being collected and the collector as the distinguishing criteria includes: Setting a first prompt tone for the voice information of the collector; Setting a second prompt tone for the voice information of the person being collected; The voice information of the collector and the collected person is classified and saved according to the first prompt tone and the second prompt tone.

6. The method for voice collection in a multi-factor collection room according to claim 5, characterized in that: The step of classifying and saving the voice information of the collector and the collected person according to the first prompt tone and the second prompt tone comprises: Setting the waiting time thresholds for the first prompt tone and the second prompt tone; Record the time interval between the voice information of the collector and the time interval between the voice information of the person being collected; In combination with the voice information interval time and the voiceprint feature, setting the first prompt tone or the second prompt tone; Storing the voice information including the first prompt tone in the voice storage space of the collector; The voice information including the second prompt tone is stored in the voice storage space of the person being collected.

7. The method for voice collection in a multi-factor collection room according to claim 6, characterized in that: In combination with the voice information interval time and the voiceprint feature, the step of setting the first prompt tone or the second prompt tone includes: Collecting the voiceprint features of the collector and the collected person, and storing them in a voice verification database; If the voice information interval is greater than the waiting time threshold, setting the first prompt tone or the second prompt tone to perform voiceprint verification; If the voiceprint verification result shows that the voiceprint features are consistent, canceling the first prompt tone or the second prompt tone that is set; If the voiceprint verification result shows that the voiceprint features are inconsistent, the first prompt tone or the second prompt tone that is set is retained.

8. A voice collection system for a multi-factor collection room, characterized in that: include: Data acquisition module, data processing module and execution module; The data acquisition module is used to collect images of the collector and the collected person, so as to extract the location information of the collector and the collected person from the image acquisition results, and the location information of the collector and the collected person is used to preliminarily confirm the voice collection direction; The data acquisition module is also used to collect characteristic information of sound sources in the multi-factor acquisition room; The data processing module is used to extract the location information of the sound source from the characteristic information of the sound source; The data processing module is also used to combine the sound source position information and the beam coverage of the microphone array for collecting voice information to perform primary positioning on the direction of the sound source to obtain a suboptimal voice collection direction; wherein, if the coverage of the microphone array beam overlaps with the sound source determination area, the microphone array beam is determined to be a suboptimal voice collection beam, and the voice collection direction where the suboptimal voice collection beam is located is determined to be a suboptimal voice collection direction; if the coverage of the microphone array beam does not overlap with the sound source determination area, the microphone array beam is determined to be an interference beam, and the voice information of the interference beam is not received; The data processing module is also used to generate a voice information waveform according to the voice information data of the suboptimal voice collection direction; The data processing module is also used to obtain the sound intensity of the voice information in the suboptimal voice collection direction according to the voice information waveform; The data processing module is also used to compare the sound intensity of the voice information in the suboptimal voice collection direction one by one to obtain the optimal voice collection direction; The data processing module is also used to perform clarity processing on the voice information collected in the optimal voice collection direction to obtain clear voice information; The execution module is used to classify and store the clarified voice information based on the person being collected and the collector as a distinction standard.

Images