Method, apparatus, device and medium for distinguishing between teacher voice and non-teacher voice

Abstract

The application relates to a teacher voice and non-teacher voice distinguishing method, device, equipment and medium, relates to the technical field of voice classification, and the method comprises the following steps: collecting classroom voice, obtaining voiceprint vectors of multiple voice segments according to the classroom voice; adopting a method combining a BIRTCH clustering algorithm and a Calinski-Harabaz index, performing voiceprint clustering based on the voiceprint vectors, and distinguishing whether the voice segments corresponding to the voiceprint vectors are non-teacher voices or teacher voices. The application has the effect of accurately distinguishing whether the voices in the classroom are teacher voices or non-teacher voices.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sound classification, and in particular to a teacher sound and non-teacher sound distinguishing method, device, equipment and medium. BACKGROUND

[0002] In a classroom, teacher sound and non-teacher sound can often be heard, and in a classroom analysis process, these different sounds in the classroom often need to be distinguished in order to analyze different teaching behaviors in the classroom, such as teacher lectures.

[0003] At present, artificial, simple K-means algorithm or GMM-Kmeans algorithm is usually used to distinguish whether the sound in the classroom is a teacher sound or a non-teacher sound, wherein the artificial distinguishing workload is large and the cost is high, and the distinguishing precision of the K-means algorithm or the GMM-Kmeans algorithm is not high and the accuracy is poor. SUMMARY

[0004] In order to accurately distinguish whether the sound in the classroom is a teacher sound or a non-teacher sound, the present application provides a teacher sound and non-teacher sound distinguishing method, device, equipment and medium.

[0005] In a first aspect, the present application provides a teacher sound and non-teacher sound distinguishing method, which adopts the following technical solution:

[0006] A teacher sound and non-teacher sound distinguishing method, comprising:

[0007] Collecting a classroom sound, and obtaining a voiceprint vector of a plurality of sound segments according to the classroom sound;

[0008] Using a BIRTCH clustering algorithm and a Calinski-Harabaz index combined method to perform voiceprint clustering based on the voiceprint vector, and distinguishing whether the sound segment corresponding to the voiceprint vector is a non-teacher sound or a teacher sound.

[0009] By using the above technical solution, the voiceprint vector is clustered by using the BIRTCH clustering algorithm, and the precision of clustering is improved by using the characteristic of the Calinski-Harabaz index to evaluate the clustering effect, so that the clustering result is more accurate, and the sound in the classroom can be accurately distinguished as a teacher sound or a non-teacher sound.

[0010] Preferably, the collecting of the classroom sound and the obtaining of the voiceprint vector of the plurality of sound segments according to the classroom sound comprises:

[0011] Inputting the classroom sound into a trained voiceprint model to obtain the voiceprint vector of the plurality of sound segments.

[0012] Preferably, the classroom sound is input into the trained voiceprint model to obtain voiceprint vectors of multiple sound segments, including:

[0013] The classroom sound is segmented into multiple sound segments.

[0014] Voiceprint extraction is performed on the multiple sound segments respectively to obtain the voiceprint vectors.

[0015] Preferably, the classroom sound is segmented into multiple sound segments, including:

[0016] The classroom sound is divided into multiple segments, and adjacent segments have common parts and non-common parts.

[0017] Voiceprint feature matching degrees of the common parts and the non-common parts of adjacent segments are calculated respectively.

[0018] Based on the voiceprint feature matching degrees, a switching point is obtained.

[0019] The classroom sound is segmented into multiple sound segments according to the switching point.

[0020] By using the above technical solution, the switching point is detected based on the voiceprint feature matching degree, and the classroom sound is segmented into multiple sound segments. Each sound segment is of the same type of sound, for example, one sound segment is a teacher's voice and one sound segment is noise. Thus, it is convenient to classify each sound segment later.

[0021] Preferably, the training method of the voiceprint model includes:

[0022] An open-source sound dataset is obtained, and pre-collected classroom sound is made into a classroom sound dataset. The open-source sound dataset and the classroom sound dataset are used as a sample set together.

[0023] Samples in the sample set are input into the voiceprint model, and the voiceprint model is trained by using a deep learning algorithm.

[0024] By using the above technical solution, the voiceprint model is currently trained only by using an open-source sound dataset. The open-source sound dataset is mostly collected from near-field recording and video sound on a video website, while the sound in a classroom environment is mostly collected from a microphone hanging on a ceiling and belongs to far-field sound. Therefore, there is a cross-domain problem between the collection environment and the use environment. The traditional voiceprint model performs slightly worse when applied to a classroom environment. Moreover, the collection cost of the sound dataset is high, and the collection standard is not uniform. The sample set in the present application adds a classroom sound dataset made of a large amount of classroom sound to the open-source sound dataset. The voiceprint model is trained based on such a sample set, so that the voiceprint model is applicable to use in a classroom environment and improves the accuracy of the voiceprint vectors output by the voiceprint model.

[0025] Preferably, the method of combining BIRTCH clustering algorithm and Calinski-Harabaz index, based on the voiceprint vector, carries out voiceprint clustering, and distinguishes whether the voice segment corresponding to the voiceprint vector is a non-teacher voice or a teacher voice, comprising:

[0026] Using the BIRTCH clustering algorithm, all voiceprint vectors are clustered, and all voiceprint vectors are divided into a first class and a second class;

[0027] Using the BIRTCH clustering algorithm, all voiceprint vectors in the first class and all voiceprint vectors in the second class are respectively subjected to secondary clustering;

[0028] First and second indexes are respectively obtained; wherein the first index is the Calinski-Harabaz index after the secondary clustering of all voiceprint vectors in the first class, and the second index is the Calinski-Harabaz index after the secondary clustering of all voiceprint vectors in the second class;

[0029] It is judged whether the first index is greater than the second index;

[0030] If yes, it is determined that the voice segment corresponding to the voiceprint vector in the first class is a non-teacher voice;

[0031] If no, it is determined that the voice segment corresponding to the voiceprint vector in the second class is a non-teacher voice.

[0032] By adopting the technical scheme, in the classroom environment, the teacher voice and the non-teacher voice have certain differences, therefore, the BIRTCH clustering algorithm is used to cluster all the voiceprint vectors for the first time, all the voiceprint vectors can be clustered into two categories through the difference between the teacher voice and the non-teacher voice, but after the first clustering is completed, it is not clear which category of voiceprint vectors corresponds to the non-teacher voice and which category of voiceprint vectors corresponds to the teacher voice. The second clustering is performed on all the voiceprint vectors in the first category and all the voiceprint vectors in the second category respectively, the first index and the second index are obtained, the Calinski-Harabaz index can evaluate the clustering effect, through this characteristic, the teacher voice and the non-teacher voice are distinguished: the teacher is a minority in the classroom, therefore, the teacher voice is single, if the voiceprint vectors in which category correspond to the teacher voice, the clustering effect is poor, therefore, the Calinski-Harabaz index is small, the voiceprint vectors in the category with the small Calinski-Harabaz index are determined as the teacher voice; the students are a minority in the classroom, therefore, the non-teacher voice is various, if the voiceprint vectors in which category correspond to the non-teacher voice, the clustering effect is good, therefore, the Calinski-Harabaz index is large, the voiceprint vectors in the category with the large Calinski-Harabaz index are determined as the non-teacher voice.

[0033] Preferably, the method combining the BIRTCH clustering algorithm and the Calinski-Harabaz index is based on the voiceprint vector to perform voiceprint clustering and distinguish whether the voice segment corresponding to the voiceprint vector is the non-teacher voice or the teacher voice, and includes the following steps.

[0034] A plurality of voiceprint feature similarities are set, the plurality of voiceprint feature similarities are filtered based on the Calinski-Harabaz index, and the optimal voiceprint feature similarity is obtained;

[0035] The two categories in the optimal clustering result are respectively taken as the first category and the second category; wherein the optimal clustering result is a result of clustering all the voiceprint vectors by using the optimal voiceprint feature similarity;

[0036] The BIRTCH clustering algorithm is adopted to perform the second clustering on all the voiceprint vectors in the first category and all the voiceprint vectors in the second category respectively;

[0037] The first index and the second index are respectively obtained; wherein the first index is the Calinski-Harabaz index after the second clustering on all the voiceprint vectors in the first category, and the second index is the Calinski-Harabaz index after the second clustering on all the voiceprint vectors in the second category;

[0038] determining whether the first index is greater than the second index;

[0039] If yes, determining that the voiceprint vector in the first category corresponds to a non-teacher voice sound segment;

[0040] If no, determining that the voiceprint vector in the second category corresponds to a non-teacher voice sound segment.

[0041] By adopting the technical solution, the Calinski-Harabaz index is used to screen the similarity of multiple voiceprint features, and then the optimal clustering result with the highest accuracy is determined, two categories in the optimal clustering result are selected as the first category and the second category, and then it is distinguished which category the voiceprint vector in the category corresponds to a non-teacher voice sound segment or a teacher voice sound segment, so that the result of the distinction is more accurate.

[0042] In a second aspect, the present application provides a teacher voice and non-teacher voice distinguishing device, which adopts the following technical solution:

[0043] A teacher voice and non-teacher voice distinguishing device, comprising,

[0044] a collection module, configured to collect a classroom voice, and obtain voiceprint vectors of multiple voice sound segments according to the classroom voice; and

[0045] a distinguishing module, configured to use a BIRTCH clustering algorithm and a Calinski-Harabaz index combined method to perform voiceprint clustering based on the voiceprint vectors, and distinguish whether the voice sound segment corresponding to the voiceprint vector is a non-teacher voice sound or a teacher voice sound.

[0046] In a third aspect, the present application provides a computer device, which adopts the following technical solution:

[0047] A computer device, comprising a memory and a processor, and the memory stores a computer program capable of being loaded and executed by the processor to perform the teacher voice and non-teacher voice distinguishing method of any one of the first aspect.

[0048] In a fourth aspect, the present application provides a computer readable storage medium, which adopts the following technical solution:

[0049] A computer readable storage medium, which stores a computer program capable of being loaded and executed by the processor to perform the teacher voice and non-teacher voice distinguishing method of any one of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a flowchart of the teacher voice and non-teacher voice distinguishing method provided by the embodiments of the present application.

[0051] Figure 2 This is a schematic diagram of classroom audio provided in an embodiment of this application.

[0052] Figure 3 This is a flowchart illustrating another method for distinguishing between teacher's voice and non-teacher's voice provided in an embodiment of this application.

[0053] Figure 4 This is a structural block diagram of the device for distinguishing between teacher's voice and non-teacher's voice provided in the embodiments of this application.

[0054] Figure 5 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0056] This embodiment provides a method for distinguishing between teacher's voice and non-teacher's voice, such as... Figure 1 As shown, the main process of this method is described as follows (steps S101 to S102):

[0057] Step S101: Collect classroom audio and obtain voiceprint vectors of multiple audio segments based on the classroom audio.

[0058] In this embodiment, classroom audio is input into a trained voiceprint model to obtain voiceprint vectors for multiple audio segments.

[0059] The specific training method for the voiceprint model is as follows:

[0060] Open-source audio data is acquired online, typically from near-field recordings and video audio from video websites. This acquired audio data is then organized and merged into an audio dataset. Real-time classroom audio data is collected using audio acquisition devices installed in the classroom. This collected classroom audio data is then organized and merged into a classroom audio dataset. Both the open-source audio dataset and the classroom audio dataset are used as a sample set.

[0061] The samples from the sample set are input into the voiceprint model, and the voiceprint model is trained using a deep learning algorithm. The audio acquisition device can be a hanging microphone in a classroom.

[0062] In this embodiment, when class begins, the classroom sound is collected by an audio acquisition device and input into a trained voiceprint model. The voiceprint model is then used to segment the classroom sound into multiple sound segments, and voiceprints are extracted from each of the multiple sound segments to obtain the voiceprint vector of each sound segment.

[0063] One embodiment of the method of dividing the classroom sound into multiple sound segments is as follows: the classroom sound is divided into multiple segments, adjacent segments have a common part and a non-common part; the voiceprint feature matching degrees of the common part and the non-common parts of adjacent segments are calculated respectively; the switching points are obtained based on the voiceprint feature matching degrees; and the classroom sound is divided into multiple sound segments according to the switching points.

[0064] The specific method of obtaining the switching points is as follows: the voiceprint feature matching degrees of the common part and the non-common parts of two adjacent segments are compared, and the segment with a higher voiceprint feature matching degree with the common part is determined as the sound segment of the same person, and thus the switching point is the overlapping point of the non-common part of the other segment and the common part.

[0065] The above content is illustrated by examples as follows: Figure 2 The coordinate axis is the time axis, and the classroom sound is collected at the beginning of the class, which is recorded as 0 seconds on the coordinate axis. The classroom sound every 2 seconds is taken as a segment, and the 1-second sound between adjacent segments is repeated, which is taken as the common part between adjacent segments.

[0066] Figure 2 There are four segments in the figure, namely segment A [0s, 2s], segment B [1s, 3s], segment C [2s, 4s] and segment D [3s, 5s]. Among them, segment B to segment C is 1s to 4s, the common part of segment B and segment C is [2s, 3s], the non-common part of segment B is [1s, 2s], and the non-common part of segment C is [3s, 4s].

[0067] Voiceprints are extracted from the [1s, 2s], [2s, 3s] and [3s, 4s] parts respectively to obtain three voiceprint feature vectors. The voiceprint feature matching degree of the voiceprint feature vector of the [2s, 3s] part and the voiceprint feature vector of the [1s, 2s] part is calculated, and the voiceprint feature matching degree is taken as the first matching degree. The voiceprint feature matching degree of the voiceprint feature vector of the [2s, 3s] part and the voiceprint feature vector of the [3s, 4s] part is calculated, and the voiceprint feature matching degree is taken as the second matching degree.

[0068] It is determined whether the first matching degree is greater than the second matching degree; if yes, the third second is taken as a switching point; if no, the second second is taken as a switching point.

[0069] It is worth noting that if the first matching degree is equal to the second matching degree, it is determined that segment B and segment C are the segments of the same person, and there is no switching point from segment B to segment C. However, this case where the first matching degree is equal to the second matching degree is extremely rare, so there is a high probability that there will be a switching point, and the classroom sound will be cut into many sound segments.

[0070] determining whether the class is over; if yes, turning to step S102.

[0071] wherein the specific method for determining whether the class is over is: determining whether there is class sound; if no, timing; determining whether the timing time is greater than a preset time; if yes, determining that the class is over.

[0072] The application also provides another implementable scheme for dividing the class sound into multiple sound segments, which is as follows:

[0073] dividing the class sound into multiple segments, the adjacent segments having a common part and a non-common part; determining the adjacent segments where the sound transformation occurs; for the adjacent segments where the sound transformation occurs, respectively calculating the voiceprint feature matching degrees of the common part and the non-common part of the adjacent segments; based on the voiceprint feature matching degrees, obtaining the switching points; and dividing the class sound into multiple sound segments according to the switching points.

[0074] wherein the specific method for determining the adjacent segments where the sound transformation occurs is: for 4 continuous segments, there are 3 groups of adjacent segments in turn, respectively calculating the voiceprint feature matching degrees between each group of adjacent segments, defining the voiceprint feature matching degree between the first group of adjacent segments as the first voiceprint feature matching degree, defining the voiceprint feature matching degree between the second group of adjacent segments as the second voiceprint feature matching degree, and defining the voiceprint feature matching degree between the third group of adjacent segments as the third voiceprint feature matching degree. Determining whether the difference between the second voiceprint feature matching degree and the first voiceprint feature matching degree is less than a first preset value, and the difference between the third voiceprint feature matching degree and the second voiceprint feature matching degree is greater than a second preset value; if yes, determining that the second group of adjacent segments is the adjacent segments where the sound transformation occurs. The method for obtaining the switching points in the adjacent segments where the sound transformation occurs is consistent with the principle of the first implementable scheme for dividing the class sound into multiple sound segments, which is not repeated here.

[0075] For example, referring to Figure 2 , segment A and segment B are the first group of adjacent segments, segment B and segment C are the second group of adjacent segments, and segment C and segment D are the third group of adjacent segments. The first voiceprint feature matching degree is calculated based on the voiceprint feature vectors of segment A and segment B, the second voiceprint feature matching degree is calculated based on the voiceprint feature vectors of segment B and segment C, and the third voiceprint feature matching degree is calculated based on the voiceprint feature vectors of segment C and segment D. After comparing the first voiceprint feature matching degree, the second voiceprint feature matching degree and the third voiceprint feature matching degree, it is determined that segment B and segment C are the adjacent segments where the sound transformation occurs.

[0076] It is worth noting that the classroom sound can be acquired in real time during the classroom, and the acquired classroom sound can be input into the voiceprint model for processing, or the classroom sound of the whole class can be input into the voiceprint model for processing after the classroom.

[0077] Step S102: The voiceprint clustering is performed based on the voiceprint vectors by using a method combining the BIRTCH clustering algorithm and the Calinski-Harabaz index, so as to distinguish whether the sound segments corresponding to the voiceprint vectors are non-teacher sounds or teacher sounds.

[0078] The Calinski-Harabaz index is used to measure the voiceprint clustering effect, and the larger the Calinski-Harabaz index is, the more compact each class is and the more dispersed the classes are, and the better the clustering result is.

[0079] Specifically, the BIRTCH clustering algorithm is used to cluster all the voiceprint vectors, and all the voiceprint vectors are divided into a first class and a second class; the BIRTCH clustering algorithm is used to perform secondary clustering on all the voiceprint vectors in the first class and all the voiceprint vectors in the second class, respectively; the first index and the second index are obtained, respectively; the first index is the Calinski-Harabaz index after the secondary clustering of all the voiceprint vectors in the first class, and the second index is the Calinski-Harabaz index after the secondary clustering of all the voiceprint vectors in the second class.

[0080] It is judged whether the first index is larger than the second index; if yes, it is determined that the sound segments corresponding to the voiceprint vectors in the first class are non-teacher sounds, and the sound segments corresponding to the voiceprint vectors in the second class are teacher sounds; if not, it is determined that the sound segments corresponding to the voiceprint vectors in the first class are teacher sounds, and the sound segments corresponding to the voiceprint vectors in the second class are non-teacher sounds.

[0081] It is worth noting that there is basically no case that the first index is equal to the second index, because there is a relatively obvious difference between the teacher sounds and the non-teacher sounds.

[0082] The principle of the BIRTCH clustering algorithm is to create a tree structure based on parameters, and the sample interval threshold parameter used to determine whether two voiceprint vectors are in the same class is the voiceprint feature similarity.

[0083] The preset voiceprint feature similarity is used to cluster all the voiceprint vectors into two categories, i.e., a first category and a second category. At this time, it is not clear which category of voiceprint vectors corresponds to the non-teacher voice. Therefore, the first category and the second category are subjected to secondary clustering, respectively, to obtain a first index and a second index. The voiceprint vectors in the category with the larger index correspond to the non-teacher voice, and the voiceprint vectors in the category with the smaller index correspond to the teacher voice.

[0084] The calculation formulas of the first index and the second index are as follows:

[0085]

[0086] wherein S represents the Calinski-Harabaz index; K represents the number of clustering categories (in this application, all the voiceprint vectors are clustered into two categories, so the number of clustering categories is two); N represents the total number of samples (the sample is a voiceprint vector, and the total number of samples refers to the number of voiceprint vectors); SS B represents the intra-class compactness; SS W represents the inter-class separation.

[0087] The expression of the intra-class compactness is as follows:

[0088]

[0089] The expression of the inter-class separation is as follows:

[0090]

[0091] wherein C k represents the number of samples in the kth category, x kc represents the feature value of the cth sample in the kth category, x kmean represents the average feature value of the kth category, X mean represents the average feature value of all samples.

[0092] The application also provides another implementation scheme, which is as follows:

[0093] A plurality of voiceprint feature similarities are preset, and all the voiceprint vectors are subjected to multiple clustering by using the plurality of voiceprint feature similarities, respectively, to obtain a plurality of clustering results. The Calinski-Harabaz index of each clustering is calculated. The clustering result corresponding to the largest Calinski-Harabaz index is selected as the optimal clustering result, the voiceprint feature similarity corresponding to the optimal clustering result is the optimal voiceprint feature similarity, and the two categories in the optimal clustering result are respectively taken as the first category and the second category.

[0094] For example, with reference toFigure 3 The four voiceprint feature similarities are set in advance by artificial experience, and the four voiceprint feature similarities are used to cluster all voiceprint vectors four times. Among them, all voiceprint vectors are clustered into two classes, class a and class b, by using a voiceprint feature similarity of 0.35, and the Calinski-Harabaz index of this clustering is the first clustering index; all voiceprint vectors are clustered into two classes, class c and class d, by using a voiceprint feature similarity of 0.5, and the Calinski-Harabaz index of this clustering is the second clustering index; all voiceprint vectors are clustered into two classes, class e and class f, by using a voiceprint feature similarity of 0.65, and the Calinski-Harabaz index of this clustering is the third clustering index; all voiceprint vectors are clustered into two classes, class g and class h, by using a voiceprint feature similarity of 0.8, and the Calinski-Harabaz index of this clustering is the fourth clustering index. After comparison, the second clustering index is the largest, so the voiceprint feature similarity of 0.5 is the optimal voiceprint feature similarity among the four voiceprint feature similarities of 0.35, 0.5, 0.65 and 0.8, and class c is taken as the first class and class d is taken as the second class.

[0095] It is worth noting that, Figure 3 The method for determining non-teacher voice and teacher voice is only one of the cases, and the values and calculation results are only used for illustration and do not limit the protection scope of the present application.

[0096] Through the above content, the voiceprint feature similarity is selected in the first round, and further, the voiceprint feature similarity can be selected in the second round, or even more rounds, to select a more optimal voiceprint feature similarity. For example, the voiceprint feature similarity of 0.5 is selected in the first round of selection, and the voiceprint feature similarities of 0.44, 0.47, 0.5, 0.53 and 0.56 are set for the second round of selection. The principle of the second round of selection is the same as that of the first round of selection, and will not be repeated here. Assuming that the result of the second round of selection is the voiceprint feature similarity of 0.53, the voiceprint feature similarity of 0.53 is correspondingly taken as the optimal voiceprint feature similarity, and the clustering result obtained by clustering voiceprints through the voiceprint feature similarity of 0.53 is taken as the optimal clustering result, and the two classes in the optimal clustering result are the first class and the second class.

[0097] The first class and the second class are distinguished by performing secondary clustering on all voiceprint vectors in the first class and all voiceprint vectors in the second class, respectively. The distinguishing method is the same as the method of distinguishing non-teacher voice and teacher voice by using the first index and the second index, and will not be repeated here.

[0098] Further, feature extraction is performed on the sound segment that is the non-teacher voice to obtain a first spectrogram, the first spectrogram is input into the trained sound classification model, and the sound segment corresponding to the voiceprint vector is classified according to the voiceprint vector; wherein the first spectrogram is a two-dimensional mel spectrogram.

[0099] The training method of the sound classification model is specifically as follows:

[0100] Each training sample in the training sample set is converted into a second spectrogram, and the second spectrogram is a two-dimensional mel spectrogram; the second spectrogram is input into the sound classification model, and the sound classification model is trained by using a VGG11 network structure.

[0101] The method for obtaining the training sample set is as follows: the sound in the classroom is collected by using an audio acquisition device installed in the classroom, the sound in the classroom is extracted into multiple sound segments, and each sound segment is taken as a training sample. Each training sample is manually labeled, and the corresponding label of each training sample is the voice of a single student, the voice of multiple students reading in unison, the discussion voice, and the noise, etc. In this embodiment, the number of finally labeled samples is 69000, of which the number of single student voices is 35000, the number of multiple student voices reading in unison is 9000, the number of discussion voices is 12000, and the number of noises is 13000.

[0102] In this embodiment, feature extraction is performed on the sound segment that is the non-teacher voice to obtain a first spectrogram, the first spectrogram is input into the trained sound classification model, and the sound classification model classifies the sound according to the first spectrogram and outputs a classification result; wherein the classification result includes the voice of a single student, the voice of multiple students reading in unison, the discussion voice, and the noise.

[0103] In summary, the voice data of the whole class is used in the present application, the voiceprint model and the sound classification model are combined, different voices in the classroom are automatically and interactively distinguished, and then the classroom teaching analysis can be more efficient and more accurate.

[0104] In order to better implement the above method, an embodiment of the present application further provides a teacher voice and non-teacher voice distinguishing device. The device can be integrated in a computer device, such as a terminal or a server, etc. The terminal can include but is not limited to a mobile phone, a tablet computer, a desktop computer, etc.

[0105] Figure 4 A structural block diagram of a teacher voice and non-teacher voice distinguishing device provided by an embodiment of the present application is shown in FIG. 2, which mainly includes: Figure 4

[0106] The acquisition module 201 is configured to collect the classroom sound and obtain the voiceprint vectors of multiple sound segments according to the classroom sound.​

[0107] The distinguishing module 202 is configured to perform voiceprint clustering based on the voiceprint vectors by using a BIRTCH clustering algorithm and a Calinski-Harabaz index combination method, and distinguish whether the sound segments corresponding to the voiceprint vectors are non-teacher sounds or teacher sounds.

[0108] The various changes and specific examples of the method provided in the above embodiments are also applicable to the teacher sound and non-teacher sound distinguishing device of the present embodiment. Through the foregoing detailed description of the teacher sound and non-teacher sound distinguishing method, those skilled in the art can clearly understand the implementation method of the teacher sound and non-teacher sound distinguishing device in the present embodiment. For the sake of brevity of the description, the implementation method of the teacher sound and non-teacher sound distinguishing device in the present embodiment will not be described in detail here.

[0109] In order to better execute the program of the above method, the present embodiment further provides a computer device, such as Figure 5 As shown in the figure, the computer device 300 includes a memory 301 and a processor 302.

[0110] The computer device 300 can be implemented in various forms, including mobile phones, tablets, palmtop computers, notebook computers, and desktop computers, etc.

[0111] The memory 301 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 301 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as obtaining voiceprint vectors of multiple sound segments from classroom sounds, etc.), and instructions for implementing the teacher sound and non-teacher sound distinguishing method provided in the above embodiments, etc.; the data storage area can store data involved in the teacher sound and non-teacher sound distinguishing method provided in the above embodiments, etc.

[0112] The processor 302 can include one or more processing cores. The processor 302 invokes the data stored in the memory 301 by running or executing the instructions, programs, code sets or instruction sets stored in the memory 301, performs various functions and processes data of the present application. The processor 302 can be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller and a microprocessor. It can be understood that, for different devices, the electronic device for implementing the functions of the processor 302 described above can also be other, and the embodiments of the present application are not limited specifically.

[0113] The embodiments of the present application provide a computer readable storage medium, for example, various media capable of storing program codes such as a U disk, a mobile hard disk, a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk. The computer readable storage medium stores a computer program capable of being loaded by a processor and executing the teacher voice and non-teacher voice distinguishing method of the above-mentioned embodiments.

[0114] The specific embodiments of the present application are only an explanation of the present application, and are not a limitation of the present application. Those skilled in the art can make modifications to the embodiments according to the needs after reading the present specification, and the modifications do not contribute to the creativity, but are protected by the patent law as long as they are within the scope of the claims of the present application.

Claims

1. A method for distinguishing between teacher's voice and non-teacher's voice, characterized in that, include: Collect classroom audio, input the classroom audio into a trained voiceprint model, and obtain voiceprint vectors for multiple audio segments. Specifically, this includes: dividing the classroom audio into multiple audio segments, extracting voiceprints from each of the multiple audio segments, and obtaining the voiceprint vectors. A method combining the BIRCH clustering algorithm and the Calinski-Harabaz index is used to perform voiceprint clustering based on the voiceprint vector, distinguishing whether the sound segment corresponding to the voiceprint vector is a non-teacher's voice or a teacher's voice. The step of dividing the classroom sound into multiple sound segments includes: dividing the classroom sound into multiple segments, with adjacent segments having common and non-common parts; calculating the voiceprint feature matching degree between the common part and the non-common part of the adjacent segments respectively; and obtaining a switching point based on the voiceprint feature matching degree, including: comparing the voiceprint feature matching degree between the common part and the non-common part of two adjacent segments, determining the segment with a higher voiceprint feature matching degree with the common part as the sound segment of the same person speaking, and determining the overlap point between the non-common part and the common part of the other segment as the switching point; The classroom audio is divided into multiple audio segments according to the switching point; Dividing classroom audio into multiple audio segments also includes: The classroom audio is divided into multiple segments, with common and non-common parts between adjacent segments; adjacent segments where audio changes occur are identified; for adjacent segments where audio changes occur, the voiceprint feature matching degree between the common part and the non-common part of the adjacent segment is calculated; based on the voiceprint feature matching degree, the switching point is obtained; the classroom audio is divided into multiple audio segments according to the switching point. The specific method for determining the adjacent segments where the voice change occurs is as follows: For four consecutive segments, there are three groups of adjacent segments in sequence. The voiceprint feature matching degree between each group of adjacent segments is calculated. The voiceprint feature matching degree between the first group of adjacent segments is defined as the first voiceprint feature matching degree, the voiceprint feature matching degree between the second group of adjacent segments is defined as the second voiceprint feature matching degree, and the voiceprint feature matching degree between the third group of adjacent segments is defined as the third voiceprint feature matching degree. It is then determined whether the difference between the second voiceprint feature matching degree and the first voiceprint feature matching degree is less than a first preset value, and whether the difference between the third voiceprint feature matching degree and the second voiceprint feature matching degree is greater than a second preset value. If so, the second group of adjacent segments is determined to be the adjacent segment where the voice change occurs.

2. The method according to claim 1, characterized in that, The training method for the voiceprint model includes: Obtain an open-source audio dataset, create a classroom audio dataset from the pre-collected classroom audio, and use both the open-source audio dataset and the classroom audio dataset as a sample set; The samples in the sample set are input into the voiceprint model, and the voiceprint model is trained using a deep learning algorithm.

3. The method according to claim 1, characterized in that, The method employing a combination of the BIRCH clustering algorithm and the Calinski-Harabaz index performs voiceprint clustering based on the voiceprint vector, distinguishing whether the voice segment corresponding to the voiceprint vector is a non-teacher's voice or a teacher's voice, including: The BIRCH clustering algorithm is used to cluster all voiceprint vectors, dividing all voiceprint vectors into Class I and Class II. The BIRCH clustering algorithm is used to perform secondary clustering on all voiceprint vectors in the first class and all voiceprint vectors in the second class. Obtain the first index and the second index respectively; wherein, the first index is the Calinski-Harabaz index after performing a second clustering on all voiceprint vectors in the first class, and the second index is the Calinski-Harabaz index after performing a second clustering on all voiceprint vectors in the second class. Determine whether the first index is greater than the second index; If so, then the sound segment corresponding to the voiceprint vector in the first category is determined to be a non-teacher's voice; If not, then the sound segment corresponding to the voiceprint vector in the second category is determined to be a non-teacher's voice.

4. The method according to claim 1, characterized in that, The method employing a combination of the BIRCH clustering algorithm and the Calinski-Harabaz index performs voiceprint clustering based on the voiceprint vector, distinguishing whether the voice segment corresponding to the voiceprint vector is a non-teacher's voice or a teacher's voice, including: Multiple voiceprint feature similarities are set, and the Calinski-Harabaz index is used to filter the multiple voiceprint feature similarities to obtain the optimal voiceprint feature similarity. The two classes in the optimal clustering result are respectively designated as the first class and the second class; wherein, the optimal clustering result is the result of clustering all voiceprint vectors using the optimal voiceprint feature similarity; The BIRCH clustering algorithm is used to perform secondary clustering on all voiceprint vectors in the first class and all voiceprint vectors in the second class. Obtain the first index and the second index respectively; wherein, the first index is the Calinski-Harabaz index after performing a second clustering on all voiceprint vectors in the first class, and the second index is the Calinski-Harabaz index after performing a second clustering on all voiceprint vectors in the second class. Determine whether the first index is greater than the second index; If so, then the sound segment corresponding to the voiceprint vector in the first category is determined to be a non-teacher's voice; If not, then the sound segment corresponding to the voiceprint vector in the second category is determined to be a non-teacher's voice.

5. A device for distinguishing between teacher's voice and non-teacher's voice, characterized in that, Used in the method according to any one of claims 1-4 include, The acquisition module is used to acquire classroom audio and obtain voiceprint vectors for multiple audio segments based on the classroom audio. as well as, The differentiation module is used to perform voiceprint clustering based on the voiceprint vector using a combination of the BIRCH clustering algorithm and the Calinski-Harabaz index, and to distinguish whether the sound segment corresponding to the voiceprint vector is a non-teacher's voice or a teacher's voice.

6. A computer device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and executed according to any one of claims 1 to 4.

Images