A teacher role recognition method, an electronic device, and a computer-readable storage medium

Abstract

The application discloses a teacher role recognition method, an electronic device and a computer readable storage medium. The method comprises the following steps: obtaining first audio data corresponding to a first lesson section and second audio data corresponding to a second lesson section; determining a first role set and the voiceprint features of each single-person speaking role in the first role set based on the voiceprint features of single-person speaking audio frames in the first audio data; determining a second role set and the voiceprint features of each single-person speaking role in the second role set based on the voiceprint features of single-person speaking audio frames in the second audio data; performing role merging on the first role set and the second role set based on the voiceprint similarity of single-person speaking roles between the first role set and the second role set to obtain a target role set; and determining the corresponding cumulative speaking time lengths of each single-person speaking role in the target role set in the first audio data and the second audio data, and taking the role with the longest cumulative speaking time length as a teacher role.

Description

Technical Field

[0001] This application relates to the field of audio technology, and in particular to a teacher role recognition method, electronic device, and computer-readable storage medium. Background Technology

[0002] With the development of information technology in education, intelligent analysis of the teaching process has become an important means to improve teaching quality. In the teaching scenario, the content taught by the teacher is a key factor in analyzing teaching effectiveness. Therefore, accurately identifying the teacher's role and determining the parts taught by the teacher is crucial for subsequent teaching content analysis and teaching evaluation.

[0003] However, existing identification methods struggle to accurately distinguish between teacher and student roles. Student roles may be misidentified as teacher roles, leading to errors in subsequent teaching content analysis and impacting the accuracy and effectiveness of teaching assessments. Therefore, improving the accuracy of teacher role identification has become a pressing technical problem. Summary of the Invention

[0004] This application discloses a teacher role recognition method, electronic device, and computer-readable storage medium, which recognizes teacher roles based on audio data from multiple lessons, thereby improving the accuracy of teacher role recognition in teaching scenarios.

[0005] In a first aspect, embodiments of this application provide a teacher role recognition method, which can be applied to electronic devices or devices compatible with electronic devices (such as devices containing chips or processors within electronic devices). The method includes: acquiring first audio data and second audio data, wherein the first audio data includes audio data of a first lesson and the second audio data includes audio data of a second lesson; determining a first role set and the voiceprint features of each individual speaking role in the first role set based on the voiceprint features of individual speaking audio frames in the first audio data; determining a second role set and the voiceprint features of each individual speaking role in the second role set based on the voiceprint features of individual speaking audio frames in the second audio data; merging the first role set and the second role set based on the similarity of the voiceprint features of individual speaking roles between the first role set and the second role set to obtain a target role set; determining the cumulative speaking time of each individual speaking role in the target role set corresponding to the first audio data and the second audio data, and determining the role with the longest cumulative speaking time in the target role set as the teacher role.

[0006] In this technical solution, by integrating the first audio data and the second audio data, and after merging the roles of the two lessons, the role with the longest cumulative speaking time in the target role set is determined as the teacher role, which can make the teacher role judgment more accurate.

[0007] In one possible implementation, the method further includes: acquiring third audio data, the third audio data including audio data of a third lesson; determining a third set of roles and the voiceprint features of each individual speaking role in the third set based on the voiceprint features of the individual speaking audio frames in the third audio data; merging the first set of roles and the second set of roles based on the similarity of the voiceprint features of the individual speaking roles between the first set of roles and the second set of roles to obtain a target set of roles, including: merging the first set of roles and the second set of roles based on the similarity of the voiceprint features of the individual speaking roles between the first set of roles and the second set of roles. The set is merged to obtain an intermediate set of roles and the voiceprint features of each individual speaking role in the intermediate set; based on the similarity of the voiceprint features of the individual speaking roles between the intermediate set and the third set of roles, the intermediate set and the third set of roles are merged to obtain a target set of roles; the cumulative speaking time of each individual speaking role in the target set is determined in the first audio data and the second audio data, including: determining the cumulative speaking time of each individual speaking role in the target set in the audio dataset, the audio dataset including the first audio data, the second audio data and the third audio data.

[0008] In this technical solution, by integrating audio data from multiple lessons and merging roles from several lessons, the role with the longest cumulative speaking time in the target role set is identified as the teacher role, which can improve the accuracy of teacher role identification.

[0009] In one possible implementation, the first, second, and third lessons are arranged in the order of their class times, from front to back.

[0010] In this technical solution, the roles for the first two lessons are merged according to the chronological order of the lessons. Then, the resulting set of intermediate roles is merged with the third lesson using the same process. Because the teachers' voices and recording environments are similar in consecutive lessons, merging them according to the chronological order of the lessons reduces interference and improves the accuracy of the merged roles.

[0011] In one possible implementation, based on the similarity of the voiceprint features of individual speaking characters between the first and second character sets, role merging is performed, including:

[0012] Determine the similarity of the voiceprint features of every two individual speaking characters between the first character set and the second character set to obtain the first similarity set;

[0013] Merge two single-person speaking roles in the first similarity set whose similarity is greater than the similarity threshold. Each single-person speaking role in the first role set and the second role set is merged at most once.

[0014] In this technical solution, each individual speaking role is merged at most once, which can reduce the amount of computation during the role merging process.

[0015] In one possible implementation, merging two single-person speaking roles with similarities greater than a similarity threshold in the first similarity set includes: if the maximum similarity in the first similarity set is greater than the similarity threshold, merging the two single-person speaking roles with the maximum similarity in the first similarity set, and deleting the maximum similarity and the first similarity in the first similarity set; wherein, the two single-person speaking roles with the first similarity include one of the single-person speaking roles with the maximum similarity in the first similarity set; if the maximum similarity in the updated first similarity set is greater than the similarity threshold, merging the two single-person speaking roles with the maximum similarity in the updated first similarity set, and deleting the maximum similarity and the second similarity in the updated first similarity set, until the maximum similarity in the updated first similarity set is less than or equal to the similarity threshold; wherein, the two single-person speaking roles with the second similarity include one of the single-person speaking roles with the maximum similarity in the updated first similarity set.

[0016] In this technical solution, roles are merged in descending order of similarity. This prioritizes merging the most similar roles, improving the accuracy of the role merging process. Furthermore, by removing the highest and lowest similarity scores from the first similarity set, it ensures that already merged single-person speaking roles will not be merged again, thus reducing the computational load during the role merging process.

[0017] In one possible implementation, the method further includes: segmenting the first audio data to obtain at least one audio segment, the audio segment categories including single-person speech, multi-person speech, and no-person speech; determining the single-person speech audio segment corresponding to the first lesson in the at least one audio segment, the single-person speech audio segment being an audio segment belonging to the single-person speech category; wherein, the single-person speech audio frame in the first audio data includes the audio frame in the single-person speech audio segment corresponding to the first lesson.

[0018] In one possible implementation, the method further includes: determining the individual speaking audio segments corresponding to each lesson in the first role set and their corresponding individual speaking roles; determining the voiceprint features of each individual speaking role in the first role set based on the voiceprint features of the individual speaking audio frames in the first audio data, including: determining the voiceprint features of each individual speaking audio segment corresponding to the first lesson based on the voiceprint features of the audio frames in each individual speaking audio segment; and determining the voiceprint features of each individual speaking role in the first role set based on the voiceprint features of the individual speaking audio segments corresponding to each individual speaking role.

[0019] In one possible implementation, determining a first set of roles based on the voiceprint features of single-person speaking audio frames in the first audio data includes: clustering the voiceprint features of single-person speaking audio frames in the first audio data to determine the first set of roles.

[0020] Secondly, embodiments of this application provide a teacher role recognition device, which includes units for implementing the method described in the first aspect.

[0021] Thirdly, embodiments of this application provide an electronic device including at least one processor; the at least one processor is configured to run a computer program or instructions to perform the method described in the first aspect.

[0022] In one possible implementation, the electronic device may further include a memory connected to the at least one processor; the memory is used to store computer programs or instructions; the at least one processor is specifically used to retrieve the computer programs or instructions from the memory and run the computer programs or instructions to perform the method described in the first aspect.

[0023] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program or computer instructions, which, when executed, cause an electronic device to perform the method described in the first aspect.

[0024] Fifthly, embodiments of this application provide a computer program product including a computer program or instructions, which, when executed on an electronic device, causes the electronic device to perform the method described in the first aspect. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating a teacher role recognition method provided in an embodiment of this application;

[0026] Figure 2 This is a schematic diagram of the structure of a first audio data provided in an embodiment of this application;

[0027] Figure 3 This is a schematic diagram illustrating a role merging method provided in an embodiment of this application;

[0028] Figure 4 This is a flowchart illustrating another teacher role recognition method provided in an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of the structure of a teacher role recognition device provided in an embodiment of this application;

[0030] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0031] The terms "first," "second," etc., used in the embodiments of this application are used to distinguish different objects, not to describe a specific order. "At least one" in the embodiments of this application refers to one or more, and "multiple" refers to two or more. "And / or" in the embodiments of this application describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the following three situations: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural.

[0032] In the embodiments of this application, "at least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent the following seven cases: a, b, c, a and b, a and c, b and c, a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0033] In teaching scenarios, to analyze the content taught by teachers, it is necessary to identify the teacher's role to determine the parts taught. In the process of determining the teacher's role, based on the assumption that the teacher has the longest speaking time in the teaching scenario, the audio character with the longest cumulative speaking time is identified as the teacher. However, in many cases, the teacher's speaking time in a single lesson is not necessarily the longest. For example, in some practice lessons or student presentations, a student's speaking time may exceed the teacher's speaking time. If the audio character with the longest speaking time in a single lesson is identified as the teacher, it may lead to misidentifying student characters as teacher characters, resulting in errors in subsequent analysis of the teaching content.

[0034] This application's embodiments, based on audio data from each lesson, determine the individual speaking roles corresponding to each lesson. Then, these individual speaking roles from different lessons are merged to obtain a target role set. The role with the longest cumulative speaking time in the target role set is identified as the teacher role. By integrating audio data from multiple lessons and merging roles across several lessons, the likelihood of the assumption that the teacher has the highest speaking time percentage is greatly increased, thus effectively improving the accuracy of teacher role identification.

[0035] The teacher role recognition method proposed in this application is described below. This teacher role recognition method can be executed by an electronic device, or by a device matched with the electronic device (such as a chip or processor located inside the electronic device). This application embodiment uses the execution of the teacher role recognition method by an electronic device as an example for illustration. The electronic device may include, but is not limited to, terminal devices, servers, devices in a cluster or distributed system. Terminal devices may include, but are not limited to, mobile phones, computers, laptops, etc.

[0036] Please see Figure 1 , Figure 1 This is a flowchart illustrating a teacher role identification method provided in an embodiment of this application. It mainly describes determining a target role set based on audio data from two class periods, and then identifying the role with the longest cumulative speaking time in the target role set as the teacher role. Figure 1 As shown, the teacher role recognition method may include, but is not limited to, the following steps.

[0037] S101: Obtain first audio data and second audio data, wherein the first audio data includes the audio data of the first lesson and the second audio data includes the audio data of the second lesson.

[0038] A lesson refers to a class in a teaching activity, and the audio data of a lesson refers to the classroom recording obtained by collecting the sound during the class of that lesson.

[0039] The first and second lessons are different lessons, meaning they are lessons with different class times. Taking a class lasting 40 minutes as an example, with the first lesson from 8:00 to 8:40 and the second lesson from 14:00 to 14:40, the audio data for the first lesson (i.e., the first audio data) can include the 40-minute classroom recording collected during the 8:00-8:40 time period, and the audio data for the second lesson (i.e., the second audio data) can include the 40-minute classroom recording collected during the 14:00-14:40 time period.

[0040] The electronic device can acquire first and second audio data from one or more microphones. These microphones are used to collect audio data in the classroom and can be installed in the classroom or worn by the teacher. The microphone used to collect the first audio data and the microphone used to collect the second audio data can be the same microphone or different microphones. For example, the microphone used to collect the first audio data and the microphone used to collect the second audio data can be worn by the same teacher.

[0041] Alternatively, the electronic device can acquire first and second audio data from one or more storage devices, and microphones installed in the classroom and / or carried by the teacher can transmit the acquired audio data to the storage devices. The storage device used to provide the first audio data and the storage device used to provide the second audio data can be the same storage device or different storage devices.

[0042] In one possible implementation, the acquisition environments for the first and second audio data can be the same or different. For example, the first and second audio data can be acquired in the same classroom. Since the teachers in the same class are relatively fixed, the teaching content of a specific class can be analyzed by analyzing the audio data acquired in the same classroom.

[0043] S102: Based on the voiceprint features of the single-person speaking audio frames in the first audio data, determine the first set of roles and the voiceprint features of each single-person speaking role in the first set of roles.

[0044] After acquiring the first audio data, the electronic device can segment the first audio data to obtain at least one audio segment (hereinafter referred to as F audio segments, where F is a positive integer). The categories of audio segments include single-person speech, multi-person speech, and no-person speech.

[0045] Optionally, the electronic device can input the first audio data into a pre-trained audio event recognition model to obtain the audio event category at each moment in the first audio data. The audio event categories include single-person speech, multi-person speech, and no one speaking. The electronic device can use the moment when the audio event category changes as the segmentation point to divide the first audio data into F audio segments. In this way, the audio event category is the same at each moment within the same audio segment, and the audio event category can be used as the category of the audio segment.

[0046] An audio segment can include multiple audio frames, each with the same duration, such as 1 second or some other duration. Different audio segments can have the same or different durations. That is, different audio segments can contain the same or different number of audio frames.

[0047] Among them, the single-person speech audio frames in the first audio data include all audio frames in the single-person speech audio segment corresponding to the first lesson. The single-person speech audio segment is an audio segment belonging to the category of single-person speech. The single-person speech audio segment corresponding to the first lesson includes all audio segments belonging to the category of single-person speech among F audio segments.

[0048] Taking F=4 as an example, the structural diagram of the first audio data can be shown as follows: Figure 2 As shown. Figure 2 As shown, the F audio segments are audio segment a, audio segment b, audio segment c, audio segment d, and audio segment e. Audio segments a, c, and e are all classified as solo speakers, audio segment b is classified as no speaker, and audio segment d is classified as multi-person speaker. In this case, the solo speaker audio segments corresponding to the first lesson include audio segments a, c, and e. The solo speaker audio frames in the first audio data include the audio frames of audio segment a, audio segment c, and audio segment e, namely audio frames a1-a4, audio frames c1-c3, and audio frames e1-e3.

[0049] After extracting single-person speech audio frames from the first audio data of the electronic device, these frames can be input into a pre-trained voiceprint feature extraction model to obtain the voiceprint features of each single-person speech audio frame in the first audio data. The single-person speech audio frames in the first audio data include... Figure 2 Taking audio frames a1-a4, c1-c3, and e1-e3 as examples, audio frames a1-a4, c1-c3, and e1-e3 can be input into the voiceprint feature extraction model to obtain the voiceprint features of each audio frame in audio frames a1-a4, c1-c3, and e1-e3.

[0050] The voiceprint feature of an audio frame can be an M-dimensional vector, where M is a positive integer. This application does not limit the value of M. For example, the value of M can be 256, meaning that the voiceprint feature of an audio frame is a 256-dimensional vector.

[0051] In one possible implementation, the electronic device can cluster the voiceprint features of individual speaker audio frames in the first audio data to determine a first set of roles, which includes at least one individual speaker role. Optionally, the electronic device can cluster based on the similarity of voiceprint features between individual speaker audio frames in the first audio data, obtaining Z categories, with each category representing one individual speaker role, thus obtaining the first set of roles, where Z is a positive integer. That is, dividing the individual speaker audio data in the first lesson into Z categories is equivalent to assuming that there are Z roles speaking in the first lesson. Each individual speaker audio segment in the first audio data corresponds to one category, i.e., one individual speaker role in the first set of roles. For example, ... Figure 2 The voiceprint features of audio frames a1-a4, c1-c3, and e1-e3 in the audio frame are clustered to determine that the first role set includes single-speaker role 1 and single-speaker role 2. Among them, audio segments a and c correspond to single-speaker role 1, and audio segment e corresponds to single-speaker role 2.

[0052] This application does not limit the clustering method; for example, spectral clustering or other methods can be used for clustering. Spectral clustering is a graph-based clustering method that uses the spectrum of the data (i.e., the feature vector of the voiceprint features) to perform clustering. This application does not limit the similarity measurement method; for example, cosine similarity or other similarity measurement methods can be used to calculate the similarity of voiceprint features.

[0053] In one possible implementation, the electronic device can determine the voiceprint features of each individual speech audio segment corresponding to the first lesson based on the voiceprint features of the audio frames in each individual speech audio segment corresponding to the first lesson. For example, based on Figure 2 The voiceprint features of audio frames a1-a4 are used to determine the voiceprint features of audio segment a; based on Figure 2 The voiceprint features of audio frames c1-c3 are used to determine the voiceprint features of audio segment c; based on Figure 2 The voiceprint features of mid-frequency audio segments e1-e3 are used to determine the voiceprint features of audio segment e.

[0054] Optionally, taking a single-person speech audio segment corresponding to the first lesson as an example, the electronic device can use the average value of the voiceprint features of each audio frame in the single-person speech audio segment as the voiceprint features of that single-person speech audio segment. For example, it can... Figure 2 The mean value of the voiceprint features of audio frames a1-a4 is used as the voiceprint feature of audio segment a. The voiceprint feature of an audio segment is an M-dimensional vector.

[0055] In one possible implementation, after the electronic device determines the voiceprint features of each individual speaking audio segment corresponding to the first lesson, it can determine the voiceprint features of each individual speaking role in the first role set based on the voiceprint features of the individual speaking audio segments corresponding to each individual speaking role in the first role set.

[0056] Optionally, taking a single speaker character from the first character set as an example, the electronic device can use the average of the voiceprint features of all single-speaker audio clips corresponding to that single speaker character as the voiceprint feature of that single speaker character. For example, Figure 2 The single-speaker voiceover character 1 has all single-speaker audio segments, including audio segment a and audio segment c. The average of the voiceprint features of audio segment a and audio segment c can be used as the voiceprint feature of single-speaker voiceover character 1. The voiceprint feature of a single-speaker voiceover character is an M-dimensional vector.

[0057] S103: Based on the voiceprint features of the single-person speaking audio frames in the second audio data, determine the second role set and the voiceprint features of each single-person speaking role in the second role set.

[0058] The process for determining the second set of characters can be found in the process for determining the first set of characters in S102. The process for determining the voiceprint characteristics of each individual speaking character in the second set of characters can also be found in the process for determining the voiceprint characteristics of each individual speaking character in the first set of characters in S102. It will not be repeated here.

[0059] S104: Based on the similarity of the voiceprint features of individual speaking characters between the first character set and the second character set, merge the characters in the first character set and the second character set to obtain the target character set.

[0060] In one possible implementation, the electronic device can determine the similarity of the voiceprint features of every two individual speaking characters between a first set of characters and a second set of characters, thus obtaining a first similarity set. Two individual speaking characters in the first similarity set whose similarity exceeds a similarity threshold are merged. Each individual speaking character in the first and second sets is merged at most once. Merging each individual speaking character at most once reduces the computational load during the character merging process.

[0061] For example, if the first set of roles includes solo speaker role 1 and solo speaker role 2, and the second set of roles includes solo speaker role 3 and solo speaker role 4, then the first similarity set includes four similarities: similarity a, similarity b, similarity c, and similarity d. Specifically, similarity a represents the similarity of the voiceprint features between solo speaker role 1 and solo speaker role 3; similarity b represents the similarity of the voiceprint features between solo speaker role 1 and solo speaker role 4; similarity c represents the similarity of the voiceprint features between solo speaker role 2 and solo speaker role 3; and similarity d represents the similarity of the voiceprint features between solo speaker role 2 and solo speaker role 4.

[0062] Assume that similarity a and similarity b are both greater than the similarity threshold, while similarity c and similarity d are both less than or equal to the similarity threshold. In this case, the two single-person speaking roles corresponding to similarity a (i.e., single-person speaking role 1 and single-person speaking role 3) can be merged into a new role (e.g., single-person speaking role 5). Since a single-person speaking role can be merged at most once, single-person speaking role 1 and single-person speaking role 3 cannot be merged with other roles. That is, the two single-person speaking roles corresponding to similarity b (i.e., single-person speaking role 1 and single-person speaking role 4) cannot be merged. The target role set includes: the new role, the single-person speaking role that has not been merged in the first role set (i.e., single-person speaking role 2), and the single-person speaking role that has not been merged in the second role set (i.e., single-person speaking role 4). In other words, the target role set includes single-person speaking role 5, single-person speaking role 2, and single-person speaking role 4. A diagram illustrating role merging can be found here. Figure 3 As shown.

[0063] Assume that similarity a and similarity d are both greater than the similarity threshold, while similarity b and similarity c are both less than or equal to the similarity threshold. In this case, the two single-person speaking roles corresponding to similarity a (i.e., single-person speaking role 1 and single-person speaking role 3) can be merged into a new role, and the two single-person speaking roles corresponding to similarity d (i.e., single-person speaking role 2 and single-person speaking role 4) can be merged into another new role. In this case, the target role set includes the two merged new roles. All single-person speaking roles in the first role set are merged, and all single-person speaking roles in the second role set are merged.

[0064] In one possible implementation, if the maximum similarity in the first similarity set is greater than the similarity threshold, the two single-person speaking roles corresponding to the maximum similarity in the first similarity set are merged, and the maximum similarity and the first similarity in the first similarity set are deleted; wherein, the two single-person speaking roles corresponding to the first similarity include one of the single-person speaking roles corresponding to the maximum similarity in the first similarity set; if the maximum similarity in the updated first similarity set is greater than the similarity threshold, the two single-person speaking roles corresponding to the maximum similarity in the updated first similarity set are merged, and the maximum similarity and the second similarity in the updated first similarity set are deleted, until the maximum similarity in the updated first similarity set is less than or equal to the similarity threshold; wherein, the two single-person speaking roles corresponding to the second similarity include one of the single-person speaking roles corresponding to the maximum similarity in the updated first similarity set.

[0065] In other words, merging roles in descending order of similarity prioritizes merging the most similar roles, which helps improve the accuracy of role merging.

[0066] For example, the four similarities in the first similarity set are arranged in descending order as follows: similarity a > similarity d > similarity c > similarity b, where similarity a > similarity d > similarity threshold. The electronic device can first merge the two single-person speaking roles (i.e., single-person speaking role 1 and single-person speaking role 3) corresponding to the maximum similarity (i.e., similarity a) into a new role (e.g., single-person speaking role 5).

[0067] Then, similarity 'a' and the first similarity are removed from the first similarity set. The two single-speaker roles corresponding to the first similarity include either single-speaker role 1 or single-speaker role 3. That is, the first similarity includes similarity 'b' and similarity 'c', because the two single-speaker roles corresponding to similarity 'b' (single-speaker role 1 and single-speaker role 4) include single-speaker role 1, and the two single-speaker roles corresponding to similarity 'c' (single-speaker role 2 and single-speaker role 3) include single-speaker role 3. By removing the maximum similarity and the first similarity from the first similarity set, it can be ensured that the single-speaker roles that have already been merged (i.e., single-speaker role 1 and single-speaker role 3) will not be merged again in the future, thus reducing the amount of computation in the role merging process.

[0068] The updated first similarity set only includes similarity d, and similarity d > similarity threshold. In this case, the two single-speaker roles (i.e., single-speaker role 2 and single-speaker role 4) corresponding to the maximum similarity (i.e., similarity d) in the updated first similarity set can be merged into a new role (e.g., single-speaker role 6).

[0069] Then, delete the similarity d in the updated first similarity set. Since there is no similarity left in the updated first similarity set after deleting similarity d, the merging process of the first and second role sets can be ended. Understandably, if similarity still exists in the updated first similarity set after deleting similarity d, the second similarity can be deleted. The process of deleting the second similarity in the updated first similarity set can be referenced to the process of deleting the first similarity in the unupdated first similarity set, and so on, iterating through the updated first similarity set until the maximum similarity in the updated first similarity set is less than or equal to the similarity threshold, or there is no similarity left in the updated first similarity set.

[0070] In one possible implementation, the similarity threshold can be set by default by the electronic device, or it can be set or modified by the user. For example, the similarity threshold could be an empirical value confirmed by the user.

[0071] S105: Determine the cumulative speaking time of each individual speaker in the target role set in the first and second audio data, and determine the role with the longest cumulative speaking time in the target role set as the teacher role.

[0072] In this embodiment, the electronic device obtains a set of individual speaking roles corresponding to the first and second lessons based on the first and second audio data. Then, it merges the individual speaking roles between the first and second lessons to obtain a target role set. The role with the longest cumulative speaking time in the target role set is identified as the teacher role. By combining the first and second audio data, the probability of the assumption that the teacher has the highest speaking time percentage is greatly increased after merging the roles in the two lessons. Therefore, identifying the role with the longest cumulative speaking time in the target role set as the teacher role ensures a high accuracy rate in teacher role identification. Furthermore, this embodiment clusters the voiceprint features of the individual speaking audio frames in the first and second lessons respectively, and then merges the individual speaking roles between the first and second lessons to obtain the target role set. This can be considered a cascaded voiceprint clustering. Compared to clustering the voiceprint features of the individual speaking audio frames in the first and second lessons all at once, this method has lower time and space complexity and can handle information from any number of lessons.

[0073] Please see Figure 4 , Figure 4 This is a flowchart illustrating another teacher role identification method provided in an embodiment of this application. It mainly describes determining a target role set based on audio data from three lessons, and then identifying the role with the longest cumulative speaking time in the target role set as the teacher role. Figure 4As shown, the teacher role recognition method may include, but is not limited to, the following steps.

[0074] S401: Obtain first audio data, second audio data, and third audio data; wherein, the first audio data includes the audio data of the first lesson, the second audio data includes the audio data of the second lesson, and the third audio data includes the audio data of the third lesson.

[0075] For information on how to obtain the third audio data, please refer to the methods for obtaining the first and second audio data. For details on how to obtain the first and second audio data, please refer to [link / reference needed]. Figure 1 The specific description in S101 of the corresponding embodiment will not be repeated here.

[0076] The first, second, and third periods are classes with different times. For example, the first period is from 8:00 to 8:40 on April 20, 2025, the second period is from 14:00 to 14:40 on April 20, 2025, and the third period is from 8:00 to 8:40 on April 21, 2025. Alternatively, the first period is from 8:00 to 8:40 on April 20, 2025, the second period is from 8:00 to 8:40 on April 21, 2025, and the third period is from 14:00 to 14:40 on April 20, 2025.

[0077] S402: Based on the voiceprint features of the single-person speaking audio frames in the first audio data, determine the first set of roles and the voiceprint features of each single-person speaking role in the first set of roles.

[0078] S403: Based on the voiceprint features of the single-person speaking audio frames in the second audio data, determine the second role set and the voiceprint features of each single-person speaking role in the second role set.

[0079] For details on the execution process of S402 and S403, please refer to [link / reference]. Figure 1 The specific descriptions of S102 and S103 in the corresponding embodiments will not be repeated here.

[0080] S404: Based on the voiceprint features of the single-person speech audio frames in the third audio data, determine the third role set and the voiceprint features of each single-person speech role in the third role set.

[0081] For details on the process of determining the third-party set, please refer to [link / reference]. Figure 1 The process of determining the first set of roles in S102 of the corresponding embodiment, and the process of determining the voiceprint features of each individual speaking role in the third set of roles, can be referred to the process of determining the voiceprint features of each individual speaking role in the first set of roles in S102, and will not be repeated here.

[0082] S405: Based on the similarity of the voiceprint features of individual speaking characters between the first character set and the second character set, merge the characters of the first character set and the second character set to obtain the intermediate character set and the voiceprint features of each individual speaking character in the intermediate character set.

[0083] For information on merging characters from the first and second character sets, please refer to [link / reference]. Figure 1 The specific description in S104 of the corresponding embodiment.

[0084] The intermediate role set includes the newly merged role. If the first role set contains any unmerged single-speaker roles, the intermediate role set can also include those unmerged single-speaker roles from the first role set. Similarly, if the second role set contains any unmerged single-speaker roles, the intermediate role set can also include those unmerged single-speaker roles from the second role set. The intermediate role set can include one or more new roles, where a new role is obtained by merging two single-speaker roles.

[0085] In this context, the voiceprint feature of a new character can be the average of the voiceprint features of the two merged single-speaker characters corresponding to that new character. Assuming that the new character is obtained by merging single-speaker character 1 and single-speaker character 3 from the first character set, then the average of the voiceprint features of single-speaker character 1 and single-speaker character 3 can be used as the voiceprint feature of the new character.

[0086] In one possible implementation, the electronic device can determine the similarity of the voiceprint features of every two individual speaking characters between the first set of characters and the second set of characters, thus obtaining a similarity matrix. For example, assuming the first set of characters includes n individual speaking characters and the second set of characters includes m individual speaking characters, the similarity matrix can be represented as follows:

[0087]

[0088] In the similarity matrix, This represents the similarity of voiceprint features between the i-th solo speaker in lesson A (i.e., the first lesson) and the j-th solo speaker in lesson B (i.e., the second lesson). Due to the symmetry of the similarity, this similarity is a symmetric matrix.

[0089] In one possible implementation, after determining the similarity matrix, the electronic device can sort the values ​​in the similarity matrix and merge the individual speaking roles between lesson A and lesson B in descending order of similarity. The specific merging process is as follows:

[0090] ① Set the lower half of the similarity matrix to zero to avoid double counting.

[0091] ② Take the maximum value in the similarity matrix (assuming it is ). If the maximum value is greater than the similarity threshold, then the single speaker role B in lesson B is considered to be... j And the solo speaker role A in lesson A i If the two are the same character, they are merged, and the average of their voiceprint features is calculated as the voiceprint feature of the merged character. If the maximum value in the similarity matrix is ​​less than a threshold, the merging process ends.

[0092] ③ Set the row and column values ​​of the matrix containing the single-speaker characters that have been merged in ② to zero. This ensures that merged characters will not be merged again.

[0093] ④ Repeat steps ② and ③ until no value in the similarity matrix is ​​greater than the similarity threshold;

[0094] ⑤ Add the newly merged character, the unmerged characters in lesson A (if they exist), and the unmerged characters in lesson B (if they exist) to the intermediate character set.

[0095] By combining the two sets of roles (i.e., the sets of roles corresponding to the two lessons) obtained through steps ①-⑤, a new set of roles (i.e., the intermediate set of roles) is obtained.

[0096] S406: Based on the similarity of the voiceprint features of individual speaking characters between the intermediate character set and the third character set, merge the characters in the intermediate character set and the third character set to obtain the target character set.

[0097] For details on merging roles between the middle and third role sets, please refer to [reference needed]. For details on merging roles between the first and second role sets, please refer to [reference needed].

[0098] In one possible implementation, the first, second, and third lessons are arranged in chronological order of their class times. For example, the first lesson might be from 8:00 AM to 8:40 AM on April 20, 2025, the second lesson from 2:00 PM to 2:40 PM on the same day, and the third lesson from 8:00 AM to 8:40 AM on April 21, 2025. In other words, the roles for the first two lessons are merged according to their class times. Then, the resulting set of roles is merged with the third lesson using the same process. Because the teachers' voices and recording environments are similar in consecutive lessons, merging them according to their class times reduces interference and improves the accuracy of role merging. For example, the voices of the same person recorded on different days might differ, resulting in lower similarity in voiceprint features, and they might not be identified as the same role, thus failing to be merged.

[0099] If the electronic device acquires audio data from more than three lessons, the same process can be repeated to obtain the target character set. For example, if the electronic device acquires audio data from five lessons, it can first merge the characters from the first two lessons according to their class time order, resulting in an intermediate character set ① and the voiceprint features of each character in intermediate character set ①. Then, intermediate character set ① is merged with the third lesson in the same process, resulting in an intermediate character set ② and the voiceprint features of each character in intermediate character set ②. Next, intermediate character set ② is merged with the fourth lesson in the same process, resulting in an intermediate character set ③ and the voiceprint features of each character in intermediate character set ③. Finally, intermediate character set ③ is merged with the fifth lesson in the same process to obtain the target character set.

[0100] S407: Determine the cumulative speaking time of each individual speaking role in the target role set in the audio dataset, and determine the role with the longest cumulative speaking time in the target role set as the teacher role. The audio dataset includes the first audio data, the second audio data, and the third audio data.

[0101] In this embodiment, the electronic device obtains a set of individual speaking roles corresponding to each of the three lessons based on audio data. Then, it merges these roles according to the order in which the lessons are taught, ultimately obtaining a target role set. The role with the longest cumulative speaking time in the target role set is identified as the teacher role. By integrating audio data from multiple lessons and merging roles across several lessons, the probability of the assumption that the teacher has the highest speaking time percentage is greatly increased. Therefore, identifying the role with the longest cumulative speaking time in the target role set as the teacher role ensures a high accuracy rate in teacher role identification.

[0102] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a teacher role recognition device provided in an embodiment of this application. Figure 5 As shown, the teacher role recognition device 50 includes an acquisition unit 501 and a processing unit 502. The teacher role recognition device 50 can perform the relevant steps of the electronic device in the aforementioned method embodiments.

[0103] The acquisition unit 501 is used to acquire first audio data and second audio data. The first audio data includes the audio data of the first lesson, and the second audio data includes the audio data of the second lesson.

[0104] Processing unit 502 is configured to: determine a first set of roles and the voiceprint features of each individual speaking role in the first set of roles based on the voiceprint features of the individual speaking audio frames in the first audio data; determine a second set of roles and the voiceprint features of each individual speaking role in the second set of roles based on the voiceprint features of the individual speaking audio frames in the second audio data; merge the roles in the first set of roles and the second set of roles based on the similarity of the voiceprint features of the individual speaking roles between the first set of roles and the second set of roles to obtain a target set of roles; determine the cumulative speaking time of each individual speaking role in the target set in the first audio data and the second audio data, and determine the role with the longest cumulative speaking time in the target set as the teacher role.

[0105] Specifically, in this case, the operations performed by the acquisition unit 501 and the processing unit 502 can be referred to Figure 1 The descriptions of the electronic devices in the corresponding embodiments will not be repeated here.

[0106] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device 60 provided in an embodiment of this application, which can be used to implement the functions of the electronic device in the above method embodiments. The electronic device 60 may include at least one processor 601. Optionally, the electronic device 60 may further include a communication interface 602. Optionally, the electronic device 60 may further include a memory 603. Figure 6 The components shown by the dashed lines are optional components, meaning that electronic device 60 may not include them. Figure 6 The component shown by the dashed line.

[0107] At least one processor 601, communication interface 602, and memory 603 can be connected via bus 604 or other means. The bus is... Figure 6 The connections between other components are shown in bold lines only and are not intended to be limiting. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, Figure 6The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0108] The coupling in this application embodiment is an indirect coupling or communication connection between devices, units, or modules, which can be electrical, mechanical, or other forms, used for information exchange between devices, units, or modules. This application embodiment does not limit the specific connection medium between the at least one processor 601, communication interface 602, and memory 603.

[0109] The memory 603 may include read-only memory and random access memory, and provides instructions and data to at least one processor 601. A portion of the memory 603 may also include non-volatile random access memory.

[0110] At least one processor 601 may be a Central Processing Unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor; optionally, at least one processor 601 may also be any conventional processor.

[0111] In one example, when the electronic device adopts Figure 6 When in the form shown, Figure 6 At least one processor 601 in the device can execute the method executed by the electronic device in any of the above method embodiments.

[0112] In one possible implementation, memory 603 is used to store computer programs or instructions; at least one processor 601 is used to invoke the computer programs or instructions stored in memory 603 for execution. Figure 1 The steps performed by the electronic device in the corresponding embodiment.

[0113] In the embodiments of this application, the methods provided in the embodiments of this application can be implemented by running a computer program (including program code) capable of performing the steps involved in the above-described methods on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a CPU, random access memory (RAM), and read-only memory (ROM). The computer program can be recorded on, for example, a computer-readable recording medium, loaded into an electronic device through the computer-readable recording medium, and run therein.

[0114] Based on the same inventive concept, the principles and beneficial effects of the devices and electronic equipment provided in the embodiments of this application in solving the problem are similar to those of the electronic equipment in the method embodiments of this application in solving the problem. Please refer to the principles and beneficial effects of the method embodiments. For the sake of brevity, they will not be repeated here.

[0115] This application also provides a computer-readable storage medium storing a computer program or computer instructions, which is adapted to be loaded by an electronic device and execute the method provided in the above-described method embodiments.

[0116] This application also provides a computer program product containing a computer program or instructions, which, when run on an electronic device, causes the electronic device to perform the method provided in the above-described method embodiments.

[0117] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for devices and products applied to or integrated into a chip, all modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs running on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits. For devices and products applied to or integrated into a chip module, all modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The implementation is achieved through a software program that runs on a processor integrated within the chip module. The remaining modules / units (if any) can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into electronic devices, each of their modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components within the electronic device. Alternatively, at least some modules / units can be implemented using a software program that runs on a processor integrated within the electronic device, while the remaining modules / units (if any) can be implemented using hardware methods such as circuits.

[0118] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0119] In the above embodiments, the descriptions of each embodiment have their own emphasis, and any multiple embodiments can be used in combination. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0120] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0121] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0122] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by program instructions and related hardware. The program instructions can be stored in a computer-readable storage medium, which may include: flash drive, ROM, RAM, disk or optical disk, etc.

[0123] The above-disclosed embodiments are merely one example of this application and only a part of the embodiments of this application. They should not be construed as limiting the scope of this application.

Claims

1. A method for identifying teacher roles, characterized in that, The method includes: Acquire first audio data and second audio data, wherein the first audio data includes audio data of the first lesson and the second audio data includes audio data of the second lesson; Based on the voiceprint features of the single-person speaking audio frames in the first audio data, the first set of roles and the voiceprint features of each single-person speaking role in the first set of roles are determined. Based on the voiceprint features of the single-person speech audio frames in the second audio data, the second set of roles and the voiceprint features of each single-person speech role in the second set of roles are determined. Based on the similarity of the voiceprint features of individual speaking characters between the first character set and the second character set, the first character set and the second character set are merged to obtain the target character set; Determine the cumulative speaking time of each individual speaker in the target role set in the first audio data and the second audio data, and determine the role with the longest cumulative speaking time in the target role set as the teacher role.

2. The method according to claim 1, characterized in that, The method further includes: Obtain third audio data, which includes audio data from the third lesson; Based on the voiceprint features of the single-person speech audio frames in the third audio data, a third set of roles and the voiceprint features of each single-person speech role in the third set of roles are determined. The step involves merging the voiceprint features of individual speaking characters between the first and second character sets based on their similarity to obtain a target character set, including: Based on the similarity of the voiceprint features of individual speaking characters between the first character set and the second character set, the first character set and the second character set are merged to obtain an intermediate character set and the voiceprint features of each individual speaking character in the intermediate character set. Based on the similarity of the voiceprint features of individual speaking characters between the intermediate character set and the third character set, the intermediate character set and the third character set are merged to obtain the target character set. Determining the cumulative speaking time of each individual speaking character in the target character set in the first audio data and the second audio data includes: determining the cumulative speaking time of each individual speaking character in the target character set in the audio dataset, wherein the audio dataset includes the first audio data, the second audio data and the third audio data.

3. The method according to claim 2, characterized in that, The first lesson, the second lesson, and the third lesson are arranged in order of their class times from front to back.

4. The method according to any one of claims 1-3, characterized in that, The step of merging roles in the first and second role sets based on the similarity of voiceprint features of individual speaking roles between the first and second role sets includes: Determine the similarity of the voiceprint features of every two individual speaking characters between the first character set and the second character set to obtain the first similarity set; Merge two single-person speaking roles in the first similarity set whose similarity is greater than the similarity threshold. Each single-person speaking role in the first role set and the second role set is merged at most once.

5. The method according to claim 4, characterized in that, The step of merging two single-person speaking roles in the first similarity set whose similarity is greater than the similarity threshold includes: If the maximum similarity in the first similarity set is greater than the similarity threshold, the two single-person speaking roles corresponding to the maximum similarity in the first similarity set are merged, and the maximum similarity and the first similarity in the first similarity set are deleted; wherein, the two single-person speaking roles corresponding to the first similarity include one of the single-person speaking roles corresponding to the maximum similarity in the first similarity set; If the maximum similarity in the updated first similarity set is greater than the similarity threshold, the two single-person speaking roles corresponding to the maximum similarity in the updated first similarity set are merged, and the maximum similarity and the second similarity in the updated first similarity set are deleted, until the maximum similarity in the updated first similarity set is less than or equal to the similarity threshold; wherein, the two single-person speaking roles corresponding to the second similarity include one of the single-person speaking roles corresponding to the maximum similarity in the updated first similarity set.

6. The method according to any one of claims 1-3, characterized in that, The method further includes: The first audio data is segmented to obtain at least one audio segment. The categories of audio segments include single person speaking, multiple people speaking, and no one speaking. In the at least one audio segment, a single-person speaking audio segment corresponding to the first lesson is determined, wherein the single-person speaking audio segment is an audio segment belonging to the category of single-person speaking; wherein, the single-person speaking audio frame in the first audio data includes the audio frame in the single-person speaking audio segment corresponding to the first lesson.

7. The method according to claim 6, characterized in that, The method further includes: Determine the individual speaking audio segments corresponding to each individual speaker in the first lesson section and their corresponding individual speaking roles in the first role set; The step of determining the voiceprint features of each individual speaking character in the first character set based on the voiceprint features of the individual speaking audio frames in the first audio data includes: Based on the voiceprint features of the audio frames in each individual speech audio segment corresponding to the first lesson, the voiceprint features of each individual speech audio segment corresponding to the first lesson are determined. Based on the voiceprint features of the individual speaking audio segments corresponding to each individual speaking character in the first character set, the voiceprint features of each individual speaking character in the first character set are determined.

8. The method according to any one of claims 1-3, characterized in that, The step of determining the first set of roles based on the voiceprint features of individual speech audio frames in the first audio data includes: Cluster the voiceprint features of single-person speech audio frames in the first audio data to determine the first set of roles.

9. An electronic device, characterized in that, It includes at least one processor, said at least one processor being configured to run a computer program or instructions to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or computer instructions that, when executed by a processor, cause the method of any one of claims 1-8 to be performed.

Images