Conference audio data processing method, device, equipment, medium and program product

Abstract

The invention provides a conference audio data processing method which can be applied to the technical field of audio processing and can also be applied to the field of financial science and technology. The conference audio data processing method comprises the steps that audio data are acquired, the audio data comprise N pieces of voice data, and N is a positive integer larger than or equal to 2; determining an acquisition path of the audio data, wherein the acquisition path represents an acquisition mode of the audio data; according to the acquisition path, a target matching strategy corresponding to the audio data is determined, and the target matching strategy is used for determining a spokesman corresponding to each piece of voice data; according to the spokesman and the voice data, target text information corresponding to the audio data is determined, and the target text information records text information speaking by the spokesman. The invention further provides a conference audio data processing device and equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of audio processing, and more specifically to a conference audio data processing method, device, equipment, medium and program product. Background Art

[0002] In a meeting, the statements and opinions of the speakers are usually very important information. Therefore, compiling the full text of the meeting and the minutes is an important means to ensure the accurate communication of meeting information and follow-up.

[0003] Currently, due to the fast speaking speed of users, recorders usually record and manually record part of the user's speech information during the meeting, and after the meeting, they repeatedly listen to the recording to complete the user's speech information, resulting in low efficiency in generating meeting records. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a conference audio data processing method, apparatus, device, medium and program product.

[0005] According to the first aspect of the present disclosure, a method for processing conference audio data is provided, including: acquiring audio data, the audio data including N voice data, where N is a positive integer greater than or equal to 2; determining an acquisition path for the audio data, the acquisition path representing a method for acquiring the audio data; determining a target matching strategy corresponding to the audio data based on the acquisition path, the target matching strategy being used to determine a speaker corresponding to each voice data; determining target text information corresponding to the audio data based on the speakers and the voice data, the target text information recording text information of speeches by multiple speakers.

[0006] According to an embodiment of the present disclosure, when audio data is not acquired in real time, the method includes: classifying voiceprint information of N voice data to obtain M voiceprint information, where M is a positive integer less than or equal to N; and determining the speaker to whom each voice data belongs based on the M voiceprint information.

[0007] According to an embodiment of the present disclosure, when audio data is acquired in real time, the method includes: in response to acquiring the nth voice data, determining the voiceprint information of the nth voice data, where n belongs to N and n is a positive integer greater than 3; matching the voiceprint information of the nth voice data with the voiceprint information of the previous n-1 voice data respectively; when the voiceprint information of the nth voice data is consistent with the voiceprint information of the nmth voice data, determining the speaker to whom the nmth voice data belongs as the speaker to whom the nth voice data belongs.

[0008] According to an embodiment of the present disclosure, the method also includes: in response to obtaining the nm-th voice data, determining the voiceprint information of the nm-th voice data; based on a first database, determining the speaker corresponding to the nm-th voiceprint information, the first database including multiple sample voiceprints and speakers corresponding to the sample voiceprints.

[0009] According to an embodiment of the present disclosure, based on the first database, determining the speaker corresponding to the nm-th voiceprint information includes: obtaining association information with the nm-th voice data; when the association information cannot determine the speaker to which the nm-th voice data belongs, determining a second database according to the association information, the first database including the second database; determining the speaker corresponding to the nm-th voiceprint information according to the second database.

[0010] According to an embodiment of the present disclosure, acquiring audio data includes: segmenting N speech data from the audio data according to preset segmentation parameters.

[0011] According to an embodiment of the present disclosure, the method also includes: obtaining target parameters of an audio data conference, the target parameters representing the theme of the conference; determining a third database based on the target parameters, the third database including multiple speakers and sample voiceprints corresponding to each speaker; and determining the speaker to whom each voice data belongs based on the third database.

[0012] According to an embodiment of the present disclosure, target text information corresponding to audio data is determined based on a spokesperson and voice data, including: performing voice recognition on multiple voice data respectively to obtain text information corresponding to each voice data; and determining target text information corresponding to audio data based on a spokesperson and text information.

[0013] The second aspect of the present disclosure provides a conference audio data processing device, including: the device includes: an acquisition module, used to acquire audio data, the audio data includes N voice data, N is a positive integer greater than or equal to 2; a first determination module, used to determine the acquisition path of the audio data, the acquisition path represents the acquisition method of the audio data; a second determination module, used to determine the target matching strategy corresponding to the audio data according to the acquisition path, the target matching strategy is used to determine the speaker corresponding to each voice data; a third determination module, used to determine the target text information corresponding to the audio data according to the speaker and the voice data, the target text information records the text information of multiple speakers' speeches.

[0014] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0015] The fourth aspect of the present disclosure further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0016] The fifth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above contents and other purposes, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0018] Figure 1 The application scenario diagram of the conference audio data processing method, apparatus, device, medium and program product according to the embodiment of the present disclosure is schematically shown;

[0019] Figure 2 A flowchart of a conference audio data processing method according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 3 One of the flow charts of the method for determining the speaker of voice data according to the target matching strategy according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 4 Schematically shows a second flowchart of a method for determining a voice data speaker according to a target matching strategy according to an embodiment of the present disclosure;

[0022] Figure 5 One of the flow charts of the method for determining a speaker of speech data according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 6 The second flowchart of the method for determining a speaker of speech data according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 7 The third flowchart of the method for determining the speaker of speech data according to the embodiment of the present disclosure is schematically shown;

[0025] Figure 8 A flowchart of a method for determining target text information corresponding to audio data according to an embodiment of the present disclosure is schematically shown;

[0026] Fig. 9 A structural block diagram of a conference audio data processing device according to an embodiment of the present disclosure is schematically shown; and

[0027] Fig.10 A block diagram of an electronic device suitable for implementing a conference audio data processing method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0029] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0030] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0031] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0032] It should be noted that the method and device for processing conference audio data disclosed in the present invention can be used for conference audio data processing in the field of financial technology, and can also be used for conference audio data processing in any field other than the field of financial technology. The present invention does not limit the application field of the method and device for processing conference audio data.

[0033] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0034] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0035] The embodiments of the present disclosure provide a conference audio data processing method, device, equipment, medium and program product. Before introducing the technical solution provided by the embodiments of the present disclosure, the related technologies involved in the present disclosure are first described.

[0036] In a meeting, the statements and opinions of the speakers are usually very important information. Therefore, compiling the full text of the meeting and the minutes is an important means to ensure the accurate communication of meeting information and follow-up.

[0037] Currently, due to the fast speaking speed of users, recorders usually record and manually record part of the user's speech information during the meeting, and after the meeting, they repeatedly listen to the recording to complete the user's speech information, resulting in low efficiency in generating meeting records.

[0038] Before further describing the embodiments of the present disclosure in detail, the nouns and terms involved in the embodiments of the present disclosure are described. The nouns and terms involved in the embodiments of the present disclosure are subject to the following interpretations.

[0039] Speech recognition technology, also known as automatic speech recognition (ASR), is the process of converting speech information in various audios into text.

[0040] Speech segmentation: the process of decomposing a continuous speech signal into discrete speech segments. Rule-based method: the speech signal is divided into different segments according to certain rules (such as the start time, end time, or fixed length of the clause in this method).

[0041] An embodiment of the present disclosure provides a method for processing conference audio data, the method comprising: acquiring audio data, the audio data comprising N voice data, where N is a positive integer greater than or equal to 2; determining an acquisition path for the audio data, the acquisition path representing a method for acquiring the audio data; determining a target matching strategy corresponding to the audio data based on the acquisition path, the target matching strategy being used to determine a speaker corresponding to each voice data; determining target text information corresponding to the audio data based on the speakers and the voice data, the target text information recording text information of speeches by multiple speakers.

[0042] Figure 1 The application scenario diagram of the conference audio data processing method, apparatus, device, medium and program product according to the embodiments of the present disclosure is schematically shown.

[0043] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0044] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0045] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0046] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0047] It should be noted that the conference audio data processing method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the conference audio data processing device provided in the embodiment of the present disclosure can generally be set in the server 105. The conference audio data processing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the conference audio data processing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0048] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0049] The following will be based on Figure 1 The scene described by Figure 2~Figure 8 The conference audio data processing method of the disclosed embodiment is described in detail.

[0050] Figure 2 The flowchart of the conference audio data processing method according to the embodiment of the present disclosure is schematically shown.

[0051] like Figure 2 As shown, the conference audio data processing method of this embodiment includes operations S210 to S240.

[0052] In operation S210 , audio data is acquired, where the audio data includes N voice data, where N is a positive integer greater than or equal to 2.

[0053] Exemplarily, the audio data may include audio content composed of multiple user voice data. For example, a conference recording attended by multiple users, in which some users spoke in the conference to form voice data. The audio data here may be audio data acquired in real time or audio data that has been recorded.

[0054] Record the audio before the meeting. To ensure the quality and effect of subsequent voice recognition and voiceprint comparison, try to use a professional microphone when recording the audio.

[0055] You can perform noise reduction, echo removal, voice enhancement and other operations on the conference audio to ensure the subsequent processing effect. For example, you can use MFCC noise reduction, the formula is as follows:

[0056] The voice signals from the two aligned sources are then differentially calculated to remove the common ambient sound. The differential calculation formula is: [d(t) = x(t) - y(t)] where (d) is the differential signal, (x) and (y) are the aligned microphone signals.

[0057] In operation S220, an acquisition path of the audio data is determined, where the acquisition path represents a manner of acquiring the audio data.

[0058] Exemplarily, the acquisition path may be that the audio acquisition device (e.g., a microphone) directly outputs the audio data after acquiring it in real time, indicating that the audio data is acquired in real time. The acquisition path may also be that the audio acquisition device (e.g., a voice recorder) acquires the audio data and stores it as an audio file, indicating that the audio data is processed offline (non-real time). The acquisition method of the audio data can be determined based on the way the electronic device acquires the audio data (e.g., reading the audio data from the audio acquisition device, or reading the audio data file from the file library).

[0059] In operation S230, a target matching strategy corresponding to the audio data is determined according to the acquisition path, and the target matching strategy is used to determine a speaker corresponding to each voice data.

[0060] Exemplarily, different processing methods (target matching strategies) are used for real-time audio data and offline audio data to determine the identity information of the speaker corresponding to each voice data in the audio data.

[0061] In operation S240, target text information corresponding to the audio data is determined according to the speakers and the voice data, and the target text information records text information of speeches of multiple speakers.

[0062] Exemplarily, each voice data is transcribed in chronological order to form text information corresponding to each voice data, and based on the identity information and text information of the speaker corresponding to each voice data, a conference transcript content (target text information) about the audio data is formed.

[0063] For example, in an online meeting, during real-time recording, user A spoke first, followed by user B, and finally user C. Then, the target matching strategy corresponding to the real-time audio data is used to determine the identities of users A, B, and C based on their voice data during their speeches. The speakers corresponding to users A, B, and C are Zhang San, Li Si, and Wang Wu, respectively. At the same time, the voice data of users A, B, and C are recognized by voice recognition to obtain text information. The text information is matched with the speaker information to obtain the text record of the meeting.

[0064] It can be understood that by automatically acquiring the voice data of different speakers in the conference audio data, different target matching strategies are adopted according to different acquisition paths to automatically identify the identity information and text information of the speaker corresponding to each voice data, so as to obtain the complete record content of the meeting (target text information), automatically generate the speech information of the meeting, avoid users manually listening to the conference recording repeatedly to record the user's speech information, and greatly improve the efficiency of generating meeting records

[0065] Figure 3 One of the flowcharts of the method for determining the speaker of voice data according to the target matching strategy according to an embodiment of the present disclosure is schematically shown.

[0066] As described above, in the case where the audio data is acquired in non-real time, in operation S230, a target matching strategy corresponding to the audio data is determined according to the acquisition path, and the target matching strategy is used to determine the speaker corresponding to each voice data. In one possible implementation, Figure 3 As shown, the operation may further include operations S310 to S320.

[0067] In operation S310, the voiceprint information of N speech data is classified to obtain M voiceprint information, where M is a positive integer less than or equal to N.

[0068] In operation S320, the speaker to which each voice data belongs is determined according to the M voiceprint information.

[0069] Exemplarily, in the case of non-real-time (offline) acquisition of conference audio data, the voice data with the same voiceprint information in the audio data can be merged in advance, and then the merged voiceprint information can be confirmed by the speaker. For example, the audio data of the conference is imported, and there are 15 voice data in the audio data. First, the voiceprint information of the 15 voice data is obtained respectively, and the voiceprints with the same voiceprint information in the 15 voiceprint information are classified into one category, and finally 3 categories of voiceprint information are obtained. It can be determined that the multiple voice data in each category of voiceprint information are the voices corresponding to the same speaker. The 3 categories of voiceprint information are matched with the sample voiceprints in the first database respectively, and the identity information of the speaker corresponding to each category of voiceprint information is determined. Among them, the first database pre-stores the sample voiceprints and the identity information of the speaker corresponding to the sample voiceprints.

[0070] It is understandable that merging voices with similar voiceprints in advance can reduce the number of subsequent voice data speaker matches and improve work efficiency.

[0071] Figure 4 The second flowchart of the method for determining the speaker of voice data according to the target matching strategy according to an embodiment of the present disclosure is schematically shown.

[0072] As described above, in the case where the audio data is acquired in real time, in operation S230, a target matching strategy corresponding to the audio data is determined according to the acquisition path, and the target matching strategy is used to determine the speaker corresponding to each voice data. In another possible implementation, such as Figure 4 As shown, the operation may further include operation S410 to operation S450.

[0073] In operation S410, in response to acquiring nth voice data, voiceprint information of the nth voice data is determined, where n belongs to N and n is a positive integer greater than 3.

[0074] In operation S420, the voiceprint information of the nth voice data is matched with the voiceprint information of the first n-1 voice data respectively.

[0075] In operation S430, it is determined whether the voiceprint information of the nth voice data is consistent with the voiceprint information of the nmth voice data.

[0076] Here, nm is greater than 1 and less than n-1.

[0077] In operation S440, when the voiceprint information of the nth voice data is consistent with the voiceprint information of the nmth voice data, the speaker to whom the nmth voice data belongs is determined as the speaker to whom the nth voice data belongs.

[0078] In operation S450, a speaker corresponding to the nth voice data is determined based on the first database. The first database includes a plurality of sample voice prints and speakers corresponding to the sample voice prints.

[0079] Exemplarily, when audio data is collected in real time, the speaker identity information of each voice data collected is determined. For example, when the number of collected voice data is greater than 3, the first voice data determines that the speaker is Zhang San, and the second voice data determines that the speaker is Li Si. When the third voice data is obtained, the voiceprint information of the third voice data is extracted, and the voiceprint information of the third voice data is first matched with the voiceprint information of the first voice data and the second voice data respectively. If the voiceprint information of the third voice data matches the voiceprint information of the first voice data, the speaker of the third voice data is Zhang San. If the voiceprint information of the third voice data does not match the voiceprint information of the first voice data and the second voice data, the voiceprint information of the third voice data is matched with the sample voiceprint in the first database to determine the identity information of the speaker corresponding to the voiceprint information of the third voice data.

[0080] It is understandable that when collecting conference audio data in real time, when the number of voice data exceeds 3, the voiceprint of the voice data is matched with the voice data of the person who has spoken. If the match is consistent, the speaker of the current voice data can be obtained based on the speaker corresponding to the voice data of the person who has spoken. By matching the current voice data with the voice data of the person who has spoken in advance, if the match is consistent, the speaker of the current voice can be quickly determined, thereby improving work efficiency.

[0081] Figure 5 One of the flowcharts of the method for determining a speaker of speech data according to an embodiment of the present disclosure is schematically shown.

[0082] As described above, when the audio data is real-time or non-real-time, the speaker corresponding to each voice data is determined. In one possible implementation, Figure 5 As shown, the operation may further include operation S510 to operation S520.

[0083] In operation S510, in response to acquiring the nm-th voice data, voiceprint information of the nm-th voice data is determined.

[0084] In operation S520, a speaker corresponding to the nm-th voiceprint information is determined based on a first database, where the first database includes a plurality of sample voiceprints and speakers corresponding to the sample voiceprints.

[0085] For example, when audio data is collected in real time, the speaker identity information of each voice data collected will be determined. Then, the voiceprint information of the current voice data can be extracted first, and the voiceprint information of the current voice data can be matched with the sample voiceprint in the first database. When the voiceprint information of the current voice data matches the sample voiceprint in the first database, the speaker identity information corresponding to the current voice data can be determined.

[0086] In the case where the audio data is collected in non-real time, after obtaining multiple categories of voiceprint information, for each category of voiceprint information, each category of voiceprint information is matched with the sample voiceprint in the first database, and when the voiceprint information of the current voice data matches the sample voiceprint in the first database, the identity information of the speaker corresponding to each voice data can be determined.

[0087] Among them, the first database can be the voiceprint information of specific people (inside the group, or a small range of participants) collected in advance, usually by collecting high-quality audio of a certain length, converting it into a digital signal, extracting acoustic feature information (voiceprint) from the audio, and storing the feature information.

[0088] Figure 6The second flowchart of the method for determining the speaker of speech data according to an embodiment of the present disclosure is schematically shown.

[0089] As described above, when the audio data is real-time or non-real-time, the speaker corresponding to each voice data is determined. In another possible implementation, such as Figure 6 As shown, the operation may further include operation S610 to operation S650.

[0090] In operation S610, information associated with nm-th voice data is acquired.

[0091] In operation S620, it is determined whether the associated information can determine the speaker to whom the nm-th voice data belongs.

[0092] In operation S630, when the association information can determine the speaker to which the nm-th voice data belongs, the speaker corresponding to the nm-th voiceprint information is determined according to the association information.

[0093] In operation S640, when the association information cannot determine the speaker to which the nm-th voice data belongs, a second database is determined according to the association information, and the first database includes the second database.

[0094] In operation S650, a speaker corresponding to the nm-th voiceprint information is determined according to the second database.

[0095] For example, in order to improve the efficiency of voiceprint matching, after collecting voiceprints, the voiceprints are initially divided into different databases (second database). The purpose of dividing the databases is to avoid the situation where the same audio segment is used to match voiceprints in too large a range, which leads to slow query and easy matching errors. The databases can be divided according to the scope of departments, groups, groups, etc. Each person's voiceprint information is divided into different databases according to the tree structure. For example, Zhang San's voiceprint corresponds to (group database id, business department database id, second-level department database id, first-level department database id, and whole group database id).

[0096] The associated information may be image information of the speaker of the voice data, or content information of the voice speech.

[0097] In the case where audio data is collected in real time, the speaker identity information of each voice data collected will be determined. The image information containing the speaker of the current voice data can be captured by a camera, and the image information can be analyzed to obtain the identity information of the current voice speaker, thereby determining the identity information of the speaker of the current voice data. If the analysis of the image information cannot obtain the identity information of the current voice speaker, the content information of the voice data is analyzed to obtain the organizational information of the current voice speaker (for example, the department to which he belongs). The voiceprint database (second database) of the department is determined based on the organizational information. The voiceprint information of the current voice data is matched with the sample voiceprint in the second database. If the voiceprint information of the current voice data matches the sample voiceprint in the second database, the identity information of the speaker corresponding to the current voice data can be determined.

[0098] In the case where the audio data is collected in non-real time, after obtaining multiple categories of voiceprint information. For each category of voiceprint information, the same method as above can also be used to determine the organizational information of the speaker corresponding to each category of voiceprint, and determine the second database. Each category of voiceprint information is matched with the sample voiceprint in the second database respectively. When the voiceprint information of the current voice data matches the sample voiceprint in the second database, the identity information of the speaker corresponding to each voice data can be determined.

[0099] It is understandable that when voiceprint matching is performed based on a database, the organization information to which the speaker belongs can be pre-determined based on the associated information of the voice data. Based on the organization information, the database can be narrowed down to achieve rapid search for voiceprint matching.

[0100] As described above, in operation S210, audio data is acquired. In one practicable manner, the operation may further include: segmenting N speech data from the audio data according to a preset segmentation parameter.

[0101] Exemplarily, audio data can be segmented according to the time interval of voice data. An audio segmentation tool can be used to segment the audio according to the standard of intervals of more than 3s. The segmented original audio is a complete recording audio file of a meeting. The segmentation result is the audio (voice data) of the clause. The segmentation is based on the start time and end time of the previous clause, and the segmentation is performed according to the start, end or calculated offset.

[0102] Figure 7 The third flowchart of the method for determining the speaker of speech data according to an embodiment of the present disclosure is schematically shown.

[0103] As described above, when the audio data is real-time or non-real-time, the speaker corresponding to each voice data is determined. In another possible implementation, Figure 7As shown, the operation may further include operation S710 to operation S730.

[0104] In operation S710, a target parameter of an audio data conference is acquired, where the target parameter represents a theme of the conference.

[0105] In operation S720, a third database is determined according to the target parameter, where the third database includes a plurality of speakers and a sample voiceprint corresponding to each speaker.

[0106] In operation S730, a speaker to which each voice data belongs is determined based on the third database.

[0107] Exemplarily, if the theme of the meeting is known, the third database can be pre-determined according to the theme of the meeting. For example, if the theme of the meeting is a technical seminar for product A, the technicians and managers associated with product A can be selected as pre-selected personnel for the third database, and the voiceprint information of these pre-selected personnel can be selected to temporarily form the third database. When user A speaks, the voiceprint information of user A's voice data is extracted and matched with the voiceprint information in the third database. If the match is consistent, the identity information of user A can be obtained. If the match is inconsistent, the second database can be searched according to the associated information of user A. If the match is still inconsistent, the match can be performed in the first database.

[0108] For example, you can also find the list of participants of this meeting from the group's voiceprint library based on the actual participants of the meeting, define the scope of the participants (the default options are group, first-level department, second-level department, etc.), and use this range as the scope of voiceprint matching for subsequent audio files.

[0109] It is understandable that by determining the third database according to the conference topic, the search scope can be reduced during voiceprint search to increase the response speed and reduce the processing time for speaker identification.

[0110] In some embodiments, if the first database is not searched, the voice is registered and registered as a temporary speaker and numbered, and the speaker of this voice is marked with this number to facilitate matching the identity of other sentences in the process of confirming the speaker; if the first database is not searched and the registration is not successful, this voice data may be due to the fact that the acoustic information is not rich enough due to speaking too short, the audio quality is too low due to being too far away from the microphone, and the voiceprint information cannot be successfully matched, and it is marked as pending.

[0111] For the speech to be processed, acoustic clustering is used for matching. In the clustering method, the labeled (determined speaker) speech data and the audio clip to be processed are used as clustering inputs to analyze the characteristics of the speech data to be processed to ensure that the audio to be processed can be efficiently attributed to the corresponding speaker. Therefore, through such combined technical processing, the actual speaker information of each speech in the conference audio can be quickly and accurately identified in large-scale audio data.

[0112] Figure 8 The flowchart of the method for determining target text information corresponding to audio data according to an embodiment of the present disclosure is schematically shown.

[0113] As described above, in operation S240, target text information corresponding to the audio data is determined based on the speaker and the voice data. In one possible implementation, Figure 8 As shown, the operation may further include operation S810 to operation S820.

[0114] In operation S810, voice recognition is performed on a plurality of voice data respectively to obtain text information corresponding to each voice data.

[0115] In operation S820, target text information corresponding to the audio data is determined according to the speaker and the text information.

[0116] Exemplarily, the ASR service may be called to transcribe the voice data, and after the transcription, the start time, end time, duration of the clause, clause text and other contents of the clause may be obtained, and the response data may be recorded in a database.

[0117] The speaker information and text information corresponding to the voice data are assembled. The assembly process is to assemble each text segment, the relative time of the voice of the text in the conference audio (the start time and end time of the voice), as well as the speaker, the number of words in the clause, etc., to form a full-text report of the meeting. In the report, a complete report of "who said what at what time" can be formed based on the relative time of the clause, the transcription result of the clause, the speaker of the clause and other information. On this basis, the full-text information can also be used to intelligently generate meeting minutes, meeting summaries and other information, which is more convenient for summarization and recording.

[0118] Based on the above conference audio data processing method, the present disclosure also provides a conference audio data processing device. Fig. 9 The device is described in detail.

[0119] Fig. 9 The structure block diagram of the conference audio data processing device according to the embodiment of the present disclosure is schematically shown.

[0120] like Fig. 9As shown, the conference audio data processing device 900 of this embodiment includes an acquisition module 910 , a first determination module 920 and a second determination module 940 .

[0121] The acquisition module 910 is used to acquire audio data, and the audio data includes N voice data, where N is a positive integer greater than or equal to 2. In one embodiment, the acquisition module 910 can be used to perform the operation S210 described above, which will not be described in detail here.

[0122] The first determination module 920 is used to determine an acquisition path of the audio data, where the acquisition path represents a method of acquiring the audio data. In one embodiment, the first determination module 920 may be used to perform the operation S220 described above, which will not be described in detail herein.

[0123] The second determination module 930 is used to determine the target matching strategy corresponding to the audio data according to the acquisition path, and the target matching strategy is used to determine the speaker corresponding to each voice data. In one embodiment, the second determination module 930 can be used to perform the operation S230 described above, which will not be repeated here.

[0124] The third determination module 940 is used to determine target text information corresponding to the audio data according to the speaker and the voice data, and the target text information records the text information of the speeches of multiple speakers. In one embodiment, the third determination module 940 can be used to perform the operation S240 described above, which will not be repeated here.

[0125] According to an embodiment of the present disclosure, any multiple modules of the acquisition module 910, the first determination module 920, and the second determination module 940 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 910, the first determination module 920, and the second determination module 940 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 910, the first determination module 920, and the second determination module 940 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0126] Fig.10A block diagram of an electronic device suitable for implementing a conference audio data processing method according to an embodiment of the present disclosure is schematically shown.

[0127] like Fig.10 As shown, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 to a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0128] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0129] According to an embodiment of the present disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the input / output (I / O) interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output (I / O) interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1008 as needed.

[0130] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0131] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1002 and / or RAM 1003 described above and / or one or more memories other than ROM 1002 and RAM 1003.

[0132] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the conference audio data processing method provided by the embodiment of the present disclosure.

[0133] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 1001. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0134] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0135] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0136] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0137] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0139] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A conference audio data processing method, characterized in that: The method comprises: Acquire audio data, where the audio data includes N voice data, where N is a positive integer greater than or equal to 2; Determine a path for acquiring the audio data, where the path represents a method for acquiring the audio data; Determine, according to the acquisition path, a target matching strategy corresponding to the audio data, wherein the target matching strategy is used to determine a speaker corresponding to each of the voice data; According to the speakers and the voice data, target text information corresponding to the audio data is determined, and the target text information records text information of speeches by multiple speakers.

2. The method according to claim 1, characterized in that: In the case where the audio data is acquired in non-real time, the method includes: Classifying the voiceprint information of the N voice data to obtain M voiceprint information, where M is a positive integer less than or equal to N; The speaker to which each voice data belongs is determined according to the M voiceprint information.

3. The method according to claim 1, characterized in that In the case where the audio data is acquired in real time, the method includes: In response to acquiring the nth voice data, determining the voiceprint information of the nth voice data, where n belongs to N and n is a positive integer greater than 3; Matching the voiceprint information of the nth voice data with the voiceprint information of the first n-1 voice data respectively; When the voiceprint information of the n-th voice data is consistent with the voiceprint information of the nm-th voice data, the speaker to whom the nm-th voice data belongs is determined as the speaker to whom the n-th voice data belongs.

4. The method according to claim 3, characterized in that The method further comprises: In response to acquiring the nm-th voice data, determining the voiceprint information of the nm-th voice data; Based on a first database, a speaker corresponding to the nm-th voiceprint information is determined, wherein the first database includes a plurality of sample voiceprints and speakers corresponding to the sample voiceprints.

5. The method according to claim 4, characterized in that Determining a speaker corresponding to the nm-th voiceprint information based on the first database includes: Acquire information associated with the nm-th voice data; In a case where the association information cannot determine the speaker to which the nm-th voice data belongs, determining a second database according to the association information, wherein the first database includes the second database; A speaker corresponding to the nm-th voiceprint information is determined according to the second database.

6. The method according to claim 1, characterized in that The obtaining of audio data comprises: According to the preset segmentation parameters, N voice data are segmented from the audio data.

7. The method according to claim 2 or 4, characterized in that: The method further comprises: Acquire a target parameter of the audio data conference, where the target parameter represents a theme of the conference; Determine a third database according to the target parameter, wherein the third database includes a plurality of speakers and a sample voiceprint corresponding to each of the speakers; Based on the third database, the speaker to which each voice data belongs is determined.

8. The method according to claim 1, characterized in that Determining target text information corresponding to the audio data according to the speaker and the voice data includes: Performing speech recognition on the plurality of speech data respectively to obtain text information corresponding to each speech data; According to the speaker and the text information, target text information corresponding to the audio data is determined.

9. A conference audio data processing device, characterized in that: The device comprises: An acquisition module, used to acquire audio data, wherein the audio data includes N voice data, where N is a positive integer greater than or equal to 2; A first determining module, configured to determine a path for obtaining the audio data, wherein the path for obtaining the audio data represents a method for obtaining the audio data; A second determination module, configured to determine a target matching strategy corresponding to the audio data according to the acquisition path, wherein the target matching strategy is used to determine a speaker corresponding to each of the voice data; The third determination module is used to determine target text information corresponding to the audio data according to the speakers and the voice data, wherein the target text information records text information of speeches of multiple speakers.

10. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images