Recording method for off-line conference data and conference management system

Abstract

The invention relates to an offline conference data recording method and a conference management system, and the method comprises the steps: obtaining the identification information of a first spokesman in an online conference and the first online conference audio data of the first spokesman, and obtaining a first online voiceprint feature belonging to the first spokesman, the first spokesman identification information and the first online voiceprint feature are associated and stored in a user voiceprint library; when an offline conference is started, offline conference audio data are received in real time and voice segmentation is carried out, first offline conference audio data belonging to a first spokesman are acquired, and if an online voiceprint feature matched with a first offline voiceprint feature is found in a user voiceprint library, the first spokesman is started; if yes, the first spokesman identification information and the first off-line conference audio data or the first off-line text information are stored in an associated mode, so that a browser can accurately obtain the spokesman identification information corresponding to each segment of audio or text in an off-line conference record browsed after the conference.

Description

Technical Field

[0001] The present invention relates to the technical field of offline conferences, and in particular to a method for recording offline conference data, a conference management system and a computer program product. Background Art

[0002] With the rapid development of conference technology, the need to automatically generate conference records through computers is becoming more and more urgent. For online conferences, current conference technology can basically realize the function of automatically generating conference records.

[0003] However, offline meetings are also very common in some scenarios. For example, all participants are in the same office, so there is no need to hold an online meeting. You can just book a conference room to hold an offline meeting. Currently, large-screen conference room devices on the market generally do not have the function of automatically generating offline meeting records. Summary of the invention

[0004] The purpose of the present invention is to provide a method for recording offline conference data, a conference management system and a program product, aiming to realize the function of automatically generating offline conference records.

[0005] In a first aspect, in a possible manner, an embodiment of the present application provides a method for recording offline conference data, including: if the conference management system recognizes that an online conference is currently opened, obtaining a first speaker identification information and a first online conference audio data of the first speaker in the online conference; performing speech recognition on the first online conference audio data to obtain a first online voiceprint feature, the first online voiceprint feature belonging to the same speaker; associating the first speaker identification information with the first online voiceprint feature and saving them in a user voiceprint library; if the conference management system recognizes that an offline conference is currently opened, receiving the offline conference audio data in real time; performing speech segmentation on the offline conference audio data to obtain the first offline conference audio data, the first offline conference audio data belonging to the same speaker; performing speech recognition on the first offline conference audio data to obtain the first offline voiceprint feature; searching for an online voiceprint feature matching the first offline voiceprint feature in the user voiceprint library; if the first online voiceprint feature matching the first offline voiceprint feature is found, associating the first speaker identification information with the first offline conference audio data or associating the first speaker identification information with the first offline text information corresponding to the first offline conference audio data and saving them.

[0006] In this embodiment, the speaker's voiceprint pre-recording function is realized by utilizing the online conference. Therefore, after the offline conference is subsequently started, the speaker identification in the offline conference can be accurately identified, and then in the conference record, the speaker identification information and the offline conference audio data can be associated and saved, or the speaker identification information and the offline text information corresponding to the offline conference audio data can be associated and saved, and then after the meeting, when browsing the offline conference record, the viewer can accurately obtain the speaker identification information corresponding to each audio or text. This embodiment does not require manual entry, and the participant identification and voiceprint features are automatically associated and saved during the online conference, and then in the offline conference, the speaker identification corresponding to each offline conference audio can be directly identified according to the association relationship, which is more automated and intelligent.

[0007] Based on the above possible implementations, in another possible implementation, the conference management system includes a large screen device in a conference room, and the large screen device in the conference room is located in the venue of an offline conference; the method also includes: performing voice segmentation on the offline conference audio data to obtain the second offline conference audio data, the second offline conference audio data belongs to the second speaker in the venue of the offline conference; performing voice recognition on the second offline conference audio data to obtain the second offline voiceprint feature, and there is no online voiceprint feature matching the second offline voiceprint feature in the user voiceprint library; generating the second speaker temporary identification information, and displaying the second speaker temporary identification information and the second offline text information corresponding to the second offline conference audio data on the large screen device in real time; and associating and saving the second speaker temporary identification information with the second offline conference audio data, or associating and saving the second speaker temporary identification information with the second offline text information corresponding to the second offline conference audio data. Therefore, even if the second speaker has not pre-recorded the voiceprint, the audio data corresponding to this speaker can still be identified in real time and uniquely identified during the offline meeting, making the meeting record clearer and more intelligent.

[0008] Based on any of the above possible implementations, in another possible implementation, the method further includes: before the offline meeting starts, the conference management system receives offline conference reservation information, the offline conference reservation information includes a list of participants or a department of participants; after the offline meeting ends, the conference management system sends a second audio acquisition request to the second destination address corresponding to the second speaker, the second speaker belongs to the list of participants or the department of participants, and the second speaker identification information is not pre-stored in the user voiceprint library; the conference management system receives the second participant audio data and the second participant identification information sent by the second destination address, and searches for the second offline conference audio data matching the second participant audio data, associates and saves the second participant identification information and the second offline conference audio data, or associates and saves the second participant identification information and the second offline conference audio data corresponding to the second offline text information. In this embodiment, after the meeting ends, an audio acquisition request is sent to the speaker in the participant list who is not pre-stored in the user voiceprint library, and the corresponding relationship between the audio and the identification is obtained, and the corresponding relationship between the received audio and the speaker identification is successfully identified to which real participant identification information the temporary identification information corresponds, further improving the accuracy of participant identification in offline meeting records.

[0009] Based on any of the above possible implementations, in another possible implementation, the conference management system also includes a second conference terminal, and the second conference terminal is located in the venue of the offline conference; the method also includes: the conference room large screen device establishes a communication connection with the second conference terminal; the conference room large screen device projects its displayed screen onto the display screen of the second conference terminal. Therefore, in an offline meeting, participants in the venue can watch the content displayed on the conference room large screen device through their personal second conference terminal, such as the content displayed in the aforementioned embodiment can all be watched, which is more convenient and improves the user experience.

[0010] Furthermore, the method also includes: the conference management system obtains the third offline conference audio data in the conference room of the offline conference received by the second conference terminal; the conference management system performs voice recognition and denoising on the third offline conference audio data to obtain the offline audio data of the third speaker of the main speaker corresponding to the second conference terminal; the third speaker identification information corresponding to the second conference terminal and the third offline text information corresponding to the offline audio data of the third speaker are displayed in real time on the large screen device of the conference room, and the third speaker identification information corresponding to the second conference terminal is associated with the offline audio data of the third speaker and saved, or the third speaker identification information is associated with the third offline text information corresponding to the offline audio data of the third speaker and saved. In an offline conference, the participants can not only watch the speaker text information in the offline conference in real time through the personal terminal, but also establish a communication connection between the personal terminal and the large screen device of the conference room without pre-recording the voiceprint, and then identify the main speaker audio data in the audio received by the personal terminal through the conference management system, and further associate the personal terminal identification information with the main speaker audio data, so that even if the participant does not pre-record the voiceprint, the conference management system can accurately identify the participant in the offline conference, so as to record the offline conference more efficiently.

[0011] Based on any of the above possible implementations, in another possible implementation, performing speech recognition on the first online conference audio data to obtain a first online voiceprint feature includes: performing speech recognition on the first online conference audio data to obtain at least two online voiceprint features to be screened; among the at least two online voiceprint features to be screened, selecting one of the online voiceprint features to be screened as the first online voiceprint feature, and the volume corresponding to the first online voiceprint feature is greater than the volume corresponding to any other unselected online voiceprint features to be screened.

[0012] Based on any of the above possible implementations, in another possible implementation, the conference management system includes a large-screen device in a conference room, the large-screen device in the conference room is located in a venue of an offline conference, and the large-screen device in the conference room includes a camera, which is used to obtain video data in the venue; the method also includes: performing voice segmentation on the offline conference audio data in real time to obtain fourth offline conference audio data, the fourth offline conference audio data belongs to a fourth speaker in the venue of the offline conference; performing voice recognition on the fourth offline conference audio data to obtain a fourth offline voiceprint feature, and there is no online voiceprint feature matching the fourth offline voiceprint feature in the user voiceprint library; obtaining video data of the fourth speaker in the venue through the camera, and performing image recognition on the video data to obtain the venue avatar information of the fourth speaker; searching for a fourth speaker avatar matching the venue avatar information in an employee information database, the employee information database pre-stored with fourth speaker identification information, the fourth speaker identification information including the fourth speaker name information and the fourth speaker avatar; and displaying the fourth speaker identification information and the fourth offline text information corresponding to the fourth offline conference audio data in real time on the large-screen device in the conference room. Furthermore, the method also includes: associating and saving the fourth speaker identification information with the fourth offline conference audio data, or associating and saving the fourth speaker identification information with the fourth offline text information corresponding to the fourth offline conference audio data. In this embodiment, even if the employee does not pre-record the voiceprint before the offline meeting, the head portrait information of the participants can still be obtained through the camera at the offline meeting site, and the participants in the offline meeting site can be accurately identified based on the employee head portraits pre-stored in the employee information database, so as to more efficiently record and display the offline meeting and improve the participants' experience of the offline meeting.

[0013] Furthermore, the method further includes: associating the fourth speaker identification information with the fourth offline voiceprint feature and saving them in the user voiceprint library. Thus, in the next offline meeting, the fourth speaker can be accurately identified directly through the user voiceprint library without the need for head portrait recognition, further improving the efficiency of identifying participants in subsequent offline meetings.

[0014] In a second aspect, an embodiment of the present application provides a conference management system, which includes at least one processor and a memory, wherein the processor is used to couple with the memory, and the processor calls instructions stored in the memory to execute the method steps in any embodiment of the first aspect above.

[0015] In the third aspect, an embodiment of the present application also provides a device for processing offline conference data, which includes a transceiver module and a processing module. The transceiver module can execute the steps related to receiving, acquiring, sending, etc. in the above-mentioned first aspect, and the processing module can execute the steps related to voice data recognition, segmentation, etc., as well as storage or other processing in the above-mentioned first aspect.

[0016] In a fourth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method described in any possible implementation of the first aspect above.

[0017] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to execute the method described in any implementation method of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a principle block diagram of a conference management system in one embodiment of the present application;

[0019] Figure 2 This is a flow chart of a method for recording offline conference data based on an online conference in an embodiment of the present application;

[0020] Figure 3A A schematic diagram of an operation interface of a method for recording offline conference data in an embodiment of the present application;

[0021] Figure 3B A schematic diagram of another operation interface of the method for recording offline conference data in one embodiment of the present application;

[0022] Figure 3C It is another schematic diagram of an operation interface of a method for recording offline conference data in an embodiment of the present application;

[0023] Figure 4 This is a flowchart of a method for recording offline conference data based on an online conference in another embodiment of the present application;

[0024] Figure 5 This is a flowchart of a method for recording offline conference data based on an online conference in another embodiment of the present application;

[0025] Figure 6 This is a flowchart of a method for recording offline conference data based on an online conference in yet another embodiment of the present application;

[0026] Figure 7 This is a principle block diagram of a device for recording offline conference data in one embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present application more clear, the embodiments of the present application are described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0028] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0029] In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0030] The naming or numbering of the steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The named or numbered process steps can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0031] The mutual coupling or direct coupling or communication connection shown or discussed in this application may be through some interfaces, and the indirect coupling or communication connection between units may be electrical or other similar forms, which are not limited in this application.

[0032] The technical solutions of the embodiments of the present invention can be applied to offline meetings, which can also be called local meetings. For example, all participants are located in the same venue, and there are no online participants. Some technical solutions of the embodiments of the present invention can also be applied to online meetings. For example, before the start of an offline meeting, the voiceprints of participants can be pre-recorded in an online meeting created through a specific conference software (such as Tencent Meeting).

[0033] The specific system architecture of the embodiment of the present invention can be collectively referred to as a conference management system. Figure 1As shown, the conference management system 100 may include a processor 110, a memory 120, a large screen device 130 of a conference room arranged in a conference room (also referred to as a conference room), a plurality of first conference terminals 140 located in the conference room, and a plurality of second conference terminals 150. The large screen device of the conference room is used to display, emit sound, and receive voice data in the conference room, etc. The large screen device of the conference room may also include a camera, which may be used to collect the head portrait information of the participants in the conference room. The first conference terminal may include a microphone, which may collect the voice data in the conference room, and may also include a camera, which may collect the head portrait information of the participants. The first conference terminal sends the collected voice data and head portrait information to the processor for processing. The first conference terminal can be used for both offline conferences and online conferences. The first conference terminal is generally fixedly arranged in the conference room and used in conjunction with the large screen device of the conference room in the conference room. The second conference terminal may be located outside the conference room or in the conference room, for example, it may be a personal terminal, a personal computer or a mobile phone, etc. The second conference terminal can be used for online conferences and for screen projection in offline conferences. The second conference terminal may include a microphone to collect voice data of the corresponding participants of the online conference; it may also include a camera to collect the head portrait information of the participants. The processor and the memory may be set separately or integrated. Alternatively, the processor and the memory may also be collectively referred to as a server.

[0034] Reference Figure 2 In one embodiment of the method for recording offline conference data based on online conference, in order to automatically identify each speaker in the offline conference, the association between the speaker's audio data and the speaker's identifier can be collected in advance by using the online conference. The method includes:

[0035] S101: If the processor of the conference management system recognizes that an online conference is currently started, obtain identification information of a first speaker in the online conference and first online conference audio data of the first speaker.

[0036] In an online conference scenario, a first speaker participates in an online conference through a second conference terminal, which may be, for example, a personal terminal of the first speaker, and the second conference terminal is pre-associated with the first speaker identification information. When the online conference is started, the first speaker speaks through the second conference terminal, and the second conference terminal receives the first online audio data from the first speaker, and associates the first speaker identification information with the first online audio data and sends it to the processor.

[0037] S102: The processor performs speech recognition on the first online conference audio data to obtain a first online voiceprint feature, where the first online voiceprint feature belongs to the same speaker.

[0038] It is understandable that the first online audio data may have noise, and the processor may perform denoising on it. With respect to the denoising method, in one embodiment, performing speech recognition on the first online conference audio data to obtain the first online voiceprint feature includes: performing speech recognition on the first online conference audio data to obtain at least two online voiceprint features to be screened; among the at least two online voiceprint features to be screened, selecting one of the online voiceprint features to be screened as the first online voiceprint feature, and the volume corresponding to the first online voiceprint feature is greater than the volume corresponding to any other unselected online voiceprint features to be screened. It is understandable that in other embodiments, other denoising methods may also be used, which will not be described in detail here.

[0039] S103: associate the first speaker identification information with the first online voiceprint feature and save them in a user voiceprint database.

[0040] The memory stores the user voiceprint library. The user voiceprint library can be continuously updated. The conference management system can collect and associate speaker identification information and online voiceprint features every time an online conference is held to realize the speaker's voiceprint pre-recording function. It is also possible to obtain the reservation information of the online conference (the reservation information includes the participants) before collection, and confirm whether there are users among the participants of the online conference who have not pre-recorded their voiceprints. If so, the voiceprints are pre-recorded after the online conference is started.

[0041] S104: If the conference management system recognizes that an offline conference is currently started, the offline conference audio data is received in real time.

[0042] In one embodiment, referring to Figure 3A , the conference room large screen device can display a first control, which can display words such as "start recording" or "offline meeting start", or other words, indicating that it is used to start the offline meeting recording function. When the user triggers the first control, the conference room management system confirms that the offline meeting is currently started and starts the offline meeting function. After triggering, refer to Figure 3B , the first control switches to display text such as "end recording" or "offline meeting end", and other text may also be displayed to indicate the end of the offline meeting recording function. In another embodiment, the large-screen device in the conference room can monitor the audio data in the venue in real time, and confirm whether the offline meeting is currently started or ended by performing voice recognition on the audio data. In another embodiment, the conference management system can receive offline meeting reservation information before the offline meeting starts. The offline meeting reservation information may include the meeting time, and the meeting time includes the meeting start time and the meeting end time. After the conference management system monitors that the current meeting start time has been reached, it confirms the start of the offline meeting; after monitoring that the current meeting end time has been reached, it confirms the end of the offline meeting.

[0043] After confirming that the offline conference is started, the offline conference audio data in the conference room is received in real time. The offline conference audio data can be received in real time through the conference room large screen device or the first conference terminal.

[0044] S105, performing voice segmentation on the offline conference audio data to obtain first offline conference audio data, where the first offline conference audio data belongs to the same speaker.

[0045] The offline conference audio data generally comes from multiple speakers in the venue. The processor can perform voice segmentation on the received offline conference audio data belonging to multiple speakers through voice detection to obtain a number of segmented audio data, each of which belongs to one speaker. For example, the front audio data and the back audio data can be distinguished based on the silent interval, or denoising (such as non-human voice) can be performed by an algorithm, and the audio segments corresponding to different speakers can also be distinguished by an algorithm. In this embodiment, the first offline conference audio data belonging to the above-mentioned first speaker is used as an example for description.

[0046] S106: Perform speech recognition on the first offline conference audio data to obtain a first offline voiceprint feature.

[0047] S107: Searching the user voiceprint database for an online voiceprint feature that matches the first offline voiceprint feature.

[0048] S108: If a first online voiceprint feature matching the first offline voiceprint feature is found, the first speaker identification information is associated with the first offline conference audio data and saved, or the first speaker identification information is associated with the first offline text information corresponding to the first offline conference audio data and saved, thereby realizing the offline conference recording function.

[0049] Reference Figure 3A Or 3B, the conference room large screen device can also display a second control. When receiving a trigger operation based on the second control, the conference room large screen device displays an offline meeting record list, and each record in the list corresponds to an offline meeting association saving result in S108. The second control can display text such as "record list" or "offline meeting record", indicating that it is used to view the historical offline meeting record list.

[0050] Listed in S108, see Figure 3C , and the first offline text information corresponding to the first offline conference audio data can also be stored and displayed in real time in association with the first speaker identification information.

[0051] Since the first speaker has pre-recorded his voiceprint through the online conference in the above steps, the first speaker identification information and the first online voiceprint feature have been pre-stored in the user voiceprint database, and the first online voiceprint feature matches the first offline voiceprint feature.

[0052] In one implementation, the processor further performs speech recognition on the first offline conference audio data to generate corresponding first offline text information.

[0053] In an offline meeting, the above steps S104 to S108 may be performed in real time, or may be performed after the offline meeting.

[0054] In this embodiment, the function of pre-recording the speaker's voiceprint is realized by utilizing the online meeting. Therefore, after the offline meeting is subsequently started, the speaker identification in the offline meeting can be accurately identified, and then in the meeting minutes, the speaker identification information and the offline meeting audio data can be associated and saved, or the speaker identification information and the offline text information corresponding to the offline meeting audio data can be associated and saved, and then when browsing the offline meeting records after the meeting, the viewer can accurately obtain the speaker identification information corresponding to each audio or text segment.

[0055] Based on any of the above embodiments or any of the implementation modes, in one embodiment, the conference management system can also realize the function of displaying the speaker identification and offline text information corresponding to the speaker audio data in real time during the offline conference.

[0056] In one embodiment, referring to Figure 4 , the method also includes: S109, if the first online voiceprint feature matching the first offline voiceprint feature is found, the conference room large screen device displays the first speaker identification information and the first offline text information corresponding to the first offline conference audio data in real time. In this embodiment, the above steps S104 to S107 are performed in real time. By displaying the speaker identification and the corresponding offline text information in real time on the conference large screen in an offline meeting, the participants in the venue can understand in real time who the current speaker is and the corresponding voice text of the speaker, and understand the content and progress of the meeting more intuitively.

[0057] In another embodiment, there is a second speaker, and the second speaker fails to perform the aforementioned voiceprint pre-recording. Figure 5 The method further includes: S110, performing voice segmentation on the offline conference audio data to obtain second offline conference audio data, where the second offline conference audio data belongs to a second speaker in the conference venue of the offline conference.

[0058] S111, perform speech recognition on the second offline conference audio data to obtain a second offline voiceprint feature, and there is no online voiceprint feature matching the second offline voiceprint feature in the user voiceprint library. In this embodiment, speech segmentation, recognition, and obtaining offline voiceprint features can refer to the above embodiments, which will not be described in detail here. Since the second speaker did not pre-record the voiceprint, there is no online voiceprint feature matching the second offline voiceprint feature in the user voiceprint library.

[0059] S112, generate temporary identification information of the second speaker, and display the temporary identification information of the second speaker and the second offline text information corresponding to the second offline conference audio data on the large screen device in real time; and store the temporary identification information of the second speaker in association with the second offline conference audio data, or store the temporary identification information of the second speaker in association with the second offline text information corresponding to the second offline conference audio data. In this embodiment, the temporary identification information of the second speaker may be, for example, "Speaker A", "Speaker B" or other identifications, and each temporary identification information generated in each offline meeting is different from each other, and the temporary identification information corresponding to the same speaker is unique. Therefore, even if the second speaker has not pre-recorded the voiceprint, the audio data corresponding to this speaker can still be identified in real time and uniquely identified during the offline meeting, making the meeting record clearer and more intelligent. Reference Figure 3C The large screen device in the conference room can also associate and display the second speaker temporary identification information with the second offline text information corresponding to the second offline conference audio data in real time.

[0060] In one embodiment, before the offline conference starts, the offline conference reservation information received by the conference management system also includes a list of participants or departments of participants. Figure 6 , the method further includes: S113, after the offline meeting ends, the conference management system sends a second audio acquisition request to the second destination address corresponding to the second speaker, the second speaker belongs to the list of participants or the department of participants, and the identification information of the second speaker is not pre-stored in the user voiceprint library. In this embodiment, when there are multiple second speakers, the conference management system can send a second audio acquisition request to each second speaker respectively. In one implementation, in the list of participants or the list corresponding to the department of participants, there is a part of the first speakers, and there can be multiple first speakers, each of which has different identification information corresponding to it, for example, each first speaker represents a participant, so these first speakers can be accurately identified in the offline meeting. In the list of participants or the list corresponding to the department of participants, there is also a part of the second speakers, and since these second speakers have not been pre-recorded in voiceprint, there is no identification information of the second speakers in the user voiceprint library. After the offline meeting ends, the conference management system can identify these second speakers. The second destination address can be a mailbox, internal communication software of the enterprise, etc.

[0061] S114, the conference management system receives the second participant audio data and the second participant identification information sent by the second destination address, and searches for the second offline conference audio data that matches the second participant audio data, and associates and saves the second participant identification information and the second offline conference audio data, or associates and saves the second offline text information corresponding to the second participant identification information and the second offline conference audio data.

[0062] After receiving the second audio acquisition request, the second speaker can input the audio data of the second participant by himself, and send the audio data of the second participant and the corresponding identification information of the second participant to the conference management system.

[0063] After the conference management system finds the second participant audio data that matches the second offline conference audio data saved in the aforementioned embodiment, the second speaker temporary identification information and the second offline conference audio data saved in association with the aforementioned embodiment are replaced with the second participant identification information and the second offline conference audio data, or replaced with the second offline text information corresponding to the second participant identification information and the second offline conference audio data.

[0064] In one embodiment, the method further includes: the conference room large screen device establishes a communication connection with the second conference terminal; the conference room large screen device projects the screen displayed on the display screen of the second conference terminal. Therefore, in an offline meeting, participants in the venue can watch the content displayed on the conference room large screen device through their personal second conference terminal, such as the content displayed in the above embodiment, which is more convenient and improves the user experience.

[0065] In one embodiment, the method further includes: the conference management system obtains the third offline conference audio data in the conference venue of the offline conference received by the second conference terminal. Since the third offline conference audio data comes from the conference venue, it actually includes the voices of multiple speakers. Therefore, the conference management system performs voice recognition and denoising on the third offline conference audio data to obtain the third speaker offline audio data of the main speaker corresponding to the second conference terminal. Since the second conference terminal generally belongs to an individual, the user is also a fixed user. In the offline conference, the user (hereinafter referred to as the third speaker) is generally close to the second conference terminal. Therefore, in the audio data received by the second conference terminal, the volume of the third speaker is larger or clearer, and can be called the main speaker. The third speaker offline audio data of the main speaker in the third offline conference audio data can be identified by the existing voice recognition algorithm. Since the second conference terminal belongs to an individual (i.e., the third speaker mentioned above), the conference management system can obtain the third speaker identification information while obtaining the third offline conference audio data, and associate the third speaker identification information with the third speaker offline audio data.

[0066] The third speaker identification information corresponding to the second conference terminal and the third offline text information corresponding to the third speaker's offline conference audio data are displayed in real time on the large screen device in the conference room, and the third speaker identification information corresponding to the second conference terminal and the third speaker's offline conference audio data are associated and saved, or the third speaker identification information and the third speaker's offline text information corresponding to the third speaker's offline conference audio data are associated and saved.

[0067] In this embodiment, in an offline meeting, participants can not only watch the speaker text information in the offline meeting in real time through their personal terminals, but also, without pre-recording their voiceprints, establish a communication connection between their personal terminals and the large-screen device in the conference room, and then identify the audio data of the main speaker in the audio received by their personal terminals through the conference management system, and further associate the personal terminal identification information with the audio data of the main speaker. Therefore, even if the participant has not pre-recorded his voiceprint, the conference management system can accurately identify the participant in the offline meeting, thereby more efficiently recording the offline meeting.

[0068] In one embodiment, the method further includes: performing voice segmentation on the offline conference audio data in real time to obtain fourth offline conference audio data, where the fourth offline conference audio data belongs to a fourth speaker in the conference room of the offline conference. Performing voice recognition on the fourth offline conference audio data to obtain a fourth offline voiceprint feature, where there is no online voiceprint feature matching the fourth offline voiceprint feature in the user voiceprint library. The specific implementation of this step can refer to the above embodiment and will not be repeated here.

[0069] The video data of the fourth speaker in the venue is obtained through the camera, and the video data is image recognized to obtain the venue head portrait information of the fourth speaker. The camera of the large screen device in the conference room is used to obtain the video data in the venue; the camera of the first conference terminal can also be used to collect video data in the venue. The specific method of image recognition can adopt the existing technology, which will not be repeated here.

[0070] The fourth speaker's headshot matching the venue headshot information is searched in the employee information database, and the fourth speaker identification information is pre-stored in the employee information database, and the fourth speaker identification information includes the fourth speaker's name information and the fourth speaker's headshot. The employee information database may be an existing database within the company that stores employee-related information, including employee headshot information, employee name, employee ID and other identification information.

[0071] The fourth speaker identification information and the fourth offline text information corresponding to the fourth offline conference audio data are associated and displayed in real time on the large screen device in the conference room.

[0072] The fourth speaker identification information is associated with the fourth offline conference audio data and saved, or the fourth speaker identification information is associated with the fourth offline text information corresponding to the fourth offline conference audio data and saved.

[0073] In this embodiment, even if the employee does not pre-record the voiceprint before the offline meeting, the head portrait information of the participants can still be obtained through the camera at the offline meeting site, and the participants in the offline venue can be accurately identified based on the employee head portraits pre-stored in the employee information database, so as to more efficiently record and display the offline meeting, thereby improving the participants' experience of the offline meeting.

[0074] Furthermore, in one embodiment, the method further includes: associating the fourth speaker identification information with the fourth offline voiceprint feature and saving them in the user voiceprint library. Thus, in the next offline meeting, the fourth speaker can be accurately identified directly through the user voiceprint library without the need for head portrait recognition, further improving the efficiency of identifying participants in subsequent offline meetings.

[0075] The present application also provides a device 200 for processing offline conference data. Figure 7 The device includes: a transceiver module 210 and a processing module 220, wherein the transceiver module 210 and the processing module 220 are connected via a bus.

[0076] The transceiver module 210 is used to obtain the first speaker identification information and the first online conference audio data of the first speaker in the online conference; and is also used to receive the offline conference audio data in real time if the conference management system recognizes that an offline conference is currently started.

[0077] The processing module 220 is used to perform speech recognition on the first online conference audio data to obtain a first online voiceprint feature, which belongs to the same speaker; and associate the first speaker identification information with the first online voiceprint feature and save it in the user voiceprint library; it is also used to perform speech segmentation on the offline conference audio data to obtain the first offline conference audio data, which belongs to the same speaker; and perform speech recognition on the first offline conference audio data to obtain the first offline voiceprint feature; and search for an online voiceprint feature matching the first offline voiceprint feature in the user voiceprint library; and if the first online voiceprint feature matching the first offline voiceprint feature is found, the first speaker identification information is associated with the first offline conference audio data and saved, or the first speaker identification information is associated with the first offline text information corresponding to the first offline conference audio data and saved.

[0078] It can be understood that in the above method embodiment, the steps related to receiving, acquiring, sending, etc. can be implemented by the transceiver module 210, and the steps related to voice data recognition, segmentation, storage or other processing can be implemented by the processing module 220, which will not be repeated here.

[0079] Understandably, Figure 6 Only a simplified design of the device is shown. In actual applications, the device may include any number of transceivers, processors, memories, etc., and all devices that can implement the present application are within the protection scope of the present application.

[0080] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0081] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0082] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0083] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0084] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0085] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for recording offline conference data, applied to a conference management system, characterized in that: The method comprises: If the conference management system recognizes that an online conference is currently started, obtaining identification information of a first speaker in the online conference and first online conference audio data of the first speaker; Performing speech recognition on the first online conference audio data to obtain a first online voiceprint feature, where the first online voiceprint feature belongs to the same speaker; Associating the first speaker identification information with the first online voiceprint feature and saving them in a user voiceprint library; If the conference management system recognizes that an offline conference is currently open, the offline conference audio data is received in real time; Segmenting the offline conference audio data to obtain first offline conference audio data, where the first offline conference audio data belongs to the same speaker; Performing speech recognition on the first offline conference audio data to obtain a first offline voiceprint feature; Searching the user voiceprint database for an online voiceprint feature that matches the first offline voiceprint feature; If the first online voiceprint feature that matches the first offline voiceprint feature is found, the first speaker identification information is associated with the first offline conference audio data and saved, or the first speaker identification information is associated with the first offline text information corresponding to the first offline conference audio data and saved; or the first speaker identification information is associated with the first offline text information corresponding to the first offline conference audio data and saved and displayed in real time.

2. The method according to claim 1, characterized in that The conference management system includes a large screen device in the conference room, and the large screen device in the conference room is located in the venue of the offline conference; The method further includes: performing voice segmentation on the offline conference audio data to obtain second offline conference audio data, where the second offline conference audio data belongs to a second speaker in the conference venue of the offline conference; Performing speech recognition on the second offline conference audio data to obtain a second offline voiceprint feature, wherein there is no online voiceprint feature matching the second offline voiceprint feature in the user voiceprint library; Generate temporary identification information of the second speaker, and display the second speaker temporary identification information and the second offline text information corresponding to the second offline conference audio data in real time on the large screen device of the conference room; and associate and save the second speaker temporary identification information with the second offline conference audio data, or associate and save the second speaker temporary identification information with the second offline text information corresponding to the second offline conference audio data.

3. The method according to claim 2, characterized in that The method further comprises: Before the offline conference starts, the conference management system receives offline conference reservation information, wherein the offline conference reservation information includes a list of participants or departments of participants; After the offline conference ends, the conference management system sends a second audio acquisition request to a second destination address corresponding to the second speaker, the second speaker belongs to the participant list or the participant department, and the second speaker identification information is not pre-stored in the user voiceprint library; The conference management system receives the second participant audio data and the second participant identification information sent by the second destination address, searches for the second offline conference audio data matching the second participant audio data, associates and saves the second participant identification information and the second offline conference audio data, or associates and saves the second participant identification information and the second offline text information corresponding to the second offline conference audio data.

4. The method according to claim 2, characterized in that The conference management system further includes a second conference terminal, and the second conference terminal is located in the venue of the offline conference; the method further includes: The conference room large screen device establishes a communication connection with the second conference terminal; The large screen device in the conference room projects the displayed image onto the display screen of the second conference terminal.

5. The method according to claim 4, characterized in that The method further comprises: The conference management system acquires the third offline conference audio data in the conference venue of the offline conference received by the second conference terminal; The conference management system performs speech recognition and denoising on the third offline conference audio data to obtain the third speaker offline audio data of the main speaker corresponding to the second conference terminal; The third speaker identification information corresponding to the second conference terminal and the third offline text information corresponding to the third speaker offline audio data are displayed in real time on the large screen device in the conference room, and the third speaker identification information corresponding to the second conference terminal and the third speaker offline audio data are associated and saved, or the third speaker identification information and the third offline text information corresponding to the third speaker offline audio data are associated and saved.

6. The method according to claim 1, characterized in that The performing speech recognition on the first online conference audio data to obtain the first online voiceprint feature includes: Performing voice recognition on the first online conference audio data to obtain at least two online voiceprint features to be screened; Among the at least two online voiceprint features to be screened, one of the online voiceprint features to be screened is selected as the first online voiceprint feature, and the volume corresponding to the first online voiceprint feature is greater than the volume corresponding to any other unselected online voiceprint features to be screened.

7. The method according to claim 1, characterized in that The conference management system includes a large screen device in a conference room, the large screen device in the conference room is located in the conference room of the offline conference, and the large screen device in the conference room includes a camera, and the camera is used to obtain video data in the conference room; The method further includes: performing voice segmentation on the offline conference audio data in real time to obtain fourth offline conference audio data, where the fourth offline conference audio data belongs to a fourth speaker in the conference room of the offline conference; Performing speech recognition on the fourth offline conference audio data to obtain a fourth offline voiceprint feature, wherein no online voiceprint feature matching the fourth offline voiceprint feature exists in the user voiceprint library; Acquire video data of the fourth speaker in the conference room through the camera, and perform image recognition on the video data to acquire conference room head portrait information of the fourth speaker; Searching for a fourth speaker's head portrait that matches the venue head portrait information in an employee information database, wherein the employee information database pre-stores fourth speaker identification information, and the fourth speaker identification information includes the fourth speaker's name information and the fourth speaker's head portrait; The fourth speaker identification information and the fourth offline text information corresponding to the fourth offline conference audio data are associated and displayed in real time on the large screen device in the conference room.

8. The method according to claim 7, characterized in that The method also includes: associating and saving the fourth speaker identification information with the fourth offline conference audio data, or associating and saving the fourth speaker identification information with the fourth offline text information corresponding to the fourth offline conference audio data; associating and saving the fourth speaker identification information with the fourth offline voiceprint feature in the user voiceprint library.

9. A conference management system, characterized in that: The conference management system includes at least one processor and a memory, wherein the processor is configured to be coupled to the memory, and the processor calls instructions stored in the memory to execute the method according to any one of claims 1 to 8.

10. A computer program product comprising instructions, characterized in that When the method is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 8.

Images