Real-time display method for offline conference data and conference management system

Abstract

The invention relates to a real-time display method for offline conference data and a conference management system. The method comprises the following steps: performing voice segmentation on received offline conference voice data in real time, obtaining first offline voice data of a first spokesman, performing voice recognition on the first offline voice data, and obtaining a first offline voiceprint feature; searching voiceprint features matched with the first offline voiceprint features and first spokesman identification information in a spokesman voice database, wherein the first spokesman identification information comprises first spokesman name information and a first spokesman cartoon head portrait; and displaying the cartoonized head portrait of the first spokesman and the first offline text information corresponding to the first offline voice data on the large screen device of the conference room in real time in an associated manner. Therefore, the identification of the current spokesman in the offline meeting place can be identified according to the spokesman voice database in the offline meeting, the text of the spokesman and the cartoonized head portrait of the spokesman are displayed on the large meeting screen in real time in an associated mode, and the method is more intelligent and interesting.

Description

Technical Field

[0001] The present invention relates to the field of offline conference technology, and in particular to a real-time display method for offline conference data, a conference management system and a computer program product. Background Art

[0002] With the rapid development of conference technology, the need to automatically generate conference records through computers is becoming more and more urgent. For online conferences, current conference technology can basically realize the function of automatically generating conference records.

[0003] However, offline meetings are also very common in some scenarios. For example, all participants are in the same office, so there is no need to hold an online meeting. You can just book a conference room to hold an offline meeting. Currently, the large-screen devices in conference rooms on the market can generally only recognize and display the text information corresponding to the audio in the offline venue in real time. The functions are relatively monotonous and not intelligent enough. Summary of the invention

[0004] The purpose of the present invention is to provide a method for real-time display of offline conference data, a conference management system and a computer program product, aiming to realize a more intelligent real-time display of text information corresponding to audio in an offline conference venue.

[0005] In a first aspect, in a possible manner, an embodiment of the present application provides a method for real-time display of offline conference data, the method comprising: if the conference management system recognizes that the current first speaker has started an online video conference, then obtaining the first speaker's avatar and the first speaker's name information in the online video conference; based on a preset cartoon image conversion model, performing cartoon conversion on the first speaker's avatar to obtain a cartoon avatar of the first speaker, and associating and saving the first speaker's cartoon avatar and the first speaker's name information. In this implementation, by detecting whether each employee has started an online video conference before the offline meeting begins, for example, if it is detected that the first speaker has started an online video conference, the name and avatar of the first speaker are automatically obtained, and the avatar is cartoon converted, and further associating and saving the identification information such as the name and cartoon avatar of the first speaker, it is realized that the cartoon avatar and name of each employee are automatically pre-stored using the online meeting. If the conference management system recognizes that an offline conference is currently open, the offline conference voice data is received in real time; the offline conference voice data is segmented to obtain the first offline voice data, which belongs to the first speaker; the first offline voice data is recognized to obtain the first offline voiceprint feature; the voiceprint feature and the first speaker identification information matching the first offline voiceprint feature are searched in the speaker voice database, and the speaker voice database is associated with the speaker identification information and the voiceprint feature, and the first speaker identification information includes the name information of the first speaker and the cartoonized avatar of the first speaker; if the voiceprint feature and the first speaker identification information matching the first offline voiceprint feature are found, the cartoonized avatar of the first speaker and the first offline text information corresponding to the first offline voice data are displayed in real time on the large screen device of the conference room, or the cartoonized avatar of the first speaker, the name information of the first speaker and the first offline text information corresponding to the first offline voice data are displayed in real time on the large screen device of the conference room. Therefore, in the offline meeting, the current speaker identification in the offline venue can be identified according to the speaker voice database, and the speaker text and its cartoonized avatar can be displayed in real time on the large screen of the conference room, which is more intelligent and interesting.

[0006] Based on the above possible implementation methods, in another possible implementation method, the association between the speaker identification information and the voiceprint features satisfies at least one of the following: it comes from the online meeting before the offline meeting starts, or it comes from requesting and obtaining from the terminals corresponding to the participants before the offline meeting starts, or it comes from monitoring the terminals corresponding to the participants before the offline meeting starts.

[0007] In one embodiment, for the request and acquisition from the terminal corresponding to the participant before the offline meeting starts, the method also includes: the conference management system receives the offline meeting reservation information, the offline meeting reservation information includes the meeting time, and the list of participants or the department of the participant; the conference management system determines whether there is a first participant among the participants in the list of participants or the department of the participant, and the first participant does not match any speaker identification information in the speaker voice database; if there is a first participant, the conference management system outputs audio input prompt information before the meeting time arrives to prompt the first participant to input audio. By outputting the audio input prompt information before the offline meeting, the personnel who have not pre-recorded the voiceprint can be informed of the input prompt information in time, which is conducive to accurate prompting of the participants who have not pre-recorded.

[0008] Furthermore, in one embodiment, the conference management system outputs audio input prompt information including: the conference management system displays the audio input prompt information, the audio input prompt information includes a first control associated with the first participant; the method also includes: the conference management system responds to the trigger operation for the first control, and sends a first audio acquisition request to the first destination address, the first destination address is the first terminal address associated with the first participant. In this method, by triggering the first control, the audio acquisition request to the first participant is triggered, so that the conference organizer or user can selectively identify the participants who need to pre-record voiceprints, and filter out the participants who do not need to pre-record voiceprints, so as to obtain the information of participants more flexibly and accurately, and avoid entering the voiceprint information of irrelevant participants.

[0009] In another embodiment, the conference management system outputs the audio input prompt information including: the conference management system sends a first audio acquisition request to a first destination address, where the first destination address is a first terminal address associated with the first conference participant. In this manner, the conference management system directly sends the audio acquisition request to the first destination address without relying on the triggering operation of the first control, which is more convenient and conducive to automation.

[0010] Based on any of the above possible implementations, in another possible implementation, the method further includes: the conference management system receives the voice data of the first participant sent by the first destination address, the voice data of the first participant is recorded by the first participant based on the terminal associated with the first destination address; the conference management system performs voice recognition on the voice data of the first participant to obtain the voiceprint feature of the first participant; the first participant identification information corresponding to the first participant is associated with the voiceprint feature of the first participant and saved in the speaker voice database. In this embodiment, by requesting and obtaining the voice data of the participants whose voiceprints have not been pre-recorded before the offline meeting starts, the voiceprint pre-recording of all participants can be realized before the offline meeting starts, and the real participant identification information corresponding to each voice segment can be displayed and recorded in real time during the offline meeting, further improving the experience during the offline meeting.

[0011] Based on any of the above possible implementations, in another possible implementation, the conference management system further includes a second terminal, and the method further includes: when detecting that the second terminal is in a powered-on state, monitoring the voice data received by the second terminal; performing voice recognition and denoising on the voice data to obtain the second voiceprint feature of the second speaker; associating the second speaker identification information corresponding to the second terminal with the second voiceprint feature and saving it in the speaker voice database. In this way, the background automatically associates and saves the user's identification information and voiceprint feature without the user having to perform additional operations. The pre-entry of voiceprints for all employees is realized in a more automated and imperceptible manner.

[0012] Based on any of the above possible implementations, in another possible implementation, after the second speaker identification information is associated with the second voiceprint feature and saved in the speaker voice database, the monitoring of the voice data received by the second terminal is stopped. Thus, on the basis of realizing automatic voiceprint pre-recording, the monitoring of the user terminal can be reduced as much as possible, taking into account the protection of user privacy and improving data security.

[0013] In a second aspect, an embodiment of the present application provides a conference management system, which includes at least one processor and a memory, wherein the processor is used to couple with the memory, and the processor calls instructions stored in the memory to execute the method steps in any embodiment of the first aspect above.

[0014] In the third aspect, an embodiment of the present application also provides a device for processing offline conference data, which includes a transceiver module and a processing module. The transceiver module can execute the steps related to receiving, acquiring, sending, etc. in the above-mentioned first aspect, and the processing module can execute the steps related to voice data recognition, segmentation, etc., as well as storage or other processing in the above-mentioned first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method described in any possible implementation of the first aspect above.

[0016] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to execute the method described in any implementation method of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a principle block diagram of a conference management system in one embodiment of the present application;

[0018] Figure 2 This is a flow chart of a method for real-time display of offline conference data in one embodiment of the present application;

[0019] Figure 3A This is a schematic diagram of an interface of a method for real-time display of offline conference data based on cartoonized avatars in one embodiment of the present application;

[0020] Figure 3B This is another interface schematic diagram of a method for real-time display of offline conference data based on cartoonized avatars in one embodiment of the present application;

[0021] Figure 4 This is a principle block diagram of a device for processing offline conference data in one embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present application more clear, the embodiments of the present application are described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0023] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0024] In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0025] The naming or numbering of the steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The named or numbered process steps can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0026] The mutual coupling or direct coupling or communication connection shown or discussed in this application may be through some interfaces, and the indirect coupling or communication connection between units may be electrical or other similar forms, which are not limited in this application.

[0027] The technical solutions of the embodiments of the present invention can be applied to offline meetings, which can also be called local meetings or offline meetings. For example, all participants are located in the same venue, and there are no online participants. Some technical solutions of the embodiments of the present invention can also be applied to online meetings. For example, before the offline meeting starts, the voiceprints of participants are pre-recorded in an online meeting created through specific conference software (such as Tencent Conference).

[0028] The specific system architecture of the embodiment of the present invention can be collectively referred to as a conference management system. Figure 1As shown, the conference management system 100 may include a processor 110, a memory 120, a large screen device 130 of a conference room arranged in a conference room (also referred to as a conference room), a plurality of first conference terminals 140 located in the conference room, and a plurality of second conference terminals 150. The large screen device of the conference room is used to display, emit sound, and receive voice data in the conference room, etc. The large screen device of the conference room may also include a camera, which may be used to collect the head portrait information of the participants in the conference room. The first conference terminal may include a microphone, which may collect the voice data in the conference room, and may also include a camera, which may collect the head portrait information of the participants. The first conference terminal sends the collected voice data and head portrait information to the processor for processing. The first conference terminal can be used for offline meetings or online meetings. The first conference terminal is generally fixedly arranged in the conference room and used in conjunction with the large screen device of the conference room in the conference room. The second conference terminal may be located outside the conference room or in the conference room, for example, it may be a personal terminal, a personal computer or a mobile phone, etc. The second conference terminal can be used for online meetings or for screen projection in offline meetings. The second conference terminal may include a microphone to collect voice data of the corresponding participants of the online conference; it may also include a camera to collect the head portrait information of the participants. The processor and the memory may be set separately or integrated. Alternatively, the processor and the memory may also be collectively referred to as a server.

[0029] Reference Figure 2 In one embodiment of the method for real-time display of offline conference data based on online conferences, in order to automatically identify each speaker in an offline conference, the association between the speaker's voice data and the speaker identifier can be obtained in advance by collecting the speaker's voice data through online conferences before the offline conference starts, or pre-collecting the speaker's voice data from the participant's terminal before the offline conference starts, or pre-monitoring the participant's terminal before the offline conference starts. This method embodiment includes:

[0030] S101, if the conference management system recognizes that the current first speaker has started an online video conference, obtain the first speaker's head portrait and the first speaker's name information in the first online video conference;

[0031] S102: Based on a preset cartoon image conversion model, the first speaker's avatar is cartoon-converted to obtain a cartoon avatar of the first speaker, and the cartoon avatar of the first speaker and the name information of the first speaker are associated and saved.

[0032] In one implementation, the name information of the first speaker and the cartoonized head portrait of the first speaker may be associated and saved in an employee center or a user database or a speaker voice database.

[0033] In one embodiment, a method for cartooning an avatar based on a cartoon image conversion model is as follows: a first speaker avatar is meshed and feature points are extracted before modeling, a cartoon avatar matching the modeled speaker avatar is retrieved from a preset cartoon database, and the cartoon avatar is deformed according to the feature points. Finally, the deformed cartoon avatar is image processed, such as texture mapping and rendering, to obtain a cartoonized avatar of the first speaker. In one embodiment, the preset cartoon database may be pre-stored after being acquired from the network, or may be drawn by the user and uploaded. In one embodiment, the matching of the cartoon avatar in the preset cartoon database with the speaker avatar may be understood as: the similarity between the two is greater than a preset threshold value, and the cartoon avatar with the highest matching degree with the speaker avatar and greater than a preset threshold value may be considered to match the two. The preset threshold value may be, for example, 85%, or other values. In one embodiment, deforming the cartoon avatar according to feature points includes: establishing mapping functions for feature points corresponding to the eyes, mouth, nose, eyebrows, ears, face shape, etc. of the spokesperson avatar, and deforming the cartoon avatar based on the mapping function, so that the similarity between the cartoonized avatar and the spokesperson avatar is further improved.

[0034] In other embodiments, the method for cartoonizing an avatar based on a cartoonized image conversion model may also be performed using a neural network model in the prior art, or other existing algorithms, which will not be described in detail herein.

[0035] In this embodiment, before the offline meeting starts, it is detected whether each employee has started the online video conference. For example, if it is detected that the first speaker has started the online video conference, the name and avatar of the first speaker are automatically obtained, and the avatar is cartoonized, and the identification information such as the name and cartoonized avatar of the first speaker are further associated and saved, so that the cartoon avatar and name of each employee can be automatically pre-stored by using the online meeting, thereby further improving the intelligence and fun of the offline meeting process.

[0036] S103: If the conference management system recognizes that an offline conference is currently started, the offline conference voice data is received in real time.

[0037] In one embodiment, the large screen device in the conference room can display a first control, and the first control can display text such as "start recording" or "offline meeting start", or other text, indicating that it is used to start the offline meeting recording function. When the user triggers the first control, the conference room management system confirms that the offline meeting is currently started and starts the offline meeting function. After triggering, the first control switches to display text such as "end recording" or "offline meeting end", or other text, indicating that it is used to end the offline meeting recording function. In another embodiment, the large screen device in the conference room can monitor the voice data in the venue in real time, and confirm whether the offline meeting is currently started or ended by performing voice recognition on the voice data. In another embodiment, the conference management system can receive offline meeting reservation information before the offline meeting starts, and the offline meeting reservation information can include the meeting time, which includes the meeting start time and the meeting end time. After the conference management system monitors that the current meeting start time has arrived, it confirms that the offline meeting has started; after monitoring that the current meeting end time has arrived, it confirms that the offline meeting has ended.

[0038] After confirming that the offline conference is started, the offline conference voice data in the conference room is received in real time. The offline conference voice data can be received in real time through the conference room large screen device or the first conference terminal.

[0039] S104, performing voice segmentation on the offline conference voice data to obtain first offline voice data, where the first offline voice data belongs to the first speaker.

[0040] Offline conference voice data generally comes from multiple speakers in the venue. The processor can segment the received offline conference voice data belonging to multiple speakers through voice detection to obtain a number of segmented voice data, each of which belongs to one speaker. For example, the front segment of voice data and the back segment of voice data can be distinguished according to the silent interval, or denoising (such as non-human voice) can be performed by an algorithm, and the audio segments corresponding to different speakers can also be distinguished by an algorithm. In this embodiment, the first offline voice data belonging to the first speaker is used as an example for description.

[0041] S105, performing speech recognition on the first offline speech data to obtain a first offline voiceprint feature.

[0042] S106, searching the speaker voice database for a voiceprint feature and first speaker identification information that matches the first offline voiceprint feature. The speaker voice database stores the speaker identification information and the voiceprint feature in association. The first speaker identification information includes the name information of the first speaker and a cartoonized avatar of the first speaker.

[0043] S107, if a voiceprint feature and the first speaker identification information matching the first offline voiceprint feature are found, the first speaker cartoon avatar and the first offline text information corresponding to the first offline voice data are displayed in real time on the large screen device in the conference room, such as Figure 3A As shown, it is a schematic diagram of an interface of a method for real-time display of offline conference data based on cartoonized avatars in one embodiment of the present application. Or the first offline text information corresponding to the first offline voice data and the cartoonized avatar of the first speaker, the name information of the first speaker, and the first offline voice data are displayed in real time on the large screen device in the conference room, such as Figure 3B As shown, it is another interface schematic diagram of a method for real-time display of offline conference data based on cartoonized avatars in one embodiment of the present application.

[0044] In this embodiment, after the participants pre-record their voiceprints, the offline text information and speaker identification information corresponding to the voice of the current speaker can be displayed in real time during the offline meeting, so that the offline meeting participants can understand the progress of the current meeting in real time and improve the meeting experience. In addition, the speaker identification information includes the name and cartoon avatar. By displaying the speaker's cartoon avatar and offline speech text in real time on the large screen of the offline venue, the fun and intelligence of the offline meeting process can be further enhanced, and the user experience can be improved.

[0045] In one embodiment, the association between the speaker identification information and the voiceprint feature satisfies at least one of the following: it is derived from an online meeting before the offline meeting starts, or it is derived from requesting and obtaining from a terminal corresponding to a participant before the offline meeting starts, or it is derived from monitoring a terminal corresponding to a participant before the offline meeting starts.

[0046] In one embodiment, the association between the speaker identification information and the voiceprint feature in the speaker voice database is derived from an online meeting before the offline meeting begins. If the processor of the conference management system recognizes that an online meeting is currently open, the first speaker identification information and the first online meeting voice data of the first speaker in the online meeting are obtained. In the online meeting scenario, the first speaker participates in the online meeting through a second conference terminal, and the second conference terminal can be, for example, a personal terminal of the first speaker, and the second conference terminal is pre-associated with the first speaker identification information. When the online meeting is started, the first speaker speaks through the second conference terminal, and the second conference terminal receives the first online voice data from the first speaker, and associates the first speaker identification information with the first online voice data and sends it to the processor. The processor performs voice recognition on the first online meeting voice data to obtain the first online voiceprint feature, and the first online voiceprint feature belongs to the same speaker. It can be understood that the first online voice data may have noise, and the processor can perform denoising on it.

[0047] For the denoising method, in one embodiment, performing voice recognition on the first online conference voice data to obtain the first online voiceprint feature includes: performing voice recognition on the first online conference voice data to obtain at least two online voiceprint features to be screened; among the at least two online voiceprint features to be screened, selecting one of the online voiceprint features to be screened as the first online voiceprint feature, and the volume corresponding to the first online voiceprint feature is greater than the volume corresponding to any other online voiceprint features to be screened that have not been selected. It can be understood that in other embodiments, other denoising methods can also be used, which will not be repeated here. The first speaker identification information is associated with the first online voiceprint feature and saved in the speaker voice database. The memory stores the speaker voice database. The speaker voice database can be continuously updated. The conference management system can collect and associate the speaker identification information and the online voiceprint feature each time an online conference is held to realize the speaker's voiceprint pre-recording function. It is also possible to obtain the reservation information of the online conference (the reservation information includes the participants) before collection, and confirm whether there are users among the participants of the online conference who have not pre-recorded their voiceprints. If so, the voiceprint is pre-recorded after the online conference is started.

[0048] In another embodiment, the association between the speaker identification information and the voiceprint feature in the speaker voice database is obtained from the request to the corresponding terminal of the participant before the offline meeting starts. In one embodiment, before the offline meeting starts, users who have not pre-recorded their voiceprints are automatically prompted to record the audio in advance. Specifically, the method also includes: the conference management system receives offline conference reservation information, the offline conference reservation information includes the meeting time, and the list of participants or the department of the participant; the participant department is associated with a list of all personnel under the department.

[0049] The conference management system determines whether the first participant exists in the list of participants or in each participant's department, and the first participant does not match any first speaker identifier in the speaker voice database. That is, the conference management system identifies a person (i.e., the first participant) among the participants who does not exist in the speaker voice database.

[0050] If there is a first participant, the conference management system outputs an audio input prompt message before the meeting time arrives to prompt the first participant to input audio. Specifically, the conference management system can identify the first participant after receiving the meeting reservation information, and output the audio input prompt message immediately after identification. Alternatively, the audio input prompt message can be output at a time corresponding to a preset duration before the start of the offline meeting. It is understandable that the conference management system can also output audio input prompt messages at other times, as long as enough time is reserved for users to input audio before the start of the offline meeting.

[0051] The way to output the audio recording prompt information can be:

[0052] Method 1: The conference management system directly displays the audio input prompt information. For example, the audio input prompt information can be displayed by the large screen device in the conference room or the first conference terminal. The audio input prompt information may include, for example, a first control associated with the first participant. The first control may be triggered by the conference organizer or the user. The method also includes: the conference management system responds to the trigger operation for the first control and sends a first audio acquisition request to the first destination address, and the first destination address is the first terminal address associated with the first participant. The first destination address can be obtained from the employee information library, and the first destination address is associated with the first control. The first destination address may be, for example, the email address of the first participant, or a mobile phone number, or an employee's internal communication APP (such as WeChat for Enterprise, Feishu), or an employee's private WeChat, etc. In this method, by triggering the first control, an audio acquisition request is triggered to the first participant, so that the conference organizer or user can selectively identify the participants who need to pre-record their voiceprints, and screen out the participants who do not need to pre-record their voiceprints, so as to obtain the participant information more flexibly and accurately and avoid entering the voiceprint information of irrelevant participants.

[0053] Method 2: The conference management system outputs the audio input prompt information including: the conference management system sends a first audio acquisition request to a first destination address, where the first destination address is a first terminal address associated with a first participant. In this method, the conference management system directly sends the audio acquisition request to the first destination address without relying on the triggering operation of the first control, which is more convenient and conducive to automation.

[0054] The method also includes that after the first terminal corresponding to the first participant receives the above-mentioned audio input prompt information, the first terminal can input its voice data and the voice data of the first participant. The conference management system receives the voice data of the first participant sent by the first destination address, and the voice data of the first participant is recorded by the first participant based on the terminal associated with the first destination address; the conference management system performs voice recognition on the voice data of the first participant to obtain the voiceprint feature of the first participant; and associates the first participant identification information corresponding to the first participant with the voiceprint feature of the first participant and saves it in the speaker voice database.

[0055] In this embodiment, by requesting and obtaining the voice data of the participants whose voiceprints have not been pre-recorded from the terminals before the offline meeting starts, the voiceprints of all participants can be pre-recorded before the offline meeting starts. During the offline meeting, the real participant identification information corresponding to each voice segment can be displayed and recorded in real time, further improving the experience during the offline meeting.

[0056] In another embodiment, the association between the speaker identification information and the voiceprint feature in the speaker voice database is derived from the monitoring of the terminal corresponding to the participant before the offline meeting starts. The conference management system also includes a second terminal, and the method also includes: when the second terminal is detected to be in a powered-on state, monitoring the voice data received by the second terminal. The second terminal can be an office terminal of the user in the enterprise, such as an office computer or a mobile phone. The voice data is subjected to voice recognition and denoising to obtain the second voiceprint feature of the second speaker. Since the second terminal belongs to a fixed employee, the main speaker in the voice data received by the second terminal can be identified as the employee corresponding to the second terminal. By identifying and denoising the voice data, the voiceprint feature of the main speaker is obtained, which is the second voiceprint feature of the second speaker. The second speaker identification information corresponding to the second terminal is associated with the second voiceprint feature and saved in the speaker voice database. In this way, the background automatically associates and saves the identification information of the user with the voiceprint feature without the user's awareness, without the need for the user to perform additional operations. The pre-entry of the voiceprints of all employees is realized in a more automated and imperceptible manner.

[0057] Furthermore, after the second speaker identification information is associated with the second voiceprint feature and saved in the speaker voice database, monitoring of the voice data received by the second terminal is stopped. This can reduce monitoring of the user terminal as much as possible on the basis of realizing automatic voiceprint pre-recording, while taking into account protecting user privacy and improving data security.

[0058] Furthermore, in one embodiment, if a voiceprint feature matching the first offline voiceprint feature is found, the speaker identification information corresponding to the found voiceprint feature is associated with the first offline voice data and saved, or the speaker identification information is associated with the offline text information corresponding to the first offline voice data and saved.

[0059] Since the speaker has pre-recorded the voiceprint in the above steps through the above methods, the speaker identification information and the speaker voiceprint features have been pre-stored in the user voiceprint database, and the pre-recorded voiceprint features match the offline voiceprint features.

[0060] In one implementation, the processor further performs voice recognition on the first offline voice data to generate corresponding first offline text information.

[0061] In this embodiment, the function of pre-recording the speaker's voiceprint is realized by utilizing a plurality of the above-mentioned methods. Therefore, after the offline meeting is subsequently started, the speaker identification in the offline meeting can be accurately identified, and then in the meeting minutes, the speaker identification information and the offline meeting voice data can be associated and saved, or the speaker identification information and the offline text information corresponding to the offline meeting voice data can be associated and saved, and then when browsing the offline meeting minutes after the meeting, the viewer can accurately obtain the speaker identification information corresponding to each audio or text segment.

[0062] The present application also provides a device for processing offline conference data. Figure 4 The device includes: a transceiver module 210 and a processing module 220, wherein the transceiver module 210 and the processing module 220 are connected via a bus.

[0063] The transceiver module 210 is used to receive offline conference voice data in real time.

[0064] The processing module 220 is used to perform voice segmentation on the offline conference voice data, obtain several segments of offline conference voice data, any segment of the offline conference voice data belongs to the same speaker; and perform voice recognition on the offline conference voice data to obtain offline voiceprint features; and search for voiceprint features matching the offline voiceprint features in the speaker voice database, wherein the speaker voice database is associated with speaker identification information and voiceprint features; wherein the association relationship between the speaker identification information and the voiceprint features satisfies at least one of the following: originating from an online meeting before the start of the offline meeting, or originating from a request to and acquisition from a terminal corresponding to a participant before the start of the offline meeting, or originating from monitoring of a terminal corresponding to a participant before the start of the offline meeting; and if the voiceprint feature matching the offline voiceprint feature is found, the speaker identification information corresponding to the found voiceprint feature is associated with the offline conference voice data and saved, or the speaker identification information is associated with the offline text information corresponding to the offline conference voice data and saved.

[0065] It can be understood that in the above method embodiment, the steps related to receiving, acquiring, sending, etc. can be implemented by the transceiver module 210, and the steps related to voice data recognition, segmentation, storage or other processing can be implemented by the processing module 220, which will not be repeated here.

[0066] Understandably, Figure 4 Only a simplified design of the device is shown. In actual applications, the device may include any number of transceivers, processors, memories, etc., and all devices that can implement the present application are within the protection scope of the present application.

[0067] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0068] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0069] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0070] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0071] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0072] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for real-time display of offline conference data, applied to a conference management system, wherein the conference management system includes a large screen device in a conference room, and the large screen device in the conference room is located in the venue of the offline conference, characterized in that: The method comprises: If the conference management system recognizes that the current first speaker has started an online video conference, then obtaining the first speaker's headshot and the first speaker's name information in the online video conference; Based on a preset cartoon image conversion model, the first speaker's avatar is cartoon-converted to obtain the first speaker's cartoon avatar, and the first speaker's cartoon avatar and the first speaker's name information are associated and saved; If the conference management system recognizes that an offline conference is currently open, the offline conference voice data is received in real time; Segmenting the offline conference voice data to obtain first offline voice data, where the first offline voice data belongs to the first speaker; Performing voice recognition on the first offline voice data to obtain a first offline voiceprint feature; Searching for a voiceprint feature and first speaker identification information that match the first offline voiceprint feature in a speaker voice database, wherein the speaker voice database stores speaker identification information and voiceprint features in association, and the first speaker identification information includes the name information of the first speaker and a cartoonized avatar of the first speaker; If the voiceprint feature and the first speaker identification information that match the first offline voiceprint feature are found, the first speaker's cartoonized avatar and the first offline text information corresponding to the first offline voice data are displayed in real time on the large-screen device in the conference room, or the first speaker's cartoonized avatar, the first speaker's name information and the first offline text information corresponding to the first offline voice data are displayed in real time on the large-screen device in the conference room.

2. The method according to claim 1, characterized in that The association between the speaker identification information and the voiceprint feature satisfies at least one of the following: it is obtained by requesting and obtaining from the terminal corresponding to the participant before the offline meeting starts, or it is obtained by monitoring the terminal corresponding to the participant before the offline meeting starts, or it is obtained by an online meeting before the offline meeting starts.

3. The method according to claim 2, characterized in that The method further comprises: The conference management system receives offline conference reservation information, wherein the offline conference reservation information includes the conference time, and the list of participants or the departments of the participants; The conference management system determines whether there is a first participant in the participant list or in each participant in the participant department, and the first participant does not match any of the speaker identification information in the speaker voice database; If the first conference participant exists, the conference management system outputs audio input prompt information before the conference time arrives to prompt the first conference participant to input audio.

4. The method according to claim 3, characterized in that The conference management system outputting the audio input prompt information includes: the conference management system displaying the audio input prompt information, the audio input prompt information including a first control associated with the first conference participant; The method further includes: the conference management system responds to a trigger operation on the first control and sends a first audio acquisition request to a first destination address, where the first destination address is a first terminal address associated with the first conference participant.

5. The method according to claim 3, characterized in that The conference management system outputting audio input prompt information includes: the conference management system sending a first audio acquisition request to a first destination address, where the first destination address is a first terminal address associated with the first participant.

6. The method according to claim 4 or 5, characterized in that The method further comprises: The conference management system receives voice data of a first participant sent by the first destination address, where the voice data of the first participant is recorded by the first participant based on a terminal associated with the first destination address; The conference management system performs voice recognition on the voice data of the first conference participant to obtain a voiceprint feature of the first conference participant; The first participant identification information corresponding to the first participant is associated with the first participant's voiceprint feature and saved in the speaker voice database.

7. The method according to claim 1, characterized in that The conference management system further includes a second terminal, and the method further includes: When detecting that the second terminal is in a powered-on state, monitoring voice data received by the second terminal; Performing speech recognition and denoising on the speech data to obtain a second voiceprint feature of the second speaker; The second speaker identification information corresponding to the second terminal is associated with the second voiceprint feature and saved in the speaker voice database.

8. The method according to claim 7, characterized in that After the second speaker identification information is associated with the second voiceprint feature and saved in the speaker voice database, monitoring of the voice data received by the second terminal is stopped.

9. A conference management system, characterized in that: The conference management system includes at least one processor and a memory, wherein the processor is configured to be coupled to the memory, and the processor calls instructions stored in the memory to execute the method according to any one of claims 1 to 8.

10. A computer program product comprising instructions, characterized in that When the method is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 8.

Images