Audio processing method and device, equipment, storage medium and computer program product

Abstract

The invention provides an audio processing method and device, equipment, a storage medium and a computer program product, and the method comprises the steps: obtaining audio content correlation information and audio time sequence correlation information between a first user and a second user in an online conference; performing virtual group division on the first user and the second user according to the audio content correlation information and the audio time sequence correlation information; the audio of each group member in the virtual group where the first user is located is played, the first user is the terminal user, and the second user is other terminal users except the first user in the online conference. According to the method and the device, participants can be divided into groups with different themes based on analysis of audios of the participants, and all participants in an online conference can only hear the voice of the members of the group where the participants are located only by entering the same conference room and do not need to repeatedly enter and exit different conference rooms, so that the conference efficiency is improved, and the conference experience is improved. The problem that an online conference scheme is low in efficiency in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to an audio processing method, device, equipment, storage medium and computer program product. Background Art

[0002] In the existing conference systems, the host sets a conference theme and establishes an online conference room, and then notifies relevant personnel to attend the meeting. However, during the meeting, participants from different project groups will speak according to their own positions, causing the theme of the meeting to spread and be inconsistent with the main topic of this conference room, resulting in multiple topics being discussed simultaneously in one conference room, and people speaking on different topics at the same time will affect each other. For example, in a fault review meeting, after the host describes the problem, the front-end staff will discuss the problem of the App (application), while the back-end staff will discuss the problem of the service node. At this time, there will be two waves of discussions on the two topics of the front-end and the back-end, and the voices of the entire online conference room will affect each other, resulting in low efficiency or inability to continue the discussion. At this time, it is necessary to create other conference rooms and assign some people to other newly created conference rooms.

[0003] That is, in the prior art, participants in the same online conference room may need to discuss different topics of the same issue. Specifically, different online conference rooms are established for people who want to solve the same issue but have different focuses on the discussion topics. After the discussion in the separate conference rooms is over, they enter the same conference room for discussion. If disagreements arise again at this time, it is necessary to repeat the discussion in the separate conference rooms and then enter the same conference room in a cycle, which will result in a waste of resources, a waste of time, and low efficiency.

[0004] From the above, the online conference solutions in the prior art have problems such as low efficiency. Summary of the invention

[0005] The purpose of this application is to provide an audio processing method, device, equipment, storage medium and computer program product to solve the problem of low efficiency of online conference solutions in the prior art.

[0006] In order to solve the above technical problems, the present application provides an audio processing method, including:

[0007] Obtaining audio content correlation information and audio timing correlation information between a first user and a second user in an online conference;

[0008] Dividing the first user and the second user into virtual groups according to the audio content correlation information and the audio timing correlation information;

[0009] The audio of each member of the virtual group in which the first user is located is played, the first user is the terminal user, and the second user is other terminal users in the online meeting except the first user.

[0010] Optionally, obtaining audio content correlation information between a first user and a second user in an online conference includes:

[0011] Acquire a first vocabulary set in a first text, wherein the first text is a text of first audio data of the first user in the online conference, and the first vocabulary set includes: vocabulary ranked in the top Z positions in terms of appearance frequency in the first text, where Z is an integer greater than or equal to 1;

[0012] Acquire a second vocabulary set in a second text, wherein the second text is a text of second audio data of the second user in the online conference, and the second vocabulary set includes: vocabulary ranked in the top Z positions in terms of frequency of occurrence in the second text; and the second vocabulary set has the same total number of elements as the first vocabulary set;

[0013] Obtaining the number of words that match between the first vocabulary set and the second vocabulary set;

[0014] The similarity between the first vocabulary set and the second vocabulary set is determined according to the number of words and the total number of elements, and the similarity is used as the audio content correlation information between the first user and the second user.

[0015] Optionally, obtaining audio timing correlation information between a first user and a second user in an online conference includes:

[0016] Calculating the audio connection between the first user and the second user;

[0017] Audio timing correlation information between the first user and the second user is obtained according to the audio connectivity.

[0018] Optionally, calculating the audio connectivity between the first user and the second user includes:

[0019] Acquire a first time series formed by end times of each speech voice interval of the first user in the first audio data; the first audio data is the audio data of the first user in the online conference;

[0020] Acquire a second time series formed by the start time of each speech voice interval of the second user in the second audio data; the second audio data is the audio data of the second user in the online conference;

[0021] Acquire an audio interval value between the first user and the second user according to the first time series and the second time series;

[0022] The audio connection degree between the first user and the second user is calculated according to the audio interval value.

[0023] Optionally, the obtaining a second time series formed by start times of each speech interval of the second user in the second audio data includes:

[0024] Determine a silent interval of the first user corresponding to a speech interval of the second user; wherein the speech interval of the second user refers to a speech interval of the second user in the second audio data of the online conference, and the silent interval of the first user refers to a silent interval of the first user in the first audio data of the online conference;

[0025] Determining a speech start time of the speech voice interval of the second user in the second audio data according to a time relationship between the silence interval and the speech voice interval;

[0026] According to the determined speech start time, a second time series is obtained.

[0027] Optionally, determining the speech start time of the speech voice interval of the second user in the second audio data according to the time relationship between the silence interval and the speech voice interval includes at least one of the following:

[0028] If the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silence interval, and the interval between the actual speech start time and the start time of the silence interval is less than a first threshold, then the actual speech start time is used as the speech start time of the speech voice interval;

[0029] If the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is earlier than the start time of the silence interval, the actual speech start time is used as the speech start time of the speech voice interval;

[0030] If the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silence interval, and the interval between the actual speech start time and the start time of the silence interval is greater than or equal to a second threshold, then the start time of the silence interval is used as the speech start time of the speech voice interval;

[0031] If the speech voice interval and the silence interval do not overlap, the start time of the silence interval is used as the speech start time of the speech voice interval.

[0032] The present application also provides an audio processing device, including:

[0033] A first acquisition module is used to acquire audio content correlation information and audio timing correlation information between a first user and a second user in an online conference;

[0034] A first processing module, configured to divide the first user and the second user into virtual groups according to the audio content correlation information and the audio timing correlation information;

[0035] The first playback module is used to play the audio of each member in the virtual group where the first user is located. The first user is the terminal user, and the second user is other terminal users in the online meeting except the first user.

[0036] Optionally, obtaining audio content correlation information between a first user and a second user in an online conference includes:

[0037] Acquire a first vocabulary set in a first text, wherein the first text is a text of first audio data of the first user in the online conference, and the first vocabulary set includes: vocabulary ranked in the top Z positions in terms of appearance frequency in the first text, where Z is an integer greater than or equal to 1;

[0038] Acquire a second vocabulary set in a second text, wherein the second text is a text of second audio data of the second user in the online conference, and the second vocabulary set includes: vocabulary ranked in the top Z positions in terms of frequency of occurrence in the second text; and the second vocabulary set has the same total number of elements as the first vocabulary set;

[0039] Obtaining the number of words that match between the first vocabulary set and the second vocabulary set;

[0040] The similarity between the first vocabulary set and the second vocabulary set is determined according to the number of words and the total number of elements, and the similarity is used as the audio content correlation information between the first user and the second user.

[0041] Optionally, obtaining audio timing correlation information between a first user and a second user in an online conference includes:

[0042] Calculating the audio connection between the first user and the second user;

[0043] Audio timing correlation information between the first user and the second user is obtained according to the audio connectivity.

[0044] Optionally, calculating the audio connectivity between the first user and the second user includes:

[0045] Acquire a first time series formed by end times of each speech voice interval of the first user in the first audio data; the first audio data is the audio data of the first user in the online conference;

[0046] Acquire a second time series formed by the start time of each speech voice interval of the second user in the second audio data; the second audio data is the audio data of the second user in the online conference;

[0047] Acquire an audio interval value between the first user and the second user according to the first time series and the second time series;

[0048] The audio connection degree between the first user and the second user is calculated according to the audio interval value.

[0049] Optionally, the obtaining a second time series formed by start times of each speech interval of the second user in the second audio data includes:

[0050] Determine a silent interval of the first user corresponding to a speech interval of the second user; wherein the speech interval of the second user refers to a speech interval of the second user in the second audio data of the online conference, and the silent interval of the first user refers to a silent interval of the first user in the first audio data of the online conference;

[0051] Determining a speech start time of the speech voice interval of the second user in the second audio data according to a time relationship between the silence interval and the speech voice interval;

[0052] According to the determined speech start time, a second time series is obtained.

[0053] Optionally, determining the speech start time of the speech voice interval of the second user in the second audio data according to the time relationship between the silence interval and the speech voice interval includes at least one of the following:

[0054] If the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silence interval, and the interval between the actual speech start time and the start time of the silence interval is less than a first threshold, then the actual speech start time is used as the speech start time of the speech voice interval;

[0055] If the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is earlier than the start time of the silence interval, the actual speech start time is used as the speech start time of the speech voice interval;

[0056] If the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silence interval, and the interval between the actual speech start time and the start time of the silence interval is greater than or equal to a second threshold, then the start time of the silence interval is used as the speech start time of the speech voice interval;

[0057] If the speech voice interval and the silence interval do not overlap, the start time of the silence interval is used as the speech start time of the speech voice interval.

[0058] An embodiment of the present application also provides an audio processing device, including a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, the above-mentioned audio processing method is implemented.

[0059] The embodiment of the present application also provides a readable storage medium on which a program is stored. When the program is executed by a processor, the steps in the above-mentioned audio processing method are implemented.

[0060] The embodiment of the present application also provides a computer program product, including computer instructions, which implement the steps of the above-mentioned audio processing method when executed by a processor.

[0061] The beneficial effects of the above technical solution of the present application are as follows:

[0062] In the above scheme, the audio processing method obtains audio content correlation information and audio timing correlation information between the first user and the second user in the online meeting; divides the first user and the second user into virtual groups according to the audio content correlation information and the audio timing correlation information; plays the audio of each member of the virtual group in which the first user is located, wherein the first user is the terminal user, and the second user is other terminal users in the online meeting except the first user; and can support dividing the participants into groups with different topics based on the analysis of the audio of the participants, and all participants in the online meeting only need to enter the same conference room to only hear the voices of the members of their group, without having to repeatedly enter and exit different conference rooms, thereby improving the meeting efficiency and solving the problem of low efficiency of online conference solutions in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A schematic diagram of the flow of the audio processing method according to an embodiment of the present application;

[0064] Figure 2 A schematic diagram of a specific implementation flow of the audio processing method according to an embodiment of the present application;

[0065] Figure 3 A schematic diagram of discretization of single-person audio according to an embodiment of the present application;

[0066] Figure 4 This is a schematic diagram of multi-person audio discretization according to an embodiment of the present application;

[0067] Figure 5 A schematic diagram of a user speech timing analysis in an embodiment of the present application;

[0068] Figure 6 Schematic diagram of the structure of an audio processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the technical problems, technical solutions and advantages to be solved by the present application clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0070] This application aims to solve the problem of low efficiency of online conference solutions in the prior art and provides an audio processing method, such as Figure 1 As shown, including:

[0071] Step 11: Obtaining audio content correlation information and audio timing correlation information between the first user and the second user in the online conference;

[0072] Step 12: Divide the first user and the second user into virtual groups according to the audio content correlation information and the audio timing correlation information;

[0073] Step 13: Play the audio of each member of the virtual group in which the first user is located, where the first user is the terminal user, and the second user is other terminal users in the online meeting except the first user.

[0074] Among them, the audio content relevance information can indicate the similarity of the topics discussed in the meeting, such as the ratio of similar words between content 1 published by user A and content 2 published by user B; the audio timing relevance information can indicate the degree of connection between speeches in the meeting, such as the correlation between the speech intervals between user A and user B, etc., which is not limited here.

[0075] The audio processing method provided in the embodiment of the present application obtains audio content correlation information and audio timing correlation information between the first user and the second user in an online meeting; divides the first user and the second user into virtual groups according to the audio content correlation information and the audio timing correlation information; plays the audio of each member of the virtual group in which the first user is located, wherein the first user is the terminal user, and the second user is other terminal users in the online meeting except the first user; and can support dividing the participants into groups with different themes based on the analysis of the participant audio (such as the first audio data and the second audio data mentioned above), and all participants in the online meeting only need to enter the same conference room to hear only the voices of the members of their group, without having to repeatedly enter and exit different conference rooms, thereby improving the meeting efficiency and solving the problem of low efficiency of online conference solutions in the prior art.

[0076] Wherein, obtaining the audio content relevance information between the first user and the second user in the online meeting includes: obtaining a first vocabulary set in the first text, wherein the first text is the text of the first audio data of the first user in the online meeting, and the first vocabulary set includes: the vocabulary ranked in the first Z position by the frequency of occurrence in the first text, and Z is an integer greater than or equal to 1; obtaining a second vocabulary set in the second text, wherein the second text is the text of the second audio data of the second user in the online meeting, and the second vocabulary set includes: the vocabulary ranked in the first Z position by the frequency of occurrence in the second text; the second vocabulary set has the same total number of elements as the first vocabulary set; obtaining the number of vocabulary matched between the first vocabulary set and the second vocabulary set; according to the number of vocabulary and the total number of elements, determining the similarity between the first vocabulary set and the second vocabulary set, and using the similarity as the audio content relevance information between the first user and the second user. In this way, the audio content relevance information can be accurately obtained. Wherein, the "number of matched vocabulary" may include: the number of completely consistent and / or similar vocabulary, such as the same word or synonyms, etc., which is not limited here.

[0077] In an embodiment of the present application, obtaining the audio timing correlation information between the first user and the second user in an online meeting includes: calculating the audio connectivity between the first user and the second user; and obtaining the audio timing correlation information between the first user and the second user based on the audio connectivity. In this way, the audio timing correlation information can be accurately obtained. Among them, "calculating the audio connectivity between the first user and the second user" may include: discretizing the audio data of the first user and the second user based on the timing to obtain discretized voice data; and in the discretized voice data, marking the voice segments within the preset frequency range and with a duration exceeding the preset time as the first value, and marking the voice segments outside the preset frequency range and / or with a duration not exceeding the preset time as the second value; based on the first value and the second value marked in the timing by the first user and the second user, the audio connectivity between the first user and the second user is calculated, which can also be called voice connectivity. Regarding "obtaining the audio timing correlation information between the first user and the second user based on the audio connectivity", it may include: obtaining the audio timing correlation information between the first user and the second user based on the audio connectivity and a first formula; wherein the first formula is: P2=1-DX, P2 represents the audio timing correlation information, and DX represents the audio connectivity, but is not limited to this.

[0078] The calculation of the audio connection between the first user and the second user includes: obtaining a first time series consisting of the end time of each speech voice interval of the first user in the first audio data; the first audio data is the audio data of the first user in the online meeting; obtaining a second time series consisting of the start time of each speech voice interval of the second user in the second audio data; the second audio data is the audio data of the second user in the online meeting; according to the first time series and the second time series, obtaining the audio interval value between the first user and the second user; according to the audio interval value, calculating the audio connection between the first user and the second user. In this way, the audio connection can be obtained more specifically and accurately. The operation of obtaining the first time series and / or the second time series can be performed based on the above-mentioned first value and second value, but is not limited thereto; in addition, the voice of one person in the entire meeting can be divided into at least one speech voice interval according to silence, for example: the first audio data can include: at least one silence interval and at least one speech voice interval.

[0079] In an embodiment of the present application, the second time sequence formed by the start time of each speech voice interval of the second user in the second audio data includes: determining the silent interval of the first user corresponding to the speech voice interval of the second user; wherein the speech voice interval of the second user refers to the speech voice interval of the second user in the second audio data of the online meeting, and the silent interval of the first user refers to the silent interval of the first user in the first audio data of the online meeting; according to the time relationship between the silent interval and the speech voice interval, determining the speech start time of the speech voice interval of the second user in the second audio data; according to the determined speech start time, obtaining the second time sequence. In this way, the second time sequence can be accurately obtained. wherein, the interval can be a time interval, such as a speech voice interval is a time interval of speech voice; and / or, the time relationship includes a time overlap, or the time relationship includes a time overlap, and the early or late situation between the actual speech start time of the second user corresponding to the speech voice interval and the start time of the silent interval; but it is not limited to this. Among them, "determining the silent interval of the first user corresponding to the speech voice interval of the second user" may include: determining the silent interval of the first user corresponding to the speech voice interval of the second user by sorting in time; for example: after sorting the speech voice interval of the first user and the speech voice interval of the second user respectively (which may be sorted in chronological order), the first silent interval of the first user is obtained, and the first silent interval is used as the silent interval corresponding to the first speech voice interval of the second user, but it is not limited to this.

[0080] Wherein, the method of determining the speech start time of the speech voice interval of the second user in the second audio data according to the time relationship between the silence interval and the speech voice interval includes at least one of the following items: (1) if the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silence interval, and the interval between the actual speech start time and the start time of the silence interval is less than a first threshold, then the actual speech start time is used as the speech start time of the speech voice interval; (2) if the speech voice interval overlaps with the silence interval, and the second user corresponding to the speech voice interval (1) If the actual speech start time of the first user is earlier than the start time of the silent interval, the actual speech start time is used as the speech start time of the speech voice interval; (2) If the speech voice interval overlaps with the silent interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silent interval, and the interval between the actual speech start time and the start time of the silent interval is greater than or equal to a second threshold, the start time of the silent interval is used as the speech start time of the speech voice interval; (3) If the speech voice interval overlaps with the silent interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silent interval, and the interval between the actual speech start time and the start time of the silent interval is greater than or equal to a second threshold, the start time of the silent interval is used as the speech start time of the speech voice interval. In this way, the speech start time of the speech voice interval of the second user in the second audio data can be accurately obtained. Regarding the above items, examples are as follows:

[0081] Item (1), for example: the speech interval 1 of the second user in the second audio data corresponds to the silence interval 1 of the first user in the first audio data; the silence interval 1 is t1-t3, the actual speech start time of the second user corresponding to the speech interval 1 is t2, and the interval between t2 and t1 is less than the first threshold, then the actual speech start time t2 is used as the speech start time of the speech interval;

[0082] Item (2), for example: the speech voice interval 1 of the second user in the second audio data corresponds to the silence interval 1 of the first user in the first audio data; the silence interval 1 is t5-t6, the actual speech start time of the second user corresponding to the speech voice interval 1 is t4, and the actual speech end time of the second user corresponding to the speech voice interval 1 is between t5 and t6, then the actual speech start time t4 is used as the speech start time of the speech voice interval;

[0083] Item (3), for example: the speech interval 1 of the second user in the second audio data corresponds to the silent interval 1 of the first user in the first audio data; the silent interval 1 is t7 to t9, the actual speech start time of the second user corresponding to the speech interval 1 is t8, and the interval between t8 and t7 is greater than or equal to the second threshold, then the start time t7 of the silent interval is used as the speech start time of the speech interval;

[0084] Item (4), for example: the speaking voice interval 1 of the second user in the second audio data corresponds to the silent interval 1 of the first user in the first audio data; the silent interval 1 is t10~t11, and the actual speech start time of the second user corresponding to the speaking voice interval 1 is t12, then the start time t10 of the silent interval is used as the speaking start time of the speaking voice interval.

[0085] In the embodiment of the present application, the first time series and the second time series contain the same number of time information; the obtaining of the audio interval value between the first user and the second user according to the first time series and the second time series includes: obtaining the mean of the first difference between the time information at the corresponding sorting position in the first time series and the second time series as the audio interval value between the first user and the second user. In this way, the audio interval value can be accurately obtained. Among them, the "mean of the first difference" can specifically include: the mean of the absolute value of the first difference, but is not limited to this.

[0086] The step of calculating the audio connection between the first user and the second user according to the audio interval value includes: obtaining a second difference between the interval experience value and the audio interval value; and calculating the audio connection between the first user and the second user according to the first difference, the second difference and the interval experience value. In this way, the audio connection can be accurately obtained. The first difference may specifically include: an absolute value of the first difference, but is not limited thereto.

[0087] In the embodiment of the present application, the virtual grouping of the first user and the second user according to the audio content correlation information and the audio timing correlation information includes: obtaining the audio similarity between the first user and the second user according to the audio content correlation information and the audio timing correlation information; and the virtual grouping of the first user and the second user according to the audio similarity and the grouping threshold. In this way, accurate grouping can be achieved.

[0088] Wherein, the acquisition of the audio content correlation information and the audio timing correlation information between the first user and the second user in the online meeting includes: acquiring the audio content correlation information and the audio timing correlation information in a second manner; the second manner includes: at least one of a periodic manner and a conditional triggering manner; and / or, the virtual group division of the first user and the second user according to the audio content correlation information and the audio timing correlation information includes: in a third manner, the virtual group division of the first user and the second user according to the audio content correlation information and the audio timing correlation information; the third manner includes: at least one of a periodic manner and a conditional triggering manner. This can support dynamic grouping and further improve the accuracy of grouping. Wherein, the conditional triggering manner is, for example, detecting that the speech content of a certain group member in the virtual group has a relatively low correlation with the speech content of other group members, that is, the audio content correlation information indicates that the correlation is lower than the threshold value, then it can trigger the re-group division, such as returning to the operation of "obtaining the audio content correlation information and the audio timing correlation information between the first user and the second user in the online meeting".

[0089] Furthermore, the audio processing method further includes: canceling the virtual group division according to the instruction information, and playing the audio of all users in the online meeting. This can make the solution more user-friendly, support active or passive cancellation of virtual grouping, and meet the diverse needs of users. Among them, the instruction information may include: instructions input by the first user, and / or instructions input by other participating users, etc., which are not limited here.

[0090] The following is an example of the audio processing method provided in the embodiment of the present application.

[0091] In view of the above technical problems, the embodiment of the present application provides an audio processing method, which can be specifically implemented as an audio extraction method based on a multi-person virtual conference, which can support: dynamically dividing groups of different topics based on the analysis of the audio of the participants, and all participants only need to enter the same conference room to only hear the voice related to the topic (that is, the group they are in), without having to repeatedly enter and exit different conference rooms. The present application mainly involves: extracting each independent audio file of each participant in the meeting, and calculating the topic relevance factor of the content discussed by the two participants (including: speech content relevance factor P1 and speech timing relevance factor P2), and then combining the weight calculation of P1 and P2, dividing all participants in the same conference room into different virtual groups according to the relevance of the discussion topic, and each virtual group can only hear the voice of the participants in this group; this part of the operation can correspond to the above-mentioned acquisition of the audio content relevance information and audio timing relevance information between the first user and the second user in the online meeting; dividing the first user and the second user into virtual groups according to the audio content relevance information and the audio timing relevance information; playing the audio of each member of the virtual group where the first user is located. The speech content correlation factor P1 corresponds to the above-mentioned audio content correlation information, and the speech timing correlation factor P2 corresponds to the above-mentioned audio timing correlation information.

[0092] Specifically, Figure 2 As shown, the embodiments of the present application may specifically include the following contents:

[0093] 1. Audio preprocessing;

[0094] (1) The voice data of each participant is processed separately. For example, the audio data of the first participant is extracted, and then the high-frequency words are extracted; the voice data of the second participant is processed in the same way, and the high-frequency words are also extracted; the frequency of occurrence of words can be obtained using the following formula:

[0095] Among them, TF(t,d) represents the word frequency; the number of occurrences (t,d) represents the number of times t that a word appears in a text length d, and the text length d corresponds to the extracted audio data of the person.

[0096] (2) Analysis of the relevance factor P1 of speech content;

[0097] Arrange the language vocabulary of each participant from high to low according to the word frequency TF(t,d), and extract the first 5 words as the high-frequency word grouping of each participant. Then compare the high-frequency word grouping of a participant with that of other participants. For each identical word, the hit number N is increased by 1. The following formula 1 can be used to calculate the relevance factor P1 between the speech content of the participant and other participants, where the hit number N is the hit number of the high-frequency word grouping of the two (corresponding to the number of words mentioned above), and 5 is the total number of comparison data (corresponding to the total number of elements mentioned above). A numerical value between 0 and 1 is obtained, and the higher the value, the higher the similarity of the topics between the two. This part of the operation may correspond to the above-mentioned acquisition of a first vocabulary set in a first text, wherein the first text is a text of the first audio data of the first user in the online meeting, and the first vocabulary set includes: vocabulary ranked in the top Z positions in terms of frequency of occurrence in the first text, where Z is an integer greater than or equal to 1; acquisition of a second vocabulary set in a second text, wherein the second text is a text of the second audio data of the second user in the online meeting, and the second vocabulary set includes: vocabulary ranked in the top Z positions in terms of frequency of occurrence in the second text; the second vocabulary set has the same total number of elements as the first vocabulary set; acquisition of the number of vocabulary that matches the first vocabulary set and the second vocabulary set; determination of the similarity between the first vocabulary set and the second vocabulary set based on the number of vocabulary and the total number of elements, and use of the similarity as audio content correlation information between the first user and the second user.

[0098] The above formula 1 is: P1=N / 5.

[0099] For example: assuming that the high-frequency word grouping of participant A is {server, startup, weak network, front-end, crash}, and the high-frequency word grouping of participant B is {weak network, front-end, crash, log, startup}, then A and B will have the four phrases "weak network", "front-end", "crash" and "startup" hits, and their content similarity is 80% (4 / 5), that is, the speech content relevance factor P1 = 80%, which is highly similar.

[0100] 2. Speech timing correlation factor P2;

[0101] This part mainly involves: calculating the connection degree between each speech of the participants to obtain the time correlation parameters; specifically including:

[0102] (1) Audio data is discretized in time series;

[0103] The audio data of each participant is processed for data discretization and normalization to obtain a discretization result, so as to mark whether each person is speaking at each moment. Specifically, considering that the hertz range of human speech is 85Hz to 1100Hz, the raw audio data of each person can be judged in time order. If the hertz is within 85Hz to 1100Hz, it means that the person is speaking, which can be marked as 1; if it is not within this range, it means that the person is not speaking (i.e., silent), which can be marked as 0. Based on this, we can get Figure 3 The discretization result of a single person's audio is shown in FIG. The speaking time period with a speaking duration of less than 1 second can be marked as 0, which means no speaking, to exclude interference from other situations such as coughing. Figure 3 The effect of discretizing the audio data of a single person is shown in the form of a line graph. Non-0 indicates that the person is speaking, and 0 indicates silence. The user in the graph is speaking from 0 to t1, silent from t1 to t2, speaking from t2 to t3, silent from t3 to t4, and speaking from t4 to t5.

[0104] In the embodiment of the present application, in order to facilitate subsequent analysis, the audio dispersion results of all people in the meeting (i.e., the above discretization results) can be combined and displayed intuitively on the same picture, so that the speech situation of all participants can be found vertically from the perspective of the time axis. Figure 4 As shown in the figure, at t1, only users B and C are talking in the entire conference room; at t2, users A and B are silent, and users C and D are talking; Figure 4 It can be seen that when users B and D were speaking, the other party was silent, and a question-and-answer situation was formed between them. It is likely that they were discussing the same issue. This time interval can be quantified later. Figure 4 In (Display of Discretized Data for Multiple People), all horizontal lines represent non-zero data, and the corresponding user is speaking at this time; horizontal lines of different line types correspond to different users.

[0105] (2) Analysis of the P2 factor related to speech timing;

[0106] Under normal circumstances, participants of the same topic in the same conference room speak in turn, that is, only one person can speak on the same topic at the same time, and there may be a short period of overlapping voices and silence (i.e., quiet period) in between. Therefore, the embodiment of the present application uses the time interval of a user's silence to judge the degree of connection between other users' voices and the user's own voice. Specifically, it can be quantified based on the time interval between the other party's speech start time and the user's speech end time. Taking participant A (corresponding to the first user mentioned above) as an example, his discussion topic is tentatively set as Topic 1, Figure 5 His four silent time intervals in the meeting are marked above: Q1, Q2, Q3, and Q4. Other participants in Topic 1 (corresponding to user B in the figure) mainly speak during these four silent windows (i.e., silent time intervals). Figure 5 In (Schematic diagram of user speech timing analysis): all horizontal lines represent non-zero data, and the corresponding user is speaking at this time; horizontal lines of different line types correspond to different users; Q1...Q4 are the four silent periods of participant A (i.e., user A), i.e., silent time intervals, which can correspond to the above-mentioned silent intervals.

[0107] In the embodiment of the present application, starting from the perspective of participant A, find other potential participants of the common topic Topic 1, and calculate the time sequence similarity between participant A and participant B from the silent period of participant A; the time sequence similarity is the speech timing correlation factor P2. Assuming that participant A has M silent periods (Q1-Q4 in the figure, that is, M is 4), check the status of other participants when each speech of participant A ends; at this time, any other participant (such as participant B, which can correspond to the second user mentioned above) has only the following two states:

[0108] State 1, speaking: Figure 5 The Q2 time zone may correspond to the situation where participant B is speaking when participant A starts to be silent. This situation is generally when participant B interrupts participant A's speech or participant B is participating in other topic discussions. In this case, the start time of participant B's speech (which may correspond to the start time of the second user's speech interval) may be recorded as TY i = t4, where the start time of this speech segment refers to the start time of this speech, not the time when participant B starts speaking for the first time in the entire meeting. A person's speech in the entire meeting can be divided into at least one speech segment according to the silence period. Based on the above, in this state, the end time TX of participant A's speech segment can be marked i = t5, the start time of participant B’s speech segment TY i =t4; the processing in this state may correspond to the above-mentioned if the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is earlier than the start time of the silence interval, then the actual speech start time (corresponding to t4 in the figure) will be used as the speech start time of the speech voice interval.

[0109] State 2, silent, can include the following three situations:

[0110] Case ①: After a short period of time, participant B starts to speak, corresponding to interval Q1 in the figure; in this case, the end time of participant A's speech can be marked as TX i = t1, marking the start time of participant B’s speech segment TY i= t2; the processing in this case may correspond to the above-mentioned: if the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silence interval, and the interval between the actual speech start time and the start time of the silence interval is less than the first threshold, then the actual speech start time (corresponding to t2 in the figure) is used as the speech start time of the speech voice interval;

[0111] Case ②, after a long pause, participant B starts to speak, corresponding to interval Q3 in the figure. In this case, after participant A muted and before participant B spoke, there may be a third person speaking; in the embodiment of the present application, TS=10s can be set according to data statistics, representing the time threshold for the third person to speak; based on this, if it is determined that the pause time (i.e. t8-t7 in the figure) is greater than TS in interval Q3, the end time of participant A's speech in this segment is marked as TX i = t7, marking the start time of participant B's speech as the end time of participant A's speech, that is, TY i =t7; the processing in this case may correspond to the overlap between the above-mentioned speech voice interval and the silent interval, and the actual speech start time of the second user corresponding to the speech voice interval (corresponding to t8 in the figure) is later than the start time of the silent interval (corresponding to t7 in the figure), and the interval between the actual speech start time and the start time of the silent interval is greater than or equal to the second threshold, then the start time of the silent interval is used as the speech start time of the speech voice interval; this can support avoiding the situation where the discussion topics of participant A and participant B are highly correlated, but the timing causes a very low error in P2;

[0112] Case ③: before participant A speaks next, participant B has not spoken, which corresponds to interval Q4 in the figure. In this case, participant B has not spoken in the entire silent interval Q4, and there may be a third person speaking; the end time of participant A's speech in this section can be marked as TX i = t10, marking the start time of participant B's speech as the end time of participant A's speech, that is, TY i =t10; the processing in this case may correspond to the above-mentioned case where if there is no overlap between the speech voice interval and the silence interval, the start time of the silence interval (corresponding to t10 in the figure) is used as the speech start time of the speech voice interval; this may also support avoiding the situation where the discussion topics of participant A and participant B are highly correlated, but the timing causes a very low error in P2.

[0113] Based on the above processing, we can get the time series TX1...TX1 of each speech of participant A. m (which may correspond to the first time sequence mentioned above), and the time sequence TY1...TY1 at the beginning of each speech of participant Bm (may correspond to the above second time series); TY1...TY m The values ​​in are different values ​​according to the above situations, and are not necessarily the actual start time of each speech by participant B.

[0114] Afterwards, use the above two groups of sequences to substitute into the following formula 2 to obtain the average value T of the subtraction results of the two groups of data; this operation can correspond to the above-mentioned acquisition of the average of the first difference between the time information of the corresponding sorting positions in the first time series and the second time series, as the audio interval value between the first user and the second user.

[0115] The above formula 2 is:

[0116] In the embodiment of the present application, TU=2s can be set according to statistical analysis, and TU represents the empirical value of the voice interval between two people when two people communicate normally. In order to facilitate the calculation of the distribution discreteness of the statistical data time series relative to the empirical value TU, the difference ΔT between the mean value T and the empirical value TU can be calculated using the following formula 3; this operation can correspond to the second difference between the interval empirical value and the audio interval value obtained above.

[0117] The above formula 3 is: ΔT = TU-T;

[0118] Based on the above processing, the variance value of the difference (absolute value) between the end time of participant A's speech and the start time of participant B's speech relative to the speech interval experience value TU (that is, the variance of the speech interval relative to the experience value) can be calculated; corresponding to the above, according to the first difference, the second difference and the interval experience value, the audio connection between the first user and the second user can be calculated. Among them, the variance can help evaluate the stability and variability of the data. In a set of data, if the variance is small, it means that the speech connection between participants A and B is high, and it is likely that they are discussing the same topic; and if the variance is large, it means that the speech connection between participants A and B is low, and it is likely that they are not discussing the same topic. Specifically, the variance value DX relative to TU can be calculated using the following formula:

[0119]

[0120] Among them, DX can correspond to the above-mentioned audio cohesion. The smaller DX is, the higher the voice cohesion is, and the larger DX is, the lower the voice cohesion is. Finally, the speech timing correlation factor P2 can be obtained by the following inverse formula:

[0121] P2=1-DX; corresponding to the above, the audio timing correlation information between the first user and the second user is obtained according to the audio connection degree.

[0122] 3. Virtual group division;

[0123] After the above processing, the speech content correlation factor P1 and the speech timing correlation factor P2 can be calculated, wherein the similarity division of P1 involves coarse-grained division, which can be calibrated in combination with P2 to achieve accurate grouping.

[0124] In the embodiment of the present application, after a large number of sample statistical analyses, it is found that the speech similarity P of participants A and B is composed of P1 and P2 in proportion (i.e., P1 and P2 are used to calculate the similarity P of participants A and B in proportion), where the proportion of P1 can be 0.4, and the proportion of P2 can be 0.6. Based on this, the calculation formula for the speech similarity P (which can correspond to the above-mentioned audio similarity) is as follows:

[0125] P = P1 × 0.4 + P2 × 0.6;

[0126] In addition, an empirical value PU=0.7 may be set, which may be obtained through data statistics and may correspond to the above-mentioned grouping threshold. If the speech similarity P≥PU of the two participants, it can be confirmed that the two participants are discussing the same topic and can be divided into the same virtual group; this operation may correspond to the above-mentioned acquisition of the audio similarity between the first user and the second user based on the audio content correlation information and the audio timing correlation information; and the division of the first user and the second user into virtual groups based on the audio similarity and the grouping threshold.

[0127] In the embodiment of the present application, the speech similarity P value between the participant and all other participants can be calculated, and those with P≥PU are divided into the same virtual group; that is, the participants are grouped according to the similarity P; for example, there are participants A, B, C, and D in total, and from the perspective of participant A, the similarity P relative to participant B needs to be calculated. B , similarity P relative to participant C C , similarity P relative to participant D D ; Then P B , P C , P D They are compared with PU respectively, and those that meet the condition (P≥PU) are divided into a group, for example, {A, B} become a group.

[0128] Based on the above division of virtual groups, the terminal side of any member participating in the online meeting (which may correspond to the above-mentioned terminal, that is, the terminal corresponding to the first user or the terminal used by the first user) can already know the members who belong to the same virtual group as the user on the terminal side; after receiving the voice data, the conference software can filter and process it according to the virtual group to which the member belongs, and only send the voice data of the corresponding group member to the speaker; this operation can correspond to the above-mentioned playback of the audio of each member in the virtual group where the first user is located. For example, in the above analysis, participant A can only hear the voice of participant B, and the voices of participants C and D are not played, so as not to interfere with the discussion of participants A and B; at this time, participants C and D may also be in the same group, and they can only hear each other's voices; in this way, through the same online conference room, participants can conduct their own discussions in different groups at the same time without affecting each other.

[0129] From the above, the scheme provided by the embodiment of the present application can judge different voice discussion groups through the time sequence of the participants' speeches + the relevance of the speech content; it involves: based on the audio data of each participant in the online meeting, the content relevance factor and the time sequence relevance factor between the audio data of all participants are calculated, and the participants are divided into virtual groups according to the content relevance factor and the time sequence relevance factor. And in each virtual group, the voices of non-group members are blocked. Among them, the content relevance factor can represent the similarity of the topics discussed in the meeting, and the time sequence relevance factor can represent the degree of connection between the speeches in the meeting. If the topics are similar according to the content relevance factor, and the conference speeches are connected according to the time sequence relevance factor, it can be judged that the relevant participants are discussing the same topic, and are subsequently divided into the same virtual group; using this method, group division can be performed according to the actual discussion situation in the meeting. Specifically:

[0130] In an embodiment of the present application, audio data of each participant in an online meeting can be extracted; high-frequency words are extracted from the audio data of each participant, and content correlation factors between all participants are calculated based on the high-frequency words; the audio data of each participant is discretized based on time sequence to obtain discretized voice data, and in the discretized voice data, voice segments within a preset frequency range and with a duration exceeding a preset time are marked as 1, and voice segments outside the preset frequency range or with a duration not exceeding the preset time are marked as 0; based on the 0 and 1 marks of each participant in the time sequence, the voice connectivity between all participants is obtained, and the time sequence correlation factor between all participants is obtained based on the voice connectivity; the participants are divided into virtual groups according to the content correlation factor and the time sequence correlation factor, and the voices of non-group members are blocked in each virtual group.

[0131] In summary, the embodiment of the present application introduces a comprehensive analysis of the time sequence of participants' speeches and the relevance of their speech content, which can dynamically divide groups and perform corresponding voice playback.

[0132] The present application also provides an audio processing device, such as Figure 6 As shown, including:

[0133] A first acquisition module 61 is used to acquire audio content correlation information and audio timing correlation information between a first user and a second user in an online conference;

[0134] A first processing module 62, configured to divide the first user and the second user into virtual groups according to the audio content correlation information and the audio timing correlation information;

[0135] The first playback module 63 is used to play the audio of each member in the virtual group where the first user is located. The first user is the terminal user, and the second user is other terminal users in the online meeting except the first user.

[0136] The audio processing device provided in the embodiment of the present application obtains audio content correlation information and audio timing correlation information between a first user and a second user in an online meeting; divides the first user and the second user into virtual groups according to the audio content correlation information and the audio timing correlation information; plays the audio of each member of the virtual group in which the first user is located, wherein the first user is the terminal user, and the second user is other terminal users in the online meeting except the first user; and can support dividing the participants into groups with different themes based on the analysis of the audio of the participants, and all participants in the online meeting only need to enter the same conference room to hear only the voices of the members of their group, without having to repeatedly enter and exit different conference rooms, thereby improving the efficiency of the meeting and solving the problem of low efficiency of online meeting solutions in the prior art.

[0137] In an embodiment of the present application, obtaining audio content correlation information between a first user and a second user in an online meeting includes: obtaining a first vocabulary set in a first text, wherein the first text is the text of the first audio data of the first user in the online meeting, and the first vocabulary set includes: vocabulary ranked in the top Z positions in terms of frequency of occurrence in the first text, where Z is an integer greater than or equal to 1; obtaining a second vocabulary set in a second text, wherein the second text is the text of the second audio data of the second user in the online meeting, and the second vocabulary set includes: vocabulary ranked in the top Z positions in terms of frequency of occurrence in the second text; the second vocabulary set has the same total number of elements as the first vocabulary set; obtaining the number of words that match the first vocabulary set and the second vocabulary set; determining the similarity between the first vocabulary set and the second vocabulary set based on the number of words and the total number of elements, and using the similarity as the audio content correlation information between the first user and the second user.

[0138] Among them, obtaining the audio timing correlation information between the first user and the second user in the online meeting includes: calculating the audio connectivity between the first user and the second user; and obtaining the audio timing correlation information between the first user and the second user according to the audio connectivity.

[0139] In an embodiment of the present application, the calculation of the audio connectivity between the first user and the second user includes: obtaining a first time series formed by the end time of each speech voice interval of the first user in the first audio data; the first audio data is the audio data of the first user in the online meeting; obtaining a second time series formed by the start time of each speech voice interval of the second user in the second audio data; the second audio data is the audio data of the second user in the online meeting; according to the first time series and the second time series, obtaining the audio interval value between the first user and the second user; according to the audio interval value, calculating the audio connectivity between the first user and the second user.

[0140] Among them, the obtaining of the second time series formed by the starting time of each speech voice interval of the second user in the second audio data includes: determining the silent interval of the first user corresponding to the speech voice interval of the second user; wherein, the speech voice interval of the second user refers to the speech voice interval of the second user in the second audio data of the online meeting, and the silent interval of the first user refers to the silent interval of the first user in the first audio data of the online meeting; according to the time relationship between the silent interval and the speech voice interval, determining the speech start time of the speech voice interval of the second user in the second audio data; according to the determined speech start time, obtaining the second time series.

[0141] In the embodiment of the present application, the determination of the speech start time of the speech voice interval of the second user in the second audio data according to the time relationship between the silence interval and the speech voice interval includes at least one of the following items: (1) if the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silence interval, and the interval between the actual speech start time and the start time of the silence interval is less than a first threshold, then the actual speech start time is used as the speech start time of the speech voice interval; (2) if the speech voice interval overlaps with the silence interval, and the second user corresponding to the speech voice interval (1) if the actual speech start time of the user is earlier than the start time of the silent interval, the actual speech start time is used as the speech start time of the speech voice interval; (2) if the speech voice interval overlaps with the silent interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silent interval, and the interval between the actual speech start time and the start time of the silent interval is greater than or equal to a second threshold, the start time of the silent interval is used as the speech start time of the speech voice interval; (3) if the speech voice interval overlaps with the silent interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silent interval, and the interval between the actual speech start time and the start time of the silent interval is greater than or equal to a second threshold, the start time of the silent interval is used as the speech start time of the speech voice interval; (4) if the speech voice interval does not overlap with the silent interval, the start time of the silent interval is used as the speech start time of the speech voice interval.

[0142] Among them, the implementation embodiments of the above-mentioned audio processing method are all applicable to the embodiments of the audio processing device and can achieve the same technical effects.

[0143] An embodiment of the present application also provides an audio processing device, including a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, the above-mentioned audio processing method is implemented.

[0144] Among them, the implementation embodiments of the above-mentioned audio processing method are all applicable to the embodiments of the audio processing device and can achieve the same technical effects.

[0145] The embodiment of the present application also provides a readable storage medium on which a program is stored. When the program is executed by a processor, the steps in the above-mentioned audio processing method are implemented.

[0146] Among them, the implementation embodiments of the above-mentioned audio processing method are all applicable to the embodiments of the readable storage medium and can achieve the same technical effects.

[0147] The embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the method embodiment of the above-mentioned audio processing method are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0148] It should be noted that many functional components described in this specification are referred to as modules in order to more particularly emphasize the independence of their implementation methods.

[0149] In the embodiment of the present application, module can be implemented with software so that it can be executed by various types of processors. For example, an executable code module of an identification can include one or more physical or logical blocks of computer instructions, for example, it can be constructed as an object, process or function. Nevertheless, the executable code of the identified module does not need to be physically located together, but can include different instructions stored in different positions, and when these instructions are logically combined together, it constitutes a module and realizes the specified purpose of the module.

[0150] In fact, executable code module can be a single instruction or many instructions, and can even be distributed on a plurality of different code segments, distributed among different programs, and distributed across a plurality of memory devices. Similarly, operating data can be identified in the module, and can be implemented and organized in the data structure of any appropriate type according to any appropriate form. The operating data can be collected as a single data set, or can be distributed in different locations (including on different storage devices), and can only be present on a system or network as an electronic signal at least in part.

[0151] When a module can be implemented by software, considering the level of existing hardware technology, a person skilled in the art can build a corresponding hardware circuit to implement the corresponding function of the module that can be implemented by software without considering the cost. The hardware circuit includes a conventional very large scale integration (VLSI) circuit or gate array and existing semiconductors such as logic chips, transistors, or other discrete components. The module can also be implemented by a programmable hardware device, such as a field programmable gate array, a programmable array logic, a programmable logic device, etc.

[0152] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), including a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0153] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary personnel in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An audio processing method, characterized in that: include: Obtaining audio content correlation information and audio timing correlation information between a first user and a second user in an online conference; Dividing the first user and the second user into virtual groups according to the audio content correlation information and the audio timing correlation information; The audio of each member of the virtual group in which the first user is located is played, the first user is the terminal user, and the second user is other terminal users in the online meeting except the first user.

2. The audio processing method according to claim 1, characterized in that: Obtaining audio content correlation information between a first user and a second user in an online conference, including: Acquire a first vocabulary set in a first text, wherein the first text is a text of first audio data of the first user in the online conference, and the first vocabulary set includes: vocabulary ranked in the top Z positions in terms of appearance frequency in the first text, where Z is an integer greater than or equal to 1; Acquire a second vocabulary set in a second text, wherein the second text is a text of second audio data of the second user in the online conference, and the second vocabulary set includes: vocabulary ranked in the top Z positions in terms of frequency of occurrence in the second text; and the second vocabulary set has the same total number of elements as the first vocabulary set; Obtaining the number of words that match between the first vocabulary set and the second vocabulary set; The similarity between the first vocabulary set and the second vocabulary set is determined according to the number of words and the total number of elements, and the similarity is used as the audio content correlation information between the first user and the second user.

3. The audio processing method according to claim 1 or 2, characterized in that: Obtaining audio timing correlation information between a first user and a second user in an online conference, including: Calculating the audio connection between the first user and the second user; Audio timing correlation information between the first user and the second user is obtained according to the audio connectivity.

4. The audio processing method according to claim 3, characterized in that: The calculating the audio connection degree between the first user and the second user includes: Acquire a first time series formed by end times of each speech voice interval of the first user in the first audio data; the first audio data is the audio data of the first user in the online conference; Acquire a second time series formed by the start time of each speech voice interval of the second user in the second audio data; the second audio data is the audio data of the second user in the online conference; Acquire an audio interval value between the first user and the second user according to the first time series and the second time series; The audio connection degree between the first user and the second user is calculated according to the audio interval value.

5. The audio processing method according to claim 4, characterized in that: The step of obtaining a second time series formed by start times of each speech interval of the second user in the second audio data includes: Determine a silent interval of the first user corresponding to a speech interval of the second user; wherein the speech interval of the second user refers to a speech interval of the second user in the second audio data of the online conference, and the silent interval of the first user refers to a silent interval of the first user in the first audio data of the online conference; Determining a speech start time of the speech voice interval of the second user in the second audio data according to a time relationship between the silence interval and the speech voice interval; According to the determined speech start time, a second time series is obtained.

6. The audio processing method according to claim 5, characterized in that: The step of determining the speech start time of the speech voice interval of the second user in the second audio data according to the time relationship between the silence interval and the speech voice interval includes at least one of the following: If the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silence interval, and the interval between the actual speech start time and the start time of the silence interval is less than a first threshold, then the actual speech start time is used as the speech start time of the speech voice interval; If the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is earlier than the start time of the silence interval, the actual speech start time is used as the speech start time of the speech voice interval; If the speech voice interval overlaps with the silence interval, and the actual speech start time of the second user corresponding to the speech voice interval is later than the start time of the silence interval, and the interval between the actual speech start time and the start time of the silence interval is greater than or equal to a second threshold, then the start time of the silence interval is used as the speech start time of the speech voice interval; If the speech voice interval and the silence interval do not overlap, the start time of the silence interval is used as the speech start time of the speech voice interval.

7. An audio processing device, characterized in that: include: A first acquisition module is used to acquire audio content correlation information and audio timing correlation information between a first user and a second user in an online conference; A first processing module, configured to divide the first user and the second user into virtual groups according to the audio content correlation information and the audio timing correlation information; The first playback module is used to play the audio of each member in the virtual group where the first user is located. The first user is the terminal user, and the second user is other terminal users in the online meeting except the first user.

8. An audio processing device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that: When the processor executes the program, the audio processing method according to any one of claims 1 to 6 is implemented.

9. A readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the audio processing method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the audio processing method according to any one of claims 1 to 6.

Images