Audio data processing method and device, storage medium and computer program product

Abstract

The invention relates to an audio data processing method and device, a storage medium and a computer program product. The method comprises the steps that to-be-processed audio data in an online conference and key information of the online conference are acquired; based on the key information, determining first key content matched with the key information from the to-be-processed audio data; and adjusting an audio playing parameter of the first key content to obtain pre-processed audio data, and sending the pre-processed audio data to a terminal. The adjusted audio playing parameter can be used for highlighting and playing the first key content in the preprocessed audio data. In this way, the user can determine the key part in the conference at the first time, the situation that the user misses listening can be avoided, and then the effect of the online conference can be improved to a great extent.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to an audio data processing method, device, storage medium and computer program product. Background Art

[0002] With the continuous development of network technology, multi-person online conferences have been greatly developed. Multi-person online conferences can break through geographical restrictions and allow people in different places to participate in the conference online.

[0003] In the related art, participants often need to record the important parts of the speech according to their understanding of the speech content in the meeting. If the participants are not focused or the environment is noisy, they may miss the important content of the meeting, making the online meeting less effective. Summary of the invention

[0004] The present disclosure provides an audio data processing method, device, storage medium and computer program product.

[0005] According to a first aspect of the present disclosure, there is provided an audio data processing method, which is applied to a server, and the method comprises:

[0006] Obtaining audio data to be processed in an online conference and key information of the online conference;

[0007] Based on the key information, determining a first key content matching the key information from the audio data to be processed;

[0008] The audio playback parameters of the first key content are adjusted to obtain pre-processed audio data, and the pre-processed audio data is sent to the terminal.

[0009] According to a second aspect of the present disclosure, there is provided an audio data processing method, which is applied to a terminal, and the method includes:

[0010] Acquire the preprocessed audio data sent by the server, and acquire the first key content in the preprocessed audio data; wherein the first key content includes a target audio playback parameter, and the target audio playback parameter is used to highlight and play the first key content in the preprocessed audio data;

[0011] Play the preprocessed audio data; wherein, when playing the first key content, it is played based on the target audio playback parameters.

[0012] According to a third aspect of the present disclosure, an electronic device is provided. The electronic device includes: at least one processor;

[0013] a memory for storing the at least one processor-executable instruction;

[0014] Wherein, the at least one processor is configured to execute the instructions, and when the electronic device is a server, execute the method as described in the first aspect; when the electronic device is a terminal, execute the method as described in the second aspect.

[0015] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above method of the present disclosure is implemented.

[0016] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the above method of the present disclosure when executed by a processor.

[0017] The audio data processing method, device, storage medium and computer program product provided by the embodiments of the present disclosure obtain the audio data to be processed and the key information of the online meeting. Based on the key information, the first key content matching the key information is determined from the audio data to be processed, and the preprocessed audio data can be obtained by adjusting the audio playback parameters of the first key content, and the preprocessed audio data is sent to the terminal. Since the adjusted audio playback parameters can be used to highlight the first key content in the preprocessed audio data, the user can determine the key part of the meeting at the first time, which can avoid the user from missing the audio, and thus can greatly improve the effect of the online meeting. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Further details, features and advantages of the present disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A flowchart of an audio data processing method provided by an exemplary embodiment of the present disclosure;

[0020] Figure 2 A flowchart of an audio data processing method provided for another exemplary embodiment of the present disclosure;

[0021] Figure 3 A schematic block diagram of functional modules of an audio data processing device provided for an exemplary embodiment of the present disclosure;

[0022] Figure 4 A structural block diagram of an electronic device provided for an exemplary embodiment of the present disclosure;

[0023] Figure 5 A structural block diagram of a computer system provided for an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0025] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0026] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0027] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0028] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0029] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0030] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0031] As an optional but non-limiting implementation, in response to receiving an active request from the user, the method of sending a prompt message to the user may be, for example, a pop-up window, in which the prompt message may be presented in text. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understandable that the above notification and the process of obtaining user authorization are only illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0032] In the embodiments provided by the present disclosure, there may be multiple users participating in an online conference, and different users may participate in the online conference through corresponding terminals. During the online conference, one or more users may participate in the speech at the same time. Different user rights may be assigned to different users, such as speakers, participants in topic discussions, moderators, and conference administrators.

[0033] In the embodiment, the server may obtain the meeting materials of the meeting in advance, and process the meeting materials by natural language processing (NLP) and other methods to extract the key content of the meeting. The meeting materials may be one or more of the meeting theme name, meeting introduction, meeting theme, and the content of the speeches of each speaker. In addition, the extracted meeting key content may be key words, important data, or decision-making information, etc., and the embodiment is not limited thereto.

[0034] During the meeting, the speaker's audio data can be obtained, and semantic recognition or semantic analysis can be performed on the audio data, content can be extracted from the speaker's audio information, and the matching degree between the extracted content and the key content of the above meeting can be calculated, and the part with a matching degree greater than a preset threshold can be used as the key content that requires adjustment of audio playback parameters. For example, the audio playback parameters such as volume value, pitch or speech speed of the key content can be adjusted so that the key content can be highlighted during voice playback, so that the participants can hear the key content more clearly and attract the attention of the participants.

[0035] In the embodiment, the user's image and other information can also be collected in real time based on the terminal of the participant, and the user's current emotions, such as attention, disregard or normal state, can be analyzed. In this way, the content that the user is interested in in this meeting can be determined based on the user's emotions. In addition, the content that the user is interested in can be determined based on the user's user portrait or historical data, and the part that matches the content that the user is interested in can be determined from the speaker's content as the key content, and the above-mentioned audio playback parameter adjustment is also performed on the key content, so that when the terminal performs voice playback, the key content is played prominently.

[0036] It should be noted that, for example, in an academic conference in the computer field, there are multiple conference topics, and different users may be interested in different conference topics. Therefore, in order to distinguish the key contents that different users are interested in or concerned about in the conference, when pushing the conference video stream to each user's terminal, the video stream only includes the adjustment of the audio playback parameters of the key contents corresponding to the conference key and the key contents that the user is interested in, so as to avoid mutual influence between different users due to different key contents of interest.

[0037] In addition, in the embodiments provided by the present invention, the conference video stream pushed to each user's burden may also only include the adjustment of the audio playback parameters of the key content corresponding to the conference key points. The terminal side can obtain the user's behavior data or emotions, etc., to obtain the key content of the audio stream that the user is interested in, and adjust the audio playback parameters of the key content of the audio stream that the user is interested in. This can reduce the pressure on the server to process audio data.

[0038] In the embodiments provided by the present disclosure, if multiple users speak at the same time, the server corresponding to the conference system will receive multiple input audio streams at the same time, and the server can identify the identity or role of the speaker corresponding to the current audio stream in real time, such as the host, speaker, ordinary participant, etc. And analyze the degree of match between the current audio stream and the key content of the meeting, so as to determine the corresponding audio adjustment playback parameters, for example, the volume, pitch or speech speed in the audio playback parameters, etc., whose range values ​​can be between 0 and 1 respectively, and are different from the values ​​corresponding to the audio playback parameters of non-key content in the audio stream. By modifying the values ​​corresponding to the audio playback parameters, and merging the modified multiple audio streams into one audio stream, and sending the audio stream to each terminal.

[0039] For example, when multiple users speak at the same time, the volume of their audio streams can be adjusted appropriately according to user permissions. The volume of the user with high user permissions can be appropriately increased, and the volume of the user with the lowest user permissions can be appropriately reduced, so as to highlight the speech content of key users such as high users.

[0040] On the terminal side of the participant, the audio stream sent by the server can be received, and the audio data after the audio playback parameters have been adjusted in the audio stream can be played. Since the audio playback parameters of the key content in the audio stream are different from the audio playback parameters of the non-key content, and the audio playback parameters of the key content can be played in a breakthrough manner when playing the audio stream, when the terminal plays the audio stream sent by the server, it can play the key content in the audio stream in a breakthrough manner, which can attract the user's attention. By increasing the volume, raising the pitch or lowering the speaking speed, the user can hear the key content played in the audio stream more clearly, which is convenient for the user to record the key content and attract attention.

[0041] It should be noted that since many previous meetings involve terminals obtaining audio streams pushed by the server in real time, if the playback of key content is emphasized by reducing the playback speed, etc., it will cause increased latency for users attending the meeting. The implementation example can increase the playback speed of non-key content in the audio stream to offset the increased latency caused by reducing the playback speed of key content, thereby improving the real-time nature of users attending the meeting.

[0042] In the embodiments provided by the present disclosure, if the server only adjusts the audio playback parameters for the key content corresponding to the key content of the conference, the terminal side can extract the key content of interest to the user from the received audio stream by acquiring the user's behavior data or emotions and other features, and adjust the audio playback parameters for the key content of interest to the user, so that when the audio stream is played, the key content of interest to the user can be played in a sudden manner, and the key content related to the key content of the conference can also be played in a sudden manner. Among them, the user's behavior data can include the user's historical data or real-time interaction data, etc., and the key content of interest to the user can also be determined based on the user's user portrait, etc., and the embodiments are not limited to this.

[0043] In this embodiment, based on the user's historical behavior data and real-time interaction, each user's key focus content and points of interest are analyzed. The audio stream is filtered and the parts that highly match the user's focus are marked as key processing objects. By adjusting the parameters of the content that the user focuses on, such as increasing the volume, changing the tone or adjusting the speech speed, it is ensured that the user can clearly hear the key content and the content that the user is interested in.

[0044] In the embodiments provided by the present disclosure, firstly, a single-channel audio stream can be processed and highlighted, then multiple single-channel audio streams can be analyzed and processed, merged into one audio stream, and then personalized processing can be performed on the distribution user based on the one audio stream, ultimately achieving the purpose of highlighting key content and user interests.

[0045] In the embodiments provided by the present disclosure, since the conference is conducted in real time, the audio streams collected by the device from each speaking user to the audio streams processed and distributed to each participating user have high real-time requirements. To ensure the real-time performance of the data, the strategies adopted by the present disclosure include the following:

[0046] 1) The embodiment can use algorithms such as NLP and machine learning to ensure rapid analysis and processing of meeting content. It also uses parallel processing technology and efficient data processing frameworks such as TensorFlow or PyTorch to improve computing speed and response time.

[0047] 2) The embodiment adopts a low-latency audio transmission protocol, such as Web Real-Time Communications (WebRTC), to ensure that the delay of audio data in network transmission is minimized. In addition, distributed computing and edge computing technologies can be used to distribute processing tasks to the server closest to the user, which can reduce network transmission time and thus achieve low-latency data transmission.

[0048] 3) Realize real-time data stream processing by building a real-time streaming processing architecture, such as using technologies such as Apache Kafka or Apache Flink. Through the streaming processing framework, audio data is received and processed in real time to ensure the real-time adjustment of audio parameters.

[0049] 4) During the audio streaming process, using a cache mechanism to store audio data for a short period of time for preprocessing and analysis can reduce the pressure of real-time processing. And using a prediction strategy to pre-analyze and pre-adjust the upcoming audio content to improve the response speed.

[0050] 5) By deploying high-performance servers and network equipment, the system can maintain efficient operation when processing high concurrency and large amounts of data. And by using hardware acceleration technologies such as graphics processing unit (GPU) acceleration and field programmable gate array (FPGA), the speed of audio processing and analysis can be improved.

[0051] 6) By adopting a microservice architecture, modularizing audio analysis, processing, and transmission functions, and independently deploying and scaling them, the maintainability and responsiveness of the system can be improved. Using asynchronous processing and non-blocking input / output (I / O) technology can reduce system waiting time and resource usage.

[0052] Based on the above embodiments, the present disclosure also provides an audio data processing method, such as Figure 1As shown, the method can be applied to a server, and the method may include the following steps:

[0053] In step S110, the audio data to be processed in the online conference and the key information of the online conference are obtained.

[0054] In the embodiment, multiple users can participate in the online conference through multiple corresponding terminals respectively. When speaking, the speaking user can send audio data to the server through the terminal. The server processes the received audio data and forwards the processed audio data to each user terminal participating in the online conference.

[0055] In an embodiment, before the meeting begins, the server can obtain relevant information data of the meeting, such as the meeting theme, meeting introduction, information of participants, and speeches of participants, and then determine the key information of the online meeting from these relevant information data. For example, keywords can be extracted based on the theme of the meeting and the speeches, and the extracted keywords are used as the key information of the meeting.

[0056] In an embodiment, during a conference, the server receives audio data sent by each user terminal, which may be audio content sent by each user through the terminal when speaking. In an embodiment, the audio data generated by these users when speaking can be used as the audio data to be processed, so that the server can process the audio data to be processed.

[0057] In step S120, based on the key information, a first key content matching the key information is determined from the audio data to be processed.

[0058] In an embodiment, through the key information obtained above, such as keywords related to the conference theme, the audio data to be processed can be matched with these keywords. For example, the similarity between the audio information to be processed and the key information can be calculated, and the part of the audio data to be processed with a similarity greater than a preset threshold is used as the first key content. The first key content belongs to the content in the audio to be processed that is relatively relevant to the conference theme, etc., and can reflect the purpose of this meeting. Therefore, the first key content can be processed to facilitate the user to focus on the content of the first terminal and allow the user to hear the first key content more clearly.

[0059] In step S130, the audio playback parameters of the first key content are adjusted to obtain pre-processed audio data, and the pre-processed audio data is sent to the terminal.

[0060] In the embodiment, since the first key content belongs to the content that the speaking user needs to emphasize, it is closely related to the theme of this meeting, etc., and the user needs to focus on it. Therefore, the audio playback parameters of the first key content can be adjusted to achieve the processing of the audio data to be processed. Specifically, the audio playback parameters of the first key content in the audio data to be processed can be adjusted, such as increasing the volume of the first key content, raising the pitch, and lowering the speech speed, etc. One or several combinations. In this way, the pre-processed audio data obtained after the audio playback parameter adjustment is sent to the terminal, and the adjusted audio playback parameters are used to highlight the first key content in the pre-processed audio data, so that when the terminal plays the received pre-processed audio data, the first key content in the pre-processed audio data can be highlighted.

[0061] When other users listen to the speaker's speech, they can not only know the key points expressed by the speaker in the first time, but also hear clearly the meaning that the speaker wants to express.

[0062] The audio data processing method provided by the embodiment of the present disclosure obtains the audio data to be processed and the key information of the online conference. Based on the key information, the first key content matching the key information is determined from the audio data to be processed, and the preprocessed audio data can be obtained by adjusting the audio playback parameters of the first key content, and the preprocessed audio data is sent to the terminal. Since the adjusted audio playback parameters can be used to highlight the first key content in the preprocessed audio data, the user can determine the key part of the meeting at the first time, which can avoid the user from missing the key information, and thus can greatly improve the effect of the online meeting.

[0063] Based on the above embodiment, in another embodiment provided by the present disclosure, if multiple users speak at the same time, the server will receive multiple channels of audio data. Therefore, the above step S130 may further include the following steps:

[0064] In step S131, when multiple channels of pre-processed audio data are obtained, the priorities corresponding to the respective channels of pre-processed audio data are obtained.

[0065] In the embodiment, when multiple users speak at the same time, the multiple users will send audio streams to the server through corresponding terminals respectively, so that the server will receive multiple audio streams in the same time period. The server can process each audio stream in the above manner to adjust the audio playback parameters of the first key content in each audio stream, so that multiple pre-processed audio data can be obtained.

[0066] Since the priorities of users speaking at the same time may be different, for example, the priority of the host is higher than that of the speaking user, and the priority of the administrator is higher than that of the host. The speaking users can also be divided into primary speaking users and secondary speaking users, and the priority of the primary speaking users is higher than that of the secondary speaking users.

[0067] Therefore, when multiple users speak at the same time, for example, there is a question-and-answer situation or a debate, etc., which makes it possible that other users cannot hear the main content of the meeting clearly in the first time. The embodiment can increase the volume of the preprocessed audio data of the high priority channel in each channel of preprocessed audio data when it is played, or reduce the volume of the preprocessed audio data of the low priority channel in each channel of preprocessed audio data when it is played, based on the priority. Of course, it is also possible to increase the volume of the preprocessed audio data of the high priority channel in each channel of preprocessed audio data when it is played, while reducing the volume of the preprocessed audio data of the low priority channel in each channel of preprocessed audio data when it is played. Among them, the above-mentioned audio playback parameters may include at least one of speech speed, volume and pitch.

[0068] Specifically, the implementation method can adjust the audio playback parameters of each channel of pre-processed audio data according to the priority level corresponding to each speaking user, such as increasing the volume of high-priority speaking users as a whole, or reducing the volume of low-priority speaking users as a whole, playing the low-priority speaking users as background music, etc., so that other users can clearly hear the speech content of the high-priority speaking users at the first time.

[0069] In step S132, the audio playback parameters of each channel of pre-processed audio data are adjusted based on the priority, and the pre-processed audio data of each channel after the audio playback parameters are adjusted are combined into one channel of pre-processed audio data, and the pre-processed audio data is sent to each terminal.

[0070] In the embodiment, the audio playback parameters of each channel of audio data can be adjusted by priority, and the pre-processed audio data of each channel after the audio playback parameters are adjusted can be combined into one channel of pre-processed audio data, so that the combined pre-processed audio data can be sent to each terminal. In this way, in the audio content heard by the users of each terminal, not only can the speech content of users with different priorities be distinguished in the first time, but also the key content of the speech content of each speaking user can be determined in the first time, so that the participating users can know the key content of the meeting in time, thereby greatly improving the efficiency of participating in the meeting.

[0071] Based on the above embodiment, in another embodiment provided by the present disclosure, a method for processing audio data is further provided, which can be applied to a terminal, such as Figure 2 As shown, the method may include the following steps:

[0072] In step S210, the pre-processed audio data sent by the server is acquired, and the first key content in the pre-processed audio data is acquired.

[0073] The first key content includes a target audio playback parameter, and the target audio playback parameter is used to highlight and play the first key content in the pre-processed audio data.

[0074] In step S220, the preprocessed audio data is played, wherein the first focused content is played based on the target audio playback parameters.

[0075] In an embodiment, the audio playback parameters of the first key content in the preprocessed audio data are pre-adjusted by the server so that the first key content includes target audio playback parameters, which are different from the audio playback parameters of non-key content in the preprocessed audio data. For example, the target audio playback parameters have a louder volume, a slower speaking speed, or a louder tone, etc. In this way, in the process of playing the preprocessed audio data, when playing the first key content, the presence of the first key content in the entire preprocessed audio data can be more prominent, allowing the user to perceive it at the first time.

[0076] Based on the above embodiment, in the audio playback method provided by the embodiment of the present disclosure, the above step S220 may further include the following steps:

[0077] In step S221, the behavior data of the target user is obtained, and the interest characteristics of the target user are obtained based on the behavior data.

[0078] In an embodiment, the behavior data may include the user's historical behavior data or real-time interactive behavior data, or may include both the user's historical behavior data and real-time interactive behavior data, but the embodiment is not limited thereto. In addition, the behavior data may also include the user's current emotional characteristics or expressions, so that the user's interest characteristics can be obtained based on the user's behavior data.

[0079] In step S222, the second key content that the target user is interested in is determined from the pre-processed audio data based on the interest feature.

[0080] In an embodiment, the content of interest in the preprocessed audio data can be determined based on the user's interest characteristics, and the content of interest can be used as the second key content. For example, the user's emotions or facial features such as attention, disregard or normal state can be obtained, so that the content of interest to the user in this meeting can be determined based on the user's emotions. In addition, the user's content of interest can be determined based on the user's user portrait or historical data, and the part matching the content of interest to the user can be determined from the speaker's content as the key content. The above-mentioned audio playback parameter adjustment is also performed on the key content, so that when the terminal performs voice playback, the key content can be highlighted and played.

[0081] In step S223, the audio playback parameters when playing the second key content are adjusted.

[0082] The adjusted audio playback parameters are used to highlight the second key content in the pre-processed audio data.

[0083] In an embodiment, the audio playback parameters corresponding to the first key content, such as volume, pitch, and speech speed, can be adjusted in the same manner to facilitate the prominent playback of the second key content in the preprocessed audio data to attract the user's attention.

[0084] In step S224, the adjusted pre-processed audio data is played, wherein the second key content is played based on the adjusted audio playback parameters.

[0085] In the embodiment, when the pre-processed audio data received by the terminal is played, the non-key part of the pre-processed audio data can be played according to the default or user-set audio playback parameters, and when the second key content is played, it will be played according to the adjusted audio playback parameters. At the same time, in the process of playing the pre-processed audio data, when the first key content is played, it will also be played according to the adjusted target audio playback parameters corresponding to the first key content, so that both the first terminal content and the second terminal content can be played as the focus when playing the pre-processed audio data, which can not only highlight the key content of the meeting, but also highlight the content that the user is interested in or concerned about, thereby greatly improving the efficiency of the online meeting.

[0086] In the case of dividing each functional module according to each function, an embodiment of the present disclosure provides an audio data processing device, which may be a server. Figure 3 The following is a schematic block diagram of the functional modules of an audio data processing device provided by an exemplary embodiment of the present disclosure. Figure 3 As shown, the audio data processing device comprises:

[0087] The key information acquisition module 10 is used to acquire the audio data to be processed in the online conference and the key information of the online conference;

[0088] A first key content acquisition module 20 is used to determine, based on the key information, first key content matching the key information from the audio data to be processed;

[0089] An audio parameter adjustment module 30, used to adjust the audio playback parameters of the first key content to obtain pre-processed audio data;

[0090] The data sending module 40 is used to send the pre-processed audio data to the terminal.

[0091] In another embodiment provided by the present disclosure, the data sending module is specifically used to:

[0092] When obtaining multiple channels of pre-processed audio data, obtaining the priority corresponding to each channel of pre-processed audio data;

[0093] The audio playback parameters of the pre-processed audio data are adjusted based on the priority, the pre-processed audio data after the audio playback parameters are adjusted are combined into one pre-processed audio data, and the pre-processed audio data is sent to each terminal.

[0094] In another embodiment provided by the present disclosure, the audio playback parameter includes a volume; the audio parameter adjustment module is specifically used to:

[0095] Based on the priority, increasing the volume of the pre-processed audio data of the high priority path among the pre-processed audio data paths during playback;

[0096] The volume of the pre-processed audio data of the low priority channel among the pre-processed audio data channels is reduced during playback.

[0097] In another embodiment provided by the present disclosure, the audio playback parameter includes at least one of speech speed, volume and pitch.

[0098] In the case of dividing each functional module according to each function, the embodiment of the present disclosure provides an audio data processing device, which can be a terminal. The audio data processing device includes:

[0099] An audio stream data acquisition module, used to acquire pre-processed audio data and acquire a first key content in the pre-processed audio data; wherein the first key content includes a target audio playback parameter, and the target audio playback parameter is used to highlight and play the first key content in the pre-processed audio data;

[0100] An audio playback module is used to play the preprocessed audio data; wherein, when playing the first key content, it is played based on the target audio playback parameters.

[0101] In another embodiment provided by the present disclosure, the audio playback module is specifically used to:

[0102] Acquire behavior data of a target user, and acquire interest characteristics of the target user based on the behavior data;

[0103] Determining second key content that the target user is interested in from the preprocessed audio data based on the interest feature;

[0104] Adjusting the audio playback parameters when playing the second key content; wherein the adjusted audio playback parameters are used to highlight the second key content in the pre-processed audio data;

[0105] Play the adjusted preprocessed audio data; wherein, when playing the second key content, it is played based on the adjusted audio playback parameters.

[0106] In another embodiment provided by the present disclosure, the behavior data includes: historical behavior data and / or real-time interactive behavior data.

[0107] For the device part, please refer to the description of the corresponding embodiment of the method, which will not be repeated here.

[0108] The audio data processing device provided by the embodiment of the present disclosure obtains the audio data to be processed and the key information of the online conference. Based on the key information, the first key content matching the key information is determined from the audio data to be processed, and the pre-processed audio data can be obtained by adjusting the audio playback parameters of the first key content. Since the adjusted audio playback parameters can be used to highlight the first key content in the pre-processed audio data, the user can determine the key part of the meeting at the first time, which can avoid the user from missing the key part, and thus can greatly improve the effect of the online meeting.

[0109] An embodiment of the present disclosure also provides an electronic device, comprising: at least one processor; and a memory for storing instructions executable by the at least one processor; wherein the at least one processor is configured to execute the instructions to implement the above method disclosed in the embodiment of the present disclosure.

[0110] Figure 4 The structure diagram of an electronic device provided by an exemplary embodiment of the present disclosure is shown in FIG. Figure 4As shown, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801 , and the processor 1801 can execute corresponding steps in the above method disclosed in the embodiment of the present disclosure.

[0111] The processor 1801 may also be referred to as a central processing unit (CPU), which may be an integrated circuit chip having signal processing capabilities. Each step in the method disclosed in the embodiment of the present disclosure may be completed by an integrated logic circuit of hardware in the processor 1801 or by instructions in the form of software. The processor 1801 may be a general-purpose processor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present disclosure may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a memory 1802, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, or other mature storage media in the art. The processor 1801 reads the information in the memory 1802 and completes the steps of the method in combination with its hardware.

[0112] In addition, when various operations / processes according to the present disclosure are implemented by software and / or firmware, they can be transmitted from a storage medium or a network to a computer system having a dedicated hardware structure, such as Figure 5 The computer system 1900 shown is installed with the programs constituting the software. When the various programs are installed, the computer system can perform various functions, including the functions described above. Figure 5 A structural block diagram of a computer system provided for an exemplary embodiment of the present disclosure.

[0113] Computer system 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0114] like Figure 5 As shown, the computer system 1900 includes a computing unit 1901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 to a random access memory (RAM) 1903. In the RAM 1903, various programs and data required for the operation of the computer system 1900 can also be stored. The computing unit 1901, the ROM 1902, and the RAM 1903 are connected to each other via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0115] A plurality of components in the computer system 1900 are connected to the I / O interface 1905, including: an input unit 1906, an output unit 1907, a storage unit 1908, and a communication unit 1909. The input unit 1906 may be any type of device capable of inputting information to the computer system 1900, and the input unit 1906 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 1907 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1908 may include, but is not limited to, a disk, an optical disk. The communication unit 1909 allows the computer system 1900 to exchange information / data with other devices over a network such as the Internet, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0116] The computing unit 1901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the above methods disclosed in the embodiments of the present disclosure may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1908. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 1902 and / or communication unit 1909. In some embodiments, the computing unit 1901 may be configured to perform the above methods disclosed in the embodiments of the present disclosure in any other appropriate manner (e.g., by means of firmware).

[0117] The embodiment of the present disclosure further provides a computer-readable storage medium, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above method disclosed in the embodiment of the present disclosure.

[0118] The computer-readable storage medium in the disclosed embodiments may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. The computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specifically, the computer-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0119] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0120] The embodiments of the present disclosure further provide a computer program product, including a computer program, wherein when the computer program is executed by a processor, the above method disclosed in the embodiments of the present disclosure is implemented.

[0121] In embodiments of the present disclosure, computer program codes for performing the operations of the present disclosure may be written in one or more programming languages ​​or combinations thereof, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer.

[0122] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0123] The modules, components or units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the names of the modules, components or units do not, in some cases, limit the modules, components or units themselves.

[0124] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0125] The above descriptions are only some embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, a technical solution formed by replacing the above features with the technical features with similar functions disclosed in the present disclosure (but not limited to).

[0126] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. An audio data processing method, characterized in that: Applied to a server, the method comprises: Obtaining audio data to be processed in an online conference and key information of the online conference; Based on the key information, determining a first key content matching the key information from the audio data to be processed; The audio playback parameters of the first key content are adjusted to obtain pre-processed audio data, and the pre-processed audio data is sent to the terminal.

2. The method according to claim 1, characterized in that The sending the pre-processed audio data to the terminal includes: When obtaining multiple channels of pre-processed audio data, obtaining the priority corresponding to each channel of pre-processed audio data; The audio playback parameters of the pre-processed audio data are adjusted based on the priority, the pre-processed audio data after the audio playback parameters are adjusted are combined into one pre-processed audio data, and the pre-processed audio data is sent to each terminal.

3. The method according to claim 2, characterized in that The audio playback parameters include volume; and the step of adjusting the audio playback parameters of each channel of pre-processed audio data based on the priority includes: Based on the priority, increasing the volume of the pre-processed audio data of the high priority path among the pre-processed audio data paths during playback; And / or, reducing the volume of the pre-processed audio data of the low priority channel among the pre-processed audio data channels when playing.

4. The method according to claim 1, characterized in that The audio playback parameters include: at least one of speech speed, volume and pitch.

5. An audio data processing method, characterized in that: Applied to a terminal, the method comprises: Acquire the preprocessed audio data sent by the server, and acquire the first key content in the preprocessed audio data; wherein the first key content includes a target audio playback parameter, and the target audio playback parameter is used to highlight and play the first key content in the preprocessed audio data; Play the preprocessed audio data; wherein, when playing the first key content, it is played based on the target audio playback parameters.

6. The method according to claim 5, characterized in that The playing of the pre-processed audio data comprises: Acquire behavior data of a target user, and acquire interest characteristics of the target user based on the behavior data; Determining second key content that the target user is interested in from the preprocessed audio data based on the interest feature; Adjusting the audio playback parameters when playing the second key content; wherein the adjusted audio playback parameters are used to highlight the second key content in the pre-processed audio data; Play the adjusted preprocessed audio data; wherein, when playing the second key content, it is played based on the adjusted audio playback parameters.

7. The method according to claim 6, characterized in that The behavior data includes: historical behavior data and / or real-time interactive behavior data.

8. An electronic device, characterized in that: include: at least one processor; a memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions, and when the electronic device is a server, execute the method as claimed in any one of claims 1 to 4; when the electronic device is a terminal, execute the method as claimed in any one of claims 5 to 7.

9. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method as claimed in any one of claims 1 to 7.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images