Conference audio processing method and system based on voice intelligence

Abstract

The present invention relates to a conference audio processing method and system based on voice intelligence, wherein the method is applied to a source end, the source end characterizing the terminal device where the speaker in the audio and video conference is located, and the method comprises: in the audio and video conference, obtaining the first network transmission quality of the source end itself and the second network transmission quality of each receiving end; determining the transmission mode for each receiving end based on the first network transmission quality, each second network transmission quality and the conference type of the audio and video conference; obtaining the audio data of the speaker, and determining the target data corresponding to each receiving end based on the audio data and the transmission mode of each receiving end, and for each receiving end, the target data characterizes the data obtained after the audio data is processed by the transmission mode of the receiving end; and sending the target data corresponding to each receiving terminal to the corresponding receiving end. The present application has the effect of improving the accuracy of audio data transmission in audio and video conferences under network disturbance conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of audio and video conference calls, and in particular to a conference audio processing method and system based on voice intelligence. Background Art

[0002] With the rapid development of information technology, audio and video conferencing systems have become an important tool for remote communication and exchange in modern society. They are widely used in scenarios such as corporate meetings, online education, and telemedicine, greatly improving communication efficiency and the breadth of information dissemination. However, fluctuations in network transmission quality often have a significant impact on the effectiveness of audio and video conferencing.

[0003] In existing audio and video conferencing systems, when network transmission quality deteriorates, such as network jitter, insufficient bandwidth or packet loss, it often leads to loss of synchronization between audio and video, voice distortion, unclear sound and distorted tone. This not only reduces the communication experience of participants, but also may lead to the inability to accurately transmit key information in the meeting, thus affecting meeting decision-making and subsequent work. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a conference audio processing method and system based on voice intelligence, aiming to solve at least one of the above-mentioned technical problems.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] In the first aspect, the present application provides a conference audio processing method based on voice intelligence, which is applied to the source end and adopts the following technical solutions:

[0007] A conference audio processing method based on voice intelligence is applied to a source end, where the source end represents a terminal device where a speaker in an audio or video conference is located. The method comprises:

[0008] In an audio and video conference, obtaining the first network transmission quality of the source end itself and the second network transmission quality of each receiving end;

[0009] Determine a transmission mode for each of the receiving ends based on the first network transmission quality, each of the second network transmission qualities, and the conference type of the audio and video conference, wherein for each receiving end, the transmission mode represents a transmission method of data between the source end and the receiving end;

[0010] Acquire audio data of the speaker, and determine target data corresponding to each of the receiving ends based on the audio data and the transmission mode of each of the receiving ends, wherein for each receiving end, the target data represents data obtained after the audio data is processed by the transmission mode of the receiving end;

[0011] The target data corresponding to each of the receiving terminals is sent to the corresponding receiving end.

[0012] The beneficial effects of the present invention are as follows: by acquiring the network transmission quality of the source end and each receiving end in real time, and determining the transmission mode based on the information of the first network transmission quality of the source end, the second network transmission quality of each receiving end, and the conference type of the audio and video conference. In this way, the data transmission mode can be adjusted according to the network conditions, and it can also adapt to different conference types. By processing the audio data of the speaker and determining the target data based on the transmission mode of the receiving end, the intelligent processing of the audio data is realized, ensuring that the audio data can maintain a high quality during the transmission process, so that in the audio and video conference, each receiving end can obtain clear and smooth audio data, improve the accuracy of audio data transmission in the audio and video conference under the condition of network disturbance, and reduce communication barriers caused by network disturbance.

[0013] Based on the above technical solution, the present invention can also be improved as follows.

[0014] Further, the transmission mode is any one of an audio transmission mode and a text transmission mode, and based on the first network transmission quality, each of the second network transmission qualities and the conference type of the audio and video conference, determining the transmission mode for each of the receiving ends includes:

[0015] Based on the conference type of the audio and video conference, determine the network quality requirements corresponding to the conference type;

[0016] When the first network transmission quality does not meet the network quality requirement, determining that the transmission mode of the source end to each of the receiving ends is a text transmission mode;

[0017] When the first network transmission quality meets the network quality requirement, and the second network transmission quality of the receiving end does not meet the network quality requirement, determining that the transmission mode between the source end and the receiving end that does not meet the network quality requirement is a text transmission mode;

[0018] When the first network transmission quality meets the network quality requirement and the second network transmission quality of the receiving end meets the preset network quality requirement, it is determined that the transmission mode of the source end to the receiving end that meets the preset network quality requirement is an audio transmission mode.

[0019] The beneficial effect of adopting the above further scheme is: by flexibly adjusting the transmission mode according to the network conditions of the source and receiving ends and the type of meeting, the adaptability and flexibility of the meeting are enhanced. Different types of meetings may have different requirements for audio quality, and this dynamic adjustment can meet the needs of different meeting scenarios. Adjusting the transmission mode according to network quality can optimize the utilization of network resources. When the network quality is good, using the audio transmission mode can provide a better meeting experience; when the network quality is poor, switching to the text transmission mode ensures that the information can be accurately transmitted, avoiding audio quality degradation or interruption due to network delays, packet loss and other problems, while reducing network bandwidth usage and freeing up network resources for other applications or services.

[0020] Further, the determining the target data corresponding to each of the receiving ends based on the audio data and the transmission mode of each of the receiving ends includes:

[0021] When the transmission mode is a text transmission mode, the audio data is converted into text data based on a preset voice intelligent recognition technology;

[0022] Verifying the text data, generating a verification code corresponding to the text data, encapsulating the verification code and the text data, and using the encapsulated verification code and text data as target data corresponding to the receiving end;

[0023] When the transmission mode is an audio transmission mode, the audio data is used as target data corresponding to the receiving end.

[0024] The beneficial effect of adopting the above further scheme is that when the transmission mode is the text transmission mode, the audio data is converted into text data through the voice intelligent recognition technology, which can reduce the audio distortion or interruption problems that may occur in network transmission and improve the accuracy and reliability of information transmission. Verifying the text data and generating a check code further ensures the integrity and accuracy of the data and reduces the errors or losses that may occur in the data transmission process. Since the text data and the check code are transmitted, the data volume is smaller than that of the audio data, which can significantly reduce the bandwidth occupation and improve the transmission quality of the target data.

[0025] When the transmission mode is audio transmission mode, it means that the network conditions are good. Selecting the audio transmission mode can provide a better conference experience.

[0026] Further, when the network transmission quality of the source end does not meet the preset network quality requirement, after determining that the transmission mode of the source end to each of the receiving ends is a text transmission mode, the method further includes:

[0027] Close the audio transmission path of the source end;

[0028] When the first network transmission quality meets the network quality requirement and the second network transmission quality of the receiving end does not meet the network quality requirement, after determining that the transmission mode between the source end and the receiving end that does not meet the network quality requirement is a text transmission mode, the method further includes:

[0029] The audio transmission path of the receiving end that does not meet the preset network quality requirements is marked as unavailable.

[0030] The beneficial effect of adopting the above further scheme is: closing the audio transmission path of the source end or marking the audio transmission path of the receiving end that does not meet the conditions as unavailable, the transmission of audio data can be stopped immediately, thereby significantly reducing the network bandwidth usage. If the network conditions are not sufficient to support audio transmission, continuing to attempt audio transmission will result in a waste of resources, such as excessive use of CPU, memory and network bandwidth. By closing or marking the unavailable state, these unnecessary waste of resources can be avoided. In the case of poor network conditions, attempts to transmit audio may result in reduced sound quality, freezes, heavy sounds, and sound confusion, affecting the user experience.

[0031] Further, sending the target data corresponding to each of the receiving terminals to the corresponding receiving end includes:

[0032] The target data corresponding to each of the receiving terminals is compressed based on a preset compression algorithm, and each compressed target data is sent to the corresponding receiving terminal.

[0033] The beneficial effect of adopting the above further solution is that compressing the target data can significantly reduce its volume, reduce the network bandwidth required during transmission, and thus speed up data transmission. For audio and video conferences with high real-time requirements, this can ensure the timely transmission of information and improve meeting efficiency.

[0034] Further, after sending the target data corresponding to each of the receiving terminals to the corresponding receiving end, the method further includes:

[0035] Receive the playback completion status identifier sent by each of the receiving ends, and display each of the playback completion status identifiers.

[0036] The beneficial effect of adopting the above further solution is that by displaying the playback completion status, the speakers of the meeting can instantly understand which receiving ends have successfully received the data and which receiving ends may have problems. It is convenient to quickly identify and solve potential transmission problems, ensure that the meeting information can be accurately and timely conveyed to all participants, thereby improving the communication efficiency of the meeting.

[0037] In the second aspect, the present application provides a conference audio processing method based on voice intelligence, which is applied to the receiving end and adopts the following technical solution:

[0038] A conference audio processing method based on voice intelligence is applied to a receiving end, wherein the receiving end represents a terminal device that receives audio data of a terminal device where a speaker is located in an audio and video conference, and the method comprises:

[0039] In an audio and video conference, the target data sent by the source end based on the audio data and the transmission mode is obtained. The source end represents the terminal device where the speaker in the audio and video conference is located. The transmission mode is the transmission method of data between the source end and the receiving end. The target data represents the data obtained after the audio data is processed by the transmission method of the receiving end.

[0040] Preprocessing the target data to obtain processed speech data;

[0041] The voice data is played, and after the voice data is played, a play completion status identifier is sent to the source end.

[0042] The beneficial effects of the present invention are as follows: by preprocessing the target data, the target data is converted into voice data, and after the voice data is played, the receiving end sends the playback completion status to the source end. The timely feedback mechanism facilitates the source end to understand the data verification and playback status of the receiving end, so as to make corresponding adjustments or notify other participants as needed. This interactivity enhances the real-time and interactivity of the conference.

[0043] Further, the target data is preprocessed to obtain processed speech data, including:

[0044] When the target data is a packaged verification code and text data, verifying the text data based on a preset verification rule and the verification code;

[0045] If the text data passes the verification, the text data is converted into voice data based on a preset conversion rule;

[0046] The converting of the text data into voice data based on a preset conversion rule includes:

[0047] Acquire identification information and audio feature information of the source end, wherein the audio feature information represents the voice characteristics of the speaker of the source end in the audio and video conference;

[0048] Based on the identification information, determining a speech feature value corresponding to the source end in the feature value library;

[0049] Comparing the audio feature information with the speech feature value;

[0050] If the audio feature information is compared with the speech feature value, the speech feature value is used as an input parameter of a preset text conversion model, and the text data is converted into speech data through speech synthesis technology.

[0051] The beneficial effect of adopting the above further scheme is: by receiving the encapsulated check code and text data and performing verification at the receiving end, the accuracy and integrity of the received text data are ensured. This verification mechanism helps to reduce errors and packet loss during data transmission and improve the communication quality of the entire meeting. After receiving the correct text data, it is converted into voice data for playback through preset conversion rules, which can ensure that even if the network conditions are poor and the audio transmission quality is reduced, the receiving end user can still clearly hear the speaker's content. This processing method avoids the degradation or interruption of sound quality caused by network problems, thereby optimizing the user's conference experience.

[0052] By comparing the audio feature information of the source end with the voice feature values ​​stored in the feature value library, the voice characteristics of the speaker corresponding to the source end are determined. In this way, when converting text data into voice data for playback, the voice characteristics of the speaker can be simulated to achieve personalized voice playback. Using the voice feature values ​​that match the voice characteristics of the speaker as the input parameters of the text conversion model, the text data is converted into voice data for playback through speech synthesis technology to ensure that the generated voice data is consistent with the speaker's original voice in terms of timbre, intonation, and speaking speed. The accuracy and naturalness of voice playback are improved, so that the receiving end user can better understand and accept the speaker's content. In the case of poor network conditions, by receiving and playing the converted voice data instead of the original audio data, the dependence on network bandwidth can be reduced.

[0053] In the third aspect, the present application provides a conference audio processing system based on voice intelligence, which is applied to the source end and adopts the following technical solutions:

[0054] A conference audio processing system based on voice intelligence is applied to a source end, where the source end represents a terminal device where a speaker in an audio or video conference is located. The system includes:

[0055] A network quality identification module is used to obtain the first network transmission quality of the source end itself and the second network transmission quality of each receiving end in an audio and video conference;

[0056] A mode management module, configured to determine a transmission mode for each of the receiving ends based on the first network transmission quality, each of the second network transmission qualities, and the conference type of the audio and video conference, wherein for each receiving end, the transmission mode represents a transmission method of data between the source end and the receiving end;

[0057] A first data receiving and processing module, configured to obtain audio data of a speaker, and determine target data corresponding to each receiving end based on the audio data and the transmission mode of each receiving end, wherein for each receiving end, the target data represents data obtained after the audio data is processed by the transmission mode of the receiving end;

[0058] The transmission module is used to send the target data corresponding to each of the receiving terminals to the corresponding receiving end.

[0059] The beneficial effects of the present invention are: by acquiring the network transmission quality of the source end and each receiving end in real time, and determining the transmission mode based on the information of its own network transmission quality, the network transmission quality of each receiving end and the conference type of the audio and video conference. In this way, the data transmission mode can be adjusted according to the network conditions, and it can also adapt to different conference types. By processing the audio data of the speaker and determining the target data based on the transmission mode of the receiving end, the intelligent processing of the audio data is realized, ensuring that the audio data can maintain a high quality during the transmission process, so that in the audio and video conference, each receiving end can obtain clear and smooth audio data, improve the accuracy of audio data transmission in the audio and video conference under the condition of network disturbance, and reduce the communication barriers caused by network disturbance.

[0060] In a fourth aspect, the present application provides a conference audio processing system based on voice intelligence, which is applied to a receiving end and adopts the following technical solutions:

[0061] The second data receiving and processing module is used to obtain target data sent by the source end based on audio data and a transmission mode in an audio and video conference, wherein the source end represents a terminal device where a speaker in the audio and video conference is located, the transmission mode is a transmission mode of data between the source end and the receiving end, and the target data represents data obtained after the audio data is processed by the transmission mode of the receiving end;

[0062] A preprocessing module, used for preprocessing the target data to obtain processed speech data;

[0063] The feedback module is used to play the voice data and send a play completion status mark to the source end after the voice data is played.

[0064] The beneficial effects of the present invention are as follows: by preprocessing the target data, the target data is converted into voice data, and after the voice data is played, the receiving end sends the playback completion status to the source end. The timely feedback mechanism facilitates the source end to understand the data verification and playback status of the receiving end, so as to make corresponding adjustments or notify other participants as needed. This interactivity enhances the real-time and interactivity of the conference. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A flowchart of a conference audio processing method based on voice intelligence applied to a source end according to an embodiment of the present invention;

[0066] Figure 2 A flowchart of a conference audio processing method based on voice intelligence applied to a receiving end according to an embodiment of the present invention;

[0067] Figure 3 This is a structural block diagram of a conference audio processing system based on voice intelligence applied to a source end according to an embodiment of the present invention;

[0068] Figure 4 The present invention is a structural block diagram of a conference audio processing system based on voice intelligence applied to a receiving end according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The present application is further described in detail below in conjunction with the accompanying drawings.

[0070] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0071] An embodiment of the present application provides a conference audio processing method based on voice intelligence, which can be executed by an electronic device, which can be a mobile terminal device. The mobile terminal device can be a laptop computer, a desktop computer, a mobile phone, etc., but is not limited to this.

[0072] Figure 1 The present invention is a flowchart of a conference audio processing method based on voice intelligence applied to a source end according to an embodiment of the present invention.

[0073] The source represents the terminal device where the speaker in the audio and video conference is located, such as Figure 1 As shown, the main process of a conference audio processing method based on voice intelligence applied to the source end includes:

[0074] Step S11, in an audio and video conference, obtaining a first network transmission quality of the source end itself and a second network transmission quality of each receiving end, wherein the receiving end represents a terminal device that receives the audio of the source end in the audio and video conference;

[0075] In the implementation manner of the present application, network transmission quality may include key indicators such as network delay, bandwidth utilization, packet loss rate, and the like, as well as identification of network bottlenecks or problems that may exist in terminal devices.

[0076] The source device (i.e., the terminal device used by the speaker of the audio or video conference) will first perform self-diagnosis and use the built-in network test tool to detect key indicators such as the speed, stability, delay, and packet loss rate of its own network connection to obtain the first network transmission quality of the source itself. These tests can include Ping tests (used to measure network delays), Traceroute or Tracert tests (used to track the transmission path of data packets in the network), and network bandwidth tests.

[0077] For each receiving-end device (ie, a terminal device that receives the audio from the source end in an audio and video conference), the source-end device sends a network diagnosis request to obtain the second network transmission quality of each receiving end.

[0078] Step S12, based on the first network transmission quality, each of the second network transmission qualities and the conference type of the audio and video conference, determining a transmission mode for each of the receiving ends, wherein for each receiving end, the transmission mode represents a transmission method of data between the source end and the receiving end;

[0079] In the implementation manner of the present application, the conference type of the audio and video conference affects the setting of the network quality requirements. Therefore, different types of conferences have different requirements for the audio network quality. The conference types may include formal meetings, ordinary meetings, seminars and emergency meetings, but are not limited to these.

[0080] The transmission mode is any one of an audio transmission mode and a text transmission mode. The audio transmission mode is a process of transmitting the speaker's audio data to various receiving ends through a network in an audio or video conference. The text transmission mode is a process of converting the speaker's audio data into text data in an audio or video conference and transmitting the text data to various receiving ends through a network.

[0081] Specifically, step S12 includes the following sub-steps:

[0082] Step S121, based on the conference type of the audio and video conference, determining the network quality requirement corresponding to the conference type, and presetting different network quality requirements for different conference types;

[0083] Step S122, when the first network transmission quality does not meet the network quality requirement, determining that the transmission mode of the source end to each of the receiving ends is a text transmission mode;

[0084] Step S123, when the first network transmission quality meets the network quality requirement and the second network transmission quality of the receiving end does not meet the network quality requirement, determining that the transmission mode between the source end and the receiving end that does not meet the network quality requirement is a text transmission mode;

[0085] Step S124, when the first network transmission quality meets the network quality requirement and the second network transmission quality of the receiving end meets the preset network quality requirement, determine that the transmission mode of the source end to the receiving end that meets the preset network quality requirement is an audio transmission mode.

[0086] By flexibly adjusting the transmission mode according to the network conditions of the source and receiving ends and the type of conference, the transmission mode can be controlled to be audio transmission mode or text transmission mode, thereby enhancing the adaptability and flexibility of the conference. Different types of conferences may have different requirements for audio quality. This dynamic adjustment can meet the needs of different conference scenarios. Adjusting the transmission mode according to network quality can optimize the utilization of network resources. When the network quality is good, using the audio transmission mode can provide a better conference experience; when the network quality is poor, switching to the text transmission mode ensures that the information can be accurately transmitted, avoiding audio quality degradation or interruption caused by network delays, packet loss and other problems, while reducing network bandwidth usage and freeing up network resources for other applications or services.

[0087] Furthermore, when the first network transmission quality does not meet the network quality requirement, after determining that the transmission mode of the source end to each of the receiving ends is a text transmission mode, it also includes: closing the audio transmission path of the source end;

[0088] When the first network transmission quality meets the network quality requirement and the second network transmission quality of the receiving end does not meet the network quality requirement, after determining that the transmission mode between the source end and the receiving end that does not meet the network quality requirement is a text transmission mode, the method further includes:

[0089] The audio transmission path of the receiving end that does not meet the network quality requirement is marked as unavailable.

[0090] In an implementation manner of the present application, the audio transmission path at the end where the network transmission quality does not meet the network quality requirements is blocked. When the network transmission quality of the terminal device where the speaker is located does not meet the network quality requirements, the voice data on the side of the terminal device where the speaker is located is blocked; when the network transmission quality at the receiving end does not meet the network quality requirements, the voice data is blocked only on the side of the receiving end where the network transmission quality does not meet the network quality requirements.

[0091] By closing the audio transmission path at the source end or marking the audio transmission path at the receiving end that does not meet the conditions as unavailable, the transmission of audio data can be stopped immediately, thereby significantly reducing the network bandwidth usage. If the network conditions are not sufficient to support audio transmission, continuing to attempt audio transmission will result in a waste of resources, such as excessive use of CPU, memory, and network bandwidth. By closing or marking the unavailable status, this unnecessary waste of resources can be avoided. In the case of poor network conditions, attempting to transmit audio may result in reduced sound quality, stuttering, heavy sounds, and sound confusion, affecting the user experience.

[0092] Step S13, obtaining the audio data of the speaker, and determining the target data corresponding to each of the receiving ends based on the audio data and the transmission mode of each of the receiving ends, wherein the target data represents the data obtained after the audio data is processed by the transmission mode of the receiving end;

[0093] In the implementation manner of the present application, determining the target data corresponding to each of the receiving ends based on the audio data and the transmission mode of each of the receiving ends includes:

[0094] When the transmission mode is a text transmission mode, the audio data is converted into text data based on a preset voice intelligent recognition technology, and the preset voice intelligent recognition technology may be ASR technology (Automatic Speech Recognition);

[0095] Verifying the text data, generating a verification code corresponding to the text data, encapsulating the verification code and the text data, and using the encapsulated verification code and text data as target data corresponding to the receiving end, wherein the verification algorithm may be a CRC algorithm (Cyc lic Redundancy Check), and the verification code is used to verify the integrity of the text data at the receiving end;

[0096] When the transmission mode is an audio transmission mode, the audio data is used as the target data corresponding to the receiving end. In the audio transmission mode, the audio data does not need to be additionally converted. However, the audio data can be appropriately encoded and compressed to meet the bandwidth and delay requirements of the network transmission.

[0097] By converting audio data into text data through intelligent voice recognition technology, it is possible to reduce audio distortion or interruption problems that may occur during network transmission, and improve the accuracy and reliability of information transmission. Verifying text data and generating checksums further ensures the integrity and accuracy of the data, and reduces errors or losses that may occur during data transmission. Since text data and checksums are transmitted, the data volume is smaller than that of audio data, which can significantly reduce bandwidth usage and improve the transmission quality of target data.

[0098] Step S14, sending the target data corresponding to each of the receiving terminals to the corresponding receiving end.

[0099] In the implementation manner of the present application, the target data corresponding to each receiving terminal is converted into target data of low-bandwidth characters and sent to the corresponding receiving end.

[0100] Specifically, step S14 includes: compressing the target data corresponding to each of the receiving terminals based on a preset compression algorithm, and sending each compressed target data to the corresponding receiving terminal.

[0101] By compressing the target data, its size is significantly reduced, reducing the network bandwidth required during transmission, thereby speeding up data transmission.

[0102] As another optional implementation of the embodiment of the present application, after sending the target data corresponding to each of the receiving terminals to the corresponding receiving end, the method further includes:

[0103] The source end receives the playback completion status identifiers sent by each of the receiving ends, and displays each of the playback completion status identifiers.

[0104] By displaying the playback completion status, conference speakers can instantly understand which receivers have successfully received data and which receivers may have problems. This facilitates quick identification and resolution of potential transmission problems, ensuring that conference information can be accurately and timely conveyed to all participants, thereby improving communication efficiency in meetings.

[0105] The conference audio processing method based on voice intelligence applied to the source end obtains the network transmission quality of the source end and each receiving end in real time, and determines the transmission mode based on the information of its own first network transmission quality, the second network transmission quality of each receiving end, and the conference type of the audio and video conference. In this way, the data transmission method can be adjusted according to the network conditions, and it can also adapt to different conference types. By processing the speaker's audio data and determining the target data based on the transmission mode of the receiving end, the intelligent processing of the audio data is realized, ensuring that the audio data can maintain a high quality during the transmission process, so that in the audio and video conference, each receiving end can obtain clear and smooth audio data, improve the accuracy of audio data transmission in the audio and video conference under network disturbance, and reduce communication barriers caused by network disturbance.

[0106] Figure 2 The present invention is a flowchart of a conference audio processing method based on voice intelligence applied to a receiving end according to an embodiment of the present invention.

[0107] like Figure 2 As shown, the main process of a conference audio processing method based on voice intelligence applied to a receiving end includes:

[0108] Step S21, in an audio and video conference, obtaining target data sent by a source end based on audio data and a transmission mode, wherein the source end represents a terminal device where a speaker in the audio and video conference is located, the transmission mode is a transmission mode of data between the source end and the receiving end, and the target data represents data obtained after the audio data is processed by the transmission mode of the receiving end;

[0109] In an implementation manner of the present application, the receiving end decompresses the target data to obtain decompressed target data. When the transmission mode is an audio transmission mode, the target data is audio data. When the transmission mode is a text transmission mode, the target data is encapsulated text data and a check code corresponding to the text data.

[0110] Step S22, preprocessing the target data to obtain processed voice data;

[0111] In the implementation manner of the present application, step S22 specifically includes, when the target data is a packaged verification code and text data, verifying the text data based on a preset verification rule and the verification code;

[0112] If the text data passes the verification, the text data is converted into voice data based on a preset conversion rule;

[0113] When the target data is audio data, the audio data is directly used as voice data.

[0114] In the implementation manner of the present application, the receiving end applies a preset verification rule (CRC verification) to verify the text data.

[0115] Further, based on a preset conversion rule, converting the text data into voice data for playback includes the following sub-steps:

[0116] Step S221, obtaining identification information and audio feature information of the source end, wherein the audio feature information represents the voice characteristics of the speaker of the source end in the audio and video conference;

[0117] Step S222, based on the identification information, determining the speech feature value corresponding to the source end in the feature value library;

[0118] Step S223, comparing the audio feature information with the speech feature value;

[0119] Step S224: if the audio feature information is compared with the speech feature value, the speech feature value is used as an input parameter of a preset text conversion model, and the text data is converted into speech data for playback through speech synthesis technology.

[0120] In the above implementation, the identification information may be a unique ID, user name or other identifier of the source end, used to identify a specific speaker. The audio feature information is extracted from the audio information of the speaker's most recent speech extracted by the source end in the audio and video conference, including pitch, timbre, speech speed, etc.

[0121] The receiving end accesses a pre-stored feature value library based on the identification information of the source end, and the feature value library contains the mapping relationship between multiple source ends (ie, terminal devices where different speakers are located) and their respective speech feature values.

[0122] In the feature value library, the receiving end retrieves the corresponding speech feature value according to the identification information of the source end. These speech feature values ​​are pre-stored and can be obtained by recording and analyzing the speaker at a certain stage before the start of the audio and video conference.

[0123] Afterwards, the receiving end will compare the acquired audio feature information with the speech feature value retrieved from the feature value library. If the similarity between the audio feature information and the speech feature value reaches a preset threshold, the comparison is successful, and the receiving end will use the speech feature value as one of the input parameters of the text conversion model to convert the text data into speech data, ensuring that the converted speech data is closer to the speaker at the source end in terms of speech characteristics.

[0124] Step S23, playing the voice data, and after the voice data is played, sending a play completion status mark to the source end.

[0125] In the implementation mode of the present application, when the voice data is played, the receiving end generates a message indicating the completion of the playback status and sends it back to the source end through the network. This message may include the receiving end's identification, the timestamp of the completion of the playback, the verification completion information, and other information, so that the source end confirms that the semantic transmission is successful. In the case of poor network conditions, by receiving and playing the converted voice data instead of the original audio data, the dependence on network bandwidth can be reduced.

[0126] The conference audio processing method based on voice intelligence applied to the receiving end ensures the accuracy and integrity of the received text data by receiving the encapsulated check code and text data and performing verification at the receiving end. This verification mechanism helps to reduce errors and packet loss during data transmission and improve the communication quality of the entire meeting. After receiving the correct text data, it is converted into voice data for playback through preset conversion rules, which can ensure that even if the network conditions are poor and the audio transmission quality is reduced, the receiving end user can still clearly hear the speaker's content. This processing method avoids the degradation or interruption of sound quality caused by network problems, thereby optimizing the user's conference experience. After the voice data is played, the receiving end sends the playback completion status to the source end. The timely feedback mechanism facilitates the source end to understand the data verification and playback status of the receiving end, so as to make corresponding adjustments or notify other participants as needed. This interactivity enhances the real-time and interactive nature of the meeting.

[0127] Figure 3 This is a structural diagram of a conference audio processing system based on voice intelligence applied to a source end according to an embodiment of the present invention.

[0128] like Figure 3 As shown, a conference audio processing system 300 based on voice intelligence is applied to the source end and mainly includes:

[0129] The network quality identification module 301 is used to obtain the first network transmission quality of the source end itself and the second network transmission quality of each receiving end in the audio and video conference;

[0130] A mode management module 302, configured to determine a transmission mode for each of the receiving ends based on the first network transmission quality, each of the second network transmission qualities, and the conference type of the audio and video conference, wherein for each receiving end, the transmission mode represents a transmission method of data between the source end and the receiving end;

[0131] The first data receiving and processing module 303 is used to obtain the audio data of the speaker, and determine the target data corresponding to each receiving end based on the audio data and the transmission mode of each receiving end. For each receiving end, the target data represents the data obtained after the audio data is processed by the transmission mode of the receiving end;

[0132] The transmission module 304 is used to send the target data corresponding to each of the receiving terminals to the corresponding receiving end.

[0133] Optionally, the transmission mode is any one of an audio transmission mode and a text transmission mode, and the mode management module 302 is specifically used for:

[0134] Based on the conference type of the audio and video conference, determine the network quality requirements corresponding to the conference type;

[0135] When the first network transmission quality does not meet the network quality requirement, determining that the transmission mode of the source end to each of the receiving ends is a text transmission mode;

[0136] When the first network transmission quality meets the network quality requirement, and the second network transmission quality of the receiving end does not meet the network quality requirement, determining that the transmission mode between the source end and the receiving end that does not meet the network quality requirement is a text transmission mode;

[0137] When the first network transmission quality meets the network quality requirement and the second network transmission quality of the receiving end meets the preset network quality requirement, it is determined that the transmission mode of the source end to the receiving end that meets the preset network quality requirement is an audio transmission mode.

[0138] Optionally, the first data receiving and processing module 303 includes:

[0139] A first voice conversion submodule, used for converting the audio data into text data based on a preset voice intelligent recognition technology when the transmission mode is a text transmission mode;

[0140] A data check submodule, used to check the text data, generate a check code corresponding to the text data, encapsulate the check code and the text data, and use the encapsulated check code and text data as target data corresponding to the receiving end;

[0141] The second voice conversion submodule is used to use the audio data as target data corresponding to the receiving end when the transmission mode is the audio transmission mode.

[0142] Optionally, a conference audio processing device 200 based on voice intelligence applied to a receiving end further includes an audio blocking module, which is specifically used to: when the first network transmission quality does not meet the network quality requirement, after determining that the transmission mode of the source end to each of the receiving ends is a text transmission mode, further includes: closing the audio transmission path of the source end;

[0143] When the first network transmission quality meets the network quality requirement and the second network transmission quality of the receiving end does not meet the network quality requirement, after determining that the transmission mode between the source end and the receiving end that does not meet the network quality requirement is a text transmission mode, it also includes: marking the audio transmission path of the receiving end that does not meet the network quality requirement as unavailable.

[0144] Optionally, the transmission module 304 is specifically used for:

[0145] The target data corresponding to each of the receiving terminals is compressed based on a preset compression algorithm, and each compressed target data is sent to the corresponding receiving terminal.

[0146] Optionally, a conference audio processing device 200 based on voice intelligence applied to a receiving end further includes a display module, and the display module is specifically used for:

[0147] Receive the playback completion status identifier sent by each of the receiving ends, and display each of the playback completion status identifiers.

[0148] Figure 4 The present invention is a structural diagram of a conference audio processing system based on voice intelligence applied to a receiving end according to an embodiment of the present invention.

[0149] like Figure 4 As shown, a conference audio processing system 400 based on voice intelligence applied to a receiving end mainly includes:

[0150] The second data receiving and processing module 401 is used to obtain target data sent by the source end based on audio data and a transmission mode in an audio and video conference, wherein the source end represents a terminal device where a speaker in the audio and video conference is located, the transmission mode is a transmission mode of data between the source end and the receiving end, and the target data represents data obtained after the audio data is processed by the transmission mode of the receiving end;

[0151] A data verification module 402 is used to pre-process the target data to obtain processed voice data;

[0152] The feedback module 403 is used to play the voice data and send a play completion status mark to the source end after the voice data is played.

[0153] Optionally, the speech synthesis module 403 includes:

[0154] A data verification submodule, for verifying the text data based on a preset verification rule and the verification code when the target data is a packaged verification code and text data;

[0155] A data synthesis submodule, configured to convert the text data into voice data based on a preset conversion rule if the text data passes the verification;

[0156] The data synthesis submodule is specifically used for:

[0157] Acquire identification information and audio feature information of the source end, wherein the audio feature information represents the voice characteristics of the speaker of the source end in the audio and video conference;

[0158] Based on the identification information, determining a speech feature value corresponding to the source end in the feature value library;

[0159] Comparing the audio feature information with the speech feature value;

[0160] If the audio feature information is compared with the speech feature value, the speech feature value is used as an input parameter of a preset text conversion model, and the text data is converted into speech data for playback through speech synthesis technology.

[0161] In one example, the module in any of the above systems may be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0162] For another example, when the modules in the system can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0163] Various objects such as various messages / information / equipment / network elements / systems / devices / actions / operations / processes / concepts that may appear in this application are named. It can be understood that these specific names do not constitute a limitation on the relevant objects. The names assigned may change with factors such as scenarios, contexts or usage habits. The understanding of the technical meaning of the technical terms in this application should be mainly determined from the functions and technical effects embodied / executed in the technical scheme.

[0164] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0165] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

Claims

1. A conference audio processing method based on voice intelligence, characterized in that: Applied to a source end, the source end represents a terminal device where a speaker in an audio or video conference is located, and the method includes: In an audio and video conference, obtaining the first network transmission quality of the source end itself and the second network transmission quality of each receiving end; Determine a transmission mode for each of the receiving ends based on the first network transmission quality, each of the second network transmission qualities, and the conference type of the audio and video conference, wherein for each receiving end, the transmission mode represents a transmission method of data between the source end and the receiving end; Acquire audio data of the speaker, and determine target data corresponding to each of the receiving ends based on the audio data and the transmission mode of each of the receiving ends, wherein for each receiving end, the target data represents data obtained after the audio data is processed by the transmission mode of the receiving end; Sending target data corresponding to each of the receiving terminals to the corresponding receiving end; The transmission mode is any one of an audio transmission mode and a text transmission mode, and the determining of the transmission mode for each of the receiving ends based on the first network transmission quality, each of the second network transmission qualities, and the conference type of the audio and video conference includes: Based on the conference type of the audio and video conference, determine the network quality requirements corresponding to the conference type; When the first network transmission quality does not meet the network quality requirement, determining that the transmission mode of the source end to each of the receiving ends is a text transmission mode; When the first network transmission quality meets the network quality requirement, and the second network transmission quality of the receiving end does not meet the network quality requirement, determining that the transmission mode between the source end and the receiving end that does not meet the network quality requirement is a text transmission mode; When the first network transmission quality meets the network quality requirement and the second network transmission quality of the receiving end meets the preset network quality requirement, it is determined that the transmission mode of the source end to the receiving end that meets the preset network quality requirement is an audio transmission mode.

2. According to claim 1, a conference audio processing method based on voice intelligence is characterized in that: The determining, based on the audio data and the transmission mode of each of the receiving ends, target data corresponding to each of the receiving ends includes: When the transmission mode is a text transmission mode, the audio data is converted into text data based on a preset voice intelligent recognition technology; Verifying the text data, generating a verification code corresponding to the text data, encapsulating the verification code and the text data, and using the encapsulated verification code and text data as target data corresponding to the receiving end; When the transmission mode is an audio transmission mode, the audio data is used as target data corresponding to the receiving end.

3. According to the method for conference audio processing based on voice intelligence in claim 1, it is characterized in that: When the first network transmission quality does not meet the network quality requirement, after determining that the transmission mode of the source end to each of the receiving ends is a text transmission mode, the method further includes: Close the audio transmission path of the source end; When the first network transmission quality meets the network quality requirement and the second network transmission quality of the receiving end does not meet the network quality requirement, after determining that the transmission mode between the source end and the receiving end that does not meet the network quality requirement is a text transmission mode, the method further includes: The audio transmission path of the receiving end that does not meet the network quality requirement is marked as unavailable.

4. A conference audio processing method based on voice intelligence according to claim 2, characterized in that: The sending the target data corresponding to each of the receiving terminals to the corresponding receiving end includes: The target data corresponding to each of the receiving terminals is compressed based on a preset compression algorithm, and each compressed target data is sent to the corresponding receiving terminal.

5. The conference audio processing method based on voice intelligence according to claim 1 is characterized in that: After sending the target data corresponding to each of the receiving terminals to the corresponding receiving end, the method further includes: Receive the playback completion status identifier sent by each of the receiving ends, and display each of the playback completion status identifiers.

6. A conference audio processing method based on voice intelligence, characterized in that: Applied to a receiving end, the receiving end represents a terminal device that receives audio data of a terminal device where a speaker is located in an audio and video conference, and the method includes: In an audio and video conference, the target data sent by the source end based on the audio data and the transmission mode is obtained. The source end represents the terminal device where the speaker in the audio and video conference is located. The transmission mode is the transmission method of data between the source end and the receiving end. The target data represents the data obtained after the audio data is processed by the transmission method of the receiving end. Preprocessing the target data to obtain processed speech data; Play the voice data, and after the voice data is played, send a play completion status identifier to the source end; The preprocessing of the target data to obtain processed speech data includes: When the target data is a packaged verification code and text data, verifying the text data based on a preset verification rule and the verification code; If the text data passes the verification, the text data is converted into voice data based on a preset conversion rule; The converting of the text data into voice data based on a preset conversion rule includes: Acquire identification information and audio feature information of the source end, wherein the audio feature information represents the voice characteristics of the speaker of the source end in the audio and video conference; Based on the identification information, determining a speech feature value corresponding to the source end in the feature value library; Comparing the audio feature information with the speech feature value; If the audio feature information is compared with the speech feature value, the speech feature value is used as an input parameter of a preset text conversion model, and the text data is converted into speech data through speech synthesis technology.

7. A conference audio processing system based on voice intelligence, characterized in that: Applied to a source end, the source end represents a terminal device where a speaker in an audio or video conference is located, and the system includes: A network quality identification module is used to obtain the first network transmission quality of the source end itself and the second network transmission quality of each receiving end in an audio and video conference; a mode management module, configured to determine a transmission mode for each of the receiving ends based on the first network transmission quality, each of the second network transmission qualities, and the conference type of the audio and video conference, wherein for each receiving end, the transmission mode represents a transmission method of data between the source end and the receiving end, and the transmission mode is any one of an audio transmission mode and a text transmission mode; A first data receiving and processing module, configured to obtain audio data of a speaker, and determine target data corresponding to each receiving end based on the audio data and the transmission mode of each receiving end, wherein for each receiving end, the target data represents data obtained after the audio data is processed by the transmission mode of the receiving end; A transmission module, used for sending the target data corresponding to each of the receiving terminals to the corresponding receiving end; The mode management module is specifically used for: Based on the conference type of the audio and video conference, determine the network quality requirements corresponding to the conference type; When the first network transmission quality does not meet the network quality requirement, determining that the transmission mode of the source end to each of the receiving ends is a text transmission mode; When the first network transmission quality meets the network quality requirement, and the second network transmission quality of the receiving end does not meet the network quality requirement, determining that the transmission mode between the source end and the receiving end that does not meet the network quality requirement is a text transmission mode; When the first network transmission quality meets the network quality requirement and the second network transmission quality of the receiving end meets the preset network quality requirement, it is determined that the transmission mode of the source end to the receiving end that meets the preset network quality requirement is an audio transmission mode.

8. A conference audio processing system based on voice intelligence, characterized in that: Applied to a receiving end, the receiving end represents a terminal device that receives the audio of a terminal device where a speaker is located in an audio and video conference, and the system includes: The second data receiving and processing module is used to obtain target data sent by the source end based on audio data and a transmission mode in an audio and video conference, wherein the source end represents a terminal device where a speaker in the audio and video conference is located, the transmission mode is a transmission mode of data between the source end and the receiving end, and the target data represents data obtained after the audio data is processed by the transmission mode of the receiving end; A preprocessing module, used for preprocessing the target data to obtain processed speech data; The feedback module is used to play the voice data and send a play completion status mark to the source end after the voice data is played. The preprocessing module is specifically used to: When the target data is a packaged verification code and text data, verifying the text data based on a preset verification rule and the verification code; If the text data passes the verification, the text data is converted into voice data based on a preset conversion rule; The converting of the text data into voice data based on a preset conversion rule includes: Acquire identification information and audio feature information of the source end, wherein the audio feature information represents the voice characteristics of the speaker of the source end in the audio and video conference; Based on the identification information, determining a speech feature value corresponding to the source end in the feature value library; Comparing the audio feature information with the speech feature value; If the audio feature information is compared with the speech feature value, the speech feature value is used as an input parameter of a preset text conversion model, and the text data is converted into speech data through speech synthesis technology.

Images