A real-time voice communication system, method, device and storage medium

Abstract

The application discloses a real-time voice communication system, method, device and storage medium, and the system comprises a first terminal and a second terminal; the first terminal comprises an audio acquisition module, an audio coding module and a data transmission module, the audio acquisition module is used for acquiring audio data, the audio coding module is used for adjusting and packaging the audio data, and the data transmission module is used for transmitting the packaged audio data; the second terminal comprises a buffer area management module, an audio decoding module and a playing module, the buffer area management module is used for dynamically adjusting the size of the buffer area to receive the packaged audio data transmitted by the first terminal; the audio decoding module is used for decoding the audio data, and the playing module is used for playing the audio data; the application can reduce the complexity of voice communication between the internet of things device and the mobile device, improve the communication quality and improve the data transmission performance.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a real-time voice communication system, method, device and storage medium. Background Technology

[0002] Currently, most audio and video communication and playback transmission processes use the open-source software FFmpeg. That is, IoT devices use FFmpeg to push and transmit audio data. However, porting FFmpeg, compiling it for various platforms, and trimming it are complex, cumbersome, and have long debugging cycles. At the same time, mobile devices, as the data receiving end, also need to use FFmpeg for decoding, resulting in high porting costs. Furthermore, the buffer capacity of existing mobile devices is fixed, which prevents the transmission performance from being improved. Summary of the Invention

[0003] In order to overcome the shortcomings of the prior art, one of the objectives of this invention is to provide a real-time voice communication system that can reduce the complexity of voice communication between IoT devices and mobile devices, improve communication quality, and enhance data transmission performance.

[0004] The second objective of this invention is to provide a real-time voice communication method.

[0005] The third objective of this invention is to provide an electronic device.

[0006] The fourth objective of this invention is to provide a computer-readable storage medium.

[0007] One of the objectives of this invention is achieved through the following technical solution:

[0008] A real-time voice communication system includes: a first terminal and a second terminal;

[0009] The first terminal includes an audio acquisition module, an audio encoding module, and a data transmission module. The audio acquisition module is used to acquire audio data, adjust and encapsulate the audio data based on the audio encoding module, and transmit the encapsulated audio data using the data transmission module.

[0010] The second terminal includes a buffer management module, an audio decoding module, and a playback module. The buffer management module is used to dynamically adjust the buffer size to receive the encapsulated audio data transmitted by the first terminal; the audio decoding module decodes the audio data and plays it through the playback module.

[0011] Furthermore, the second terminal also includes a noise reduction module, which performs noise reduction processing on the decoded audio data before transmitting it to the playback module for audio playback.

[0012] Furthermore, the buffer management module predicts and adjusts the memory allocation for the buffer next time based on the data stream size of the audio data read in the previous reading.

[0013] The second objective of this invention is achieved by the following technical solution:

[0014] A real-time voice communication method is applied to a second terminal of the real-time voice communication system as described above; the method includes:

[0015] The system records the memory information of each received audio data, which is sent by a first terminal connected to the second terminal. The system compares the memory information of the audio data with the allocated memory of the buffer in the second terminal used to receive the audio data. Based on the comparison result, the allocated memory of the buffer is dynamically adjusted to receive new audio data sent by the first terminal, and the received audio data is played.

[0016] Furthermore, the method for comparing the memory information of the audio data with the allocated memory of the buffer is as follows:

[0017] The memory bytes of the previously received audio data are read, and the allocated memory corresponding to the previous buffer is obtained. If the memory bytes of the previously received audio data are less than the allocated memory corresponding to the previous buffer, and the memory bytes of the audio data received in multiple consecutive transactions are all less than the preset memory threshold of the buffer, then the allocated memory of the buffer is reduced to receive the next audio data.

[0018] Furthermore, the method for reducing the allocated memory of the cache area is as follows:

[0019] Based on the previously allocated memory for the cache area, the allocated memory for the cache area is reduced by a preset memory increment; or,

[0020] The allocated memory of the cache area is reduced according to the preset memory specification table.

[0021] Furthermore, when comparing the memory information of the audio data with the allocated memory of the buffer, if the number of memory bytes of the previously received audio data is greater than or equal to the previously allocated memory of the buffer, then the allocated memory of the buffer is increased to receive the next audio data.

[0022] Furthermore, the method for increasing the allocated memory of the cache area also includes:

[0023] Determine whether the allocated memory of the increased cache area is greater than the preset upper limit in bytes. If it is, maintain the allocated memory of the cache area at the upper limit in bytes and issue an alarm.

[0024] The third objective of this invention is achieved by the following technical solution:

[0025] An electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the real-time voice communication method described above.

[0026] The fourth objective of this invention is achieved by the following technical solution:

[0027] A computer-readable storage medium having a computer program stored thereon, which, when executed, implements the above-described real-time voice communication method.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] This invention collects audio data through a first terminal, encodes and encapsulates the audio data, and then transmits it through a network transmission module. Upon receiving the data, the second terminal can directly decode and reduce noise using its audio decoding and noise reduction modules, without needing traditional audio / video processing methods like FFmpeg, and then play the processed audio. This system replaces the traditional method of processing audio and video using FFmpeg, reducing the complexity of voice communication between the first and second terminals, improving communication quality, and lowering the integration complexity of both parties. Simultaneously, the buffer management module of this invention predicts and adjusts the receiving buffer for the next reception based on the size of the previously read data stream, maximizing its ability to meet the needs of different industry application scenarios, making it more versatile and offering higher transmission performance. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the device connection of the real-time voice communication system of the present invention;

[0031] Figure 2 This is a schematic diagram of the communication process of the real-time voice communication system of the present invention. Detailed Implementation

[0032] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0033] Example 1

[0034] This invention provides a real-time voice communication system, such as... Figure 1 As shown, the system mainly includes a first terminal and a second terminal; the first terminal is an Internet of Things (IoT) device, while the second terminal is a mobile device; the first terminal and the second terminal achieve bidirectional signal communication through a routing device.

[0035] refer to Figure 1 , Figure 2 As shown, the first terminal includes an audio acquisition module, an audio encoding module, and a data transmission module. The audio acquisition module is used to acquire audio data, which can be obtained through a serial port or a wireless network. The sampling frequency and sampling format of the audio acquisition module can be preset. In this embodiment, the sampling frequency is specified as 8000 and the sampling format is ENCODING_PCM_16BIT.

[0036] The audio encoding module is used to encode and compress the acquired audio data and encapsulate it into digital audio data. In this embodiment, the audio data is encapsulated into PCM data format, and then the encapsulated data is written into a queue in sequence; the audio data is read from the queue through the data transmission module and the encapsulated audio data is transmitted to the second terminal.

[0037] The second terminal includes a buffer management module, an audio decoding module, a noise reduction module, and a playback module. The buffer management module is used to predict and adjust the memory allocation of the buffer for the next time based on the data stream size of the audio data read in the previous time. The dynamically adjusted buffer is used to receive and store the encapsulated audio data transmitted by the first terminal. The audio decoding module reads the audio data from the buffer and decodes the audio data. The noise reduction module is used to perform noise reduction processing on the decoded audio data and plays the noise-reduced audio through the playback module.

[0038] In this embodiment, an audio acquisition module and a transmission module SDK are integrated in the first terminal. The transmission is carried out through the WebSocket protocol. At the same time, the network is monitored in the second terminal, the network data receiving buffer is adjusted in real time, and data transcoding and decoding operations are performed to achieve the effect of real-time playback.

[0039] Example 2

[0040] This embodiment provides a real-time voice communication method, which is applied to a second terminal of the real-time voice communication system as described in Embodiment 1; the method includes:

[0041] The system records the memory information of each received audio data, which is sent by a first terminal connected to the second terminal. The system compares the memory information of the audio data with the allocated memory of the buffer in the second terminal used to receive the audio data. Based on the comparison result, the allocated memory of the buffer is dynamically adjusted to receive new audio data sent by the first terminal, and the received audio data is played.

[0042] The second terminal's buffer has a pre-set initial memory size, and the buffer size is dynamically adjusted based on the initial memory size; specifically:

[0043] The memory bytes of the previously received audio data are read, and the allocated memory corresponding to the previous buffer is obtained. If the memory bytes of the previously received audio data are less than the allocated memory corresponding to the previous buffer, and the memory bytes of the audio data received consecutively are all less than the preset memory threshold of the buffer, it indicates that the previously allocated buffer memory is too large. In this case, the allocated memory of the buffer is reduced to receive the next audio data. The way to reduce the buffer size can be based on the allocated memory corresponding to the previous buffer, by reducing the allocated memory of the buffer according to a preset memory range value; or by reducing the allocated memory of the buffer according to a preset memory size table, that is, by pre-setting a memory size table, which records multiple memory sizes arranged in ascending order. In this embodiment, reducing the allocated memory of the buffer means finding the position of the current memory size in the memory size table and adjusting the allocated memory of the buffer to the size of the previous memory size, so that the allocated memory of the buffer is reduced.

[0044] When comparing the memory information of the audio data with the allocated memory of the buffer, if the number of memory bytes of the previously received audio data is greater than or equal to the previously allocated memory of the buffer, then the allocated memory of the buffer is increased to receive the next audio data; the method of increasing can be the same as the method of decreasing, or it can be that the memory specification table is increased by 4 elements based on the current memory specification, for example, if the current memory specification is 32, it will be 36 after increasing.

[0045] When increasing the allocated memory of the cache, it is also necessary to determine whether the increased allocated memory of the cache exceeds a preset upper limit in bytes. If it does, the allocated memory of the cache remains at the upper limit in bytes, and an alarm is issued. In this embodiment, the upper limit in bytes is 65536 bytes, or 64MB.

[0046] The buffer predicts and adjusts the receiving buffer for the next time based on the size of the data stream actually read last time, which can meet the needs of different application scenarios in different industries to the greatest extent, making it more versatile and with higher transmission performance.

[0047] The adjusted buffer receives and stores the encapsulated audio data sent by the first terminal. Then, the audio data is decoded and noise-reduced by the audio decoding module, noise reduction module, and playback module before being played, achieving a real-time playback effect.

[0048] In some embodiments, an electronic device is also provided, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the real-time voice communication method described above; in addition, a computer-readable storage medium is also provided, on which a computer program is stored, wherein the computer program, when executed, implements the real-time voice communication method described above.

[0049] The device and storage medium in this embodiment are two aspects of the method in the foregoing embodiments, based on the same inventive concept. The implementation process of the method has been described in detail above, so those skilled in the art can clearly understand the structure and implementation process of the device and storage medium in this embodiment based on the foregoing description. For the sake of brevity, it will not be described again here.

[0050] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A real-time voice communication system, characterized in that, include: The first terminal and the second terminal; The first terminal includes an audio acquisition module, an audio encoding module, and a data transmission module. The audio acquisition module is used to acquire audio data, adjust and encapsulate the audio data based on the audio encoding module, and transmit the encapsulated audio data using the data transmission module. The second terminal includes a buffer management module, an audio decoding module, and a playback module. The buffer management module is used to dynamically adjust the buffer size to receive the encapsulated audio data transmitted by the first terminal; the audio decoding module decodes the audio data and plays it through the playback module. The buffer management module predicts and adjusts the allocated memory of the buffer for the next time based on the data stream size of the audio data read in the previous time. This includes: reading the number of memory bytes of the previously received audio data and obtaining the allocated memory corresponding to the previous buffer; if the number of memory bytes of the previously received audio data is less than the allocated memory corresponding to the previous buffer, and the number of memory bytes of the audio data received consecutively is less than a preset memory threshold for the buffer, then the allocated memory of the buffer is reduced to receive the next audio data. The method for reducing the allocated memory of the buffer is as follows: Based on the previously allocated memory for the cache area, the allocated memory of the cache area is reduced according to a preset memory range; or, the allocated memory of the cache area is reduced according to a preset memory specification table.

2. The real-time voice communication system according to claim 1, characterized in that, The second terminal also includes a noise reduction module, which performs noise reduction processing on the decoded audio data before transmitting it to the playback module for audio playback.

3. A real-time voice communication method, characterized in that, The method is applied to a second terminal of the real-time voice communication system as described in any one of claims 1 to 2; the method includes: The system records memory information for each received audio data segment, where the audio data is transmitted by a first terminal connected to the second terminal via a signal. It compares the memory information of the audio data with the allocated memory of a buffer in the second terminal used to receive the audio data, dynamically adjusting the allocated memory of the buffer based on the comparison result to receive new audio data sent by the first terminal, and then plays the received audio data. The method for comparing the memory information of the audio data with the allocated memory of the buffer is as follows: The system reads the memory bytes of the previously received audio data and obtains the allocated memory corresponding to the previous buffer. If the memory bytes of the previously received audio data are less than the allocated memory corresponding to the previous buffer, and the memory bytes of the audio data received consecutively are all less than the preset memory threshold of the buffer, then the allocated memory of the buffer is reduced to receive the next audio data. The method for reducing the allocated memory of the buffer is as follows: Based on the previously allocated memory for the cache area, the allocated memory of the cache area is reduced according to a preset memory range; or, the allocated memory of the cache area is reduced according to a preset memory specification table.

4. The real-time voice communication method according to claim 3, characterized in that, When comparing the memory information of the audio data with the allocated memory of the buffer, if the number of memory bytes of the previously received audio data is greater than or equal to the previously allocated memory of the buffer, then the allocated memory of the buffer is increased to receive the next audio data.

5. The real-time voice communication method according to claim 4, characterized in that, The method for increasing the allocated memory of the cache area also includes: Determine whether the allocated memory of the increased cache area is greater than the preset upper limit in bytes. If it is, maintain the allocated memory of the cache area at the upper limit in bytes and issue an alarm.

6. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the real-time voice communication method according to any one of claims 3 to 5.

7. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed, implements the real-time voice communication method according to any one of claims 3 to 5.

Images