Converged communication method and apparatus based on sip protocol and half-duplex TCP protocol

Abstract

A converged communication method and apparatus based on a SIP protocol and a half-duplex TCP protocol, which method and apparatus relate to the technical field of communications. The method comprises: separately establishing session connections between a SIP client and a server and between a TCP client and the server; collecting raw audio data by means of a sound sensor of a first client, and preprocessing the raw audio data to obtain audio data; encoding the audio data into an audio packet in a specified format and sending the audio packet to the server, processing the audio packet by means of the server and then sending the processed audio packet to a second client, wherein the first client and the second client are each one of the SIP client and the TCP client, and the second client is different from the first client; and performing validity testing on the processed audio packet by means of the second client, and when the testing is successful, playing the audio packet by means of an audio player of the second client. Implementing the technical solution provided in the present application achieves efficient, secure and reliable communication between different clients in a command and dispatch platform.

Description

A converged communication method and apparatus based on SIP protocol and half-duplex TCP protocol Technical Field

[0001] This application relates to the technical field of communication technology, specifically to a converged communication method and apparatus based on SIP protocol and half-duplex TCP protocol. Background Technology

[0002] With the rapid development of information technology, various communication protocols and technologies are constantly emerging, enabling command and dispatch platforms to achieve more efficient and reliable information transmission between multiple points.

[0003] Existing technical solutions are mainly based on a single protocol communication method, which realizes data transmission between the client and the server through this protocol. That is, a connection is directly established between the client and the server, and real-time audio and video communication is carried out through a specific protocol. Although the single protocol approach is simple and easy to implement, it is inadequate when facing complex application scenarios, especially when multiple types of clients need to be supported simultaneously.

[0004] Therefore, there is an urgent need for a new converged communication method to overcome these problems in existing technologies. Summary of the Invention

[0005] This application provides a converged communication method and apparatus based on SIP protocol and half-duplex TCP protocol, which realizes efficient, secure and reliable communication between different clients in the command and dispatch platform.

[0006] The first aspect of this application provides a converged communication method based on SIP and half-duplex TCP protocols, applied to a command and dispatch platform. The command and dispatch platform includes a server, a SIP client, and a TCP client. The method includes:

[0007] Establish session connections between the SIP client, TCP client, and server respectively, and obtain access permissions for the sound sensors of the SIP client and TCP client;

[0008] Raw audio data is acquired through the sound sensor of the first client, and the raw audio data is preprocessed to obtain audio data. The first client is either the SIP client or the TCP client. The preprocessing includes format conversion and merging / compression.

[0009] The audio data is encoded into a voice packet of a specified format and sent to the server. The server processes the audio packet and then sends it to the second client. The second client is either the SIP client or the TCP client, and the second client is different from the first client.

[0010] The second client performs a validity check on the processed audio package. If the check passes, the audio package is played through the audio player on the second client.

[0011] Optionally, the preprocessing of the original audio data to obtain audio data includes:

[0012] The original audio data is converted into first audio data in ALAW format, and the first audio data is converted into second audio data in PCM format;

[0013] The second audio data is merged and compressed to convert it into audio data with a preset sampling rate and sampling bit depth.

[0014] Optionally, encoding the audio data into a voice packet of a specified format and sending it to the server includes:

[0015] The audio data is converted into Opus format audio packets using Opus encoding, and then the Opus format audio packets are encapsulated into RTP voice packets using the RTP protocol. The RTP voice packets are then sent to the server at fixed time intervals.

[0016] Optionally, the step of encapsulating Opus format audio packets into RTP voice packets using the RTP protocol includes:

[0017] Add a data length field of length 1 number of bytes and a data type field of length 2 number of bytes to the front of the Opus format audio packet to form a third number of bytes RTP data packet header, wherein the third number is the sum of the first number and the second number;

[0018] The maximum allowable size of each RTP voice packet is set according to network bandwidth, latency requirements, and terminal device processing capabilities.

[0019] The Opus format audio packets with RTP headers are divided into multiple RTP voice packets according to the maximum allowed size, and each RTP voice packet is marked with a sequence number.

[0020] Optionally, the step of processing the audio packet through the server and sending it to the second client includes:

[0021] Remove the packet header and RTP header from the RTP voice packet to restore it to the Opus format voice packet, and convert the Opus format voice packet into a PCM voice packet with the preset sampling rate and number of samples.

[0022] The PCM voice packets are converted into ALAW format voice packets according to a preset offset. The ALAW format voice packets are then sorted by data concatenation and sent to the second client.

[0023] Optionally, the step of removing the packet header and RTP header from the RTP voice packet to restore it to the Opus format voice packet, and converting the Opus format voice packet into a PCM voice packet with the preset sampling rate and number of samples includes:

[0024] Remove the third and fourth bytes of the RTP header from the received RTP voice packet to recover the Opus format audio data segment.

[0025] The Opus format audio data segment is decoded using the Opus decoding algorithm to convert it into uncompressed PCM format audio data.

[0026] Based on a preset sampling rate and sampling bit depth, the PCM audio data is resampled and bit-decreased to obtain a PCM voice packet.

[0027] Optionally, converting the PCM voice packet into an ALAW format voice packet according to a preset offset includes:

[0028] Each audio sample value in the PCM voice packet is mapped to an ALAW format encoded value according to a preset conversion rule;

[0029] The converted ALAW encoded values ​​are packaged to form a continuous ALAW format voice packet.

[0030] A second aspect of this application provides a converged communication system based on SIP and half-duplex TCP protocols, including a connection module, an acquisition module, an execution module, and a playback module, wherein:

[0031] The connection module is configured to establish session connections between the SIP client, the TCP client, and the server, respectively, and to obtain access permissions for the sound sensors of the SIP client and the TCP client.

[0032] The acquisition module is configured to acquire raw audio data through the sound sensor of the first client and preprocess the raw audio data to obtain audio data. The first client is one of the SIP client and the TCP client. The preprocessing includes format conversion and merging compression.

[0033] The execution module is configured to encode the audio data into a voice packet of a specified format and send it to the server. The server processes the audio packet and then sends it to a second client. The second client is either a SIP client or a TCP client, and the second client is different from the first client.

[0034] The playback module is configured to perform a validity check on the processed audio package through the second client, and play the audio package through the audio player of the second client after the check is passed.

[0035] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any of the foregoing.

[0036] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed, perform the method described in any of the preceding descriptions.

[0037] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0038] 1. By combining the SIP and half-duplex TCP protocols, seamless communication between clients (SIP clients and TCP clients) under different communication protocols is achieved. This improves the communication efficiency and flexibility of the command and dispatch platform, enabling information to be transmitted smoothly in different device and network environments;

[0039] 2. By obtaining access permissions to the sound sensors of the SIP and TCP clients, it is ensured that only authorized devices can collect and transmit audio data, thereby enhancing communication security and privacy protection;

[0040] 3. Preprocessing operations such as format conversion, merging, and compression of the raw audio data not only reduce the bandwidth requirements for data transmission but also improve the transmission efficiency and playback quality of the audio data. This is especially important for command and dispatch platforms that need to transmit and process large amounts of audio data in real time.

[0041] 4. Encode the audio data into audio packets of a specified format, and transmit and process them through the server, enabling the audio data to be shared and played between different clients. This encoding and decoding process ensures the integrity and consistency of the audio data, avoiding communication failures caused by format incompatibility;

[0042] 5. Performing a validity check on the received audio packets on the second client further ensures the security and reliability of communication. After successful verification, the audio packets are played through the second client's audio player, allowing users to hear audio information from the first client in real time, thereby improving the real-time performance and accuracy of command and dispatch. Attached Figure Description

[0043] Figure 1 is a flowchart illustrating the converged communication method based on SIP protocol and half-duplex TCP protocol disclosed in an embodiment of this application.

[0044] Figure 2 is a schematic diagram illustrating the principle of the converged communication method based on SIP protocol and half-duplex TCP protocol implemented on the command and dispatch platform disclosed in the embodiments of this application;

[0045] Figure 3 is a schematic diagram of the process of the SIP client acting as the initiator in the embodiments of this application;

[0046] Figure 4 is a schematic diagram of the TCP client acting as the initiator in an embodiment of this application;

[0047] Figure 5 is a schematic diagram of the process of TCP client acting as receiver for voice transmission and playback disclosed in the embodiments of this application;

[0048] Figure 6 is a schematic diagram of the process of a SIP client acting as a receiver for voice transmission and playback disclosed in an embodiment of this application;

[0049] Figure 7 is a schematic diagram of the modules of the converged communication system based on SIP protocol and half-duplex TCP protocol disclosed in the embodiments of this application;

[0050] Figure 8 is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.

[0051] Explanation of reference numerals in the attached drawings: 701, connection module; 702, acquisition module; 703, execution module; 704, playback module; 801, processor; 802, communication bus; 803, user interface; 804, network interface; 805, memory. Detailed Implementation

[0052] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0053] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0054] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0055] This embodiment discloses a converged communication method based on SIP protocol and half-duplex TCP protocol, applied to a command and dispatch platform. The command and dispatch platform includes a server, a SIP client, and a TCP client. Figure 1 is a flowchart illustrating the converged communication method based on SIP protocol and half-duplex TCP protocol disclosed in this embodiment. As shown in Figure 1, the method includes the following steps:

[0056] S101. Establish session connections between the SIP client, TCP client, and server respectively, and obtain access permissions for the sound sensors of the SIP client and TCP client;

[0057] S102. Acquire raw audio data through the sound sensor of the first client, and preprocess the raw audio data to obtain audio data. The first client is one of the SIP client and the TCP client. The preprocessing includes format conversion and merging compression.

[0058] S103. The audio data is encoded into a voice packet of a specified format and sent to the server. The server processes the audio packet and then sends it to the second client. The second client is one of the SIP client and the TCP client, and the second client is different from the first client.

[0059] S104. The processed audio package is validated by the second client. If the validation is successful, the audio package is played by the audio player of the second client.

[0060] SIP clients and TCP clients need to establish session connections with the server separately. For SIP clients, this typically involves SIP signaling exchange, including registration, invitation, and acknowledgment steps to establish a stable communication session. For TCP clients, the connection is established with the server through the TCP three-way handshake process. Simultaneously or subsequently, the server needs to request access permissions for the sound sensor from both the SIP and TCP clients. This typically involves authentication and permission checks to ensure that only authorized devices can access and use the sound sensor. The SIP protocol uses SIP URIs (Uniform Resource Identifiers) to identify users and services, and establishes and maintains sessions through SIP messages such as INVITE and ACK. The TCP protocol uses port numbers and IP addresses to establish connections, and confirms the connection establishment through a three-way handshake process (SYN-SYN / ACK-ACK). Access permissions for the sound sensor may be managed through Access Control Lists (ACLs), OAuth, or other authentication mechanisms. Once the first client (SIP or TCP client) obtains access permissions to the sound sensor, it begins acquiring raw audio data through the sensor. This typically involves the digitization of analog signals, i.e., converting the sound signal into a digital signal. The acquired raw audio data may require preprocessing operations such as format conversion and compression / merging. Format conversion may involve converting the audio data from one encoding format to another more suitable format for transmission and processing. Compression / merging may involve merging multiple audio frames into a single data packet and compressing it to reduce data size. Common audio encoding formats include PCM (Pulse Code Modulation) and ALAW. Compression / merging may involve splicing audio frames and applying compression algorithms, such as G.711 and G.729 audio compression standards. The preprocessed audio data needs to be encoded into speech packets of a specified format. This typically involves applying audio encoding algorithms such as Opus and G.723.1. The encoded speech packets have smaller data sizes, higher compression ratios, and better sound quality. The encoded speech packets are then sent to the server via a connection between the client and the server. For SIP clients, this may involve using RTP (Real-Time Transport Protocol) to transmit audio data. For TCP clients, data is sent directly through the TCP connection. The RTP protocol is typically used in conjunction with RTCP (Real-time Transmission Control Protocol) to provide real-time audio data transmission, error detection, and flow control. The TCP protocol provides reliable transmission services, ensuring data integrity and sequence through acknowledgment, retransmission, and flow control mechanisms. After receiving audio packets from the first client, the server may need to perform decoding, format conversion, resampling, and other processing operations to adapt to the playback requirements of the second client (SIP or TCP client).If the first client is a SIP client, then the second client is a TCP client; conversely, if the first client is a TCP client, then the second client is a SIP client. The processed audio packet is sent to the second client through the connection between the server and the second client. This may also involve the use of RTP (for SIP clients) or TCP (for TCP clients). The server may need to support multiple audio encoding formats and decoding algorithms to accommodate the needs of different clients. For audio data requiring real-time playback, the server may need to implement low-latency transmission and processing mechanisms. After receiving the audio packet from the server, the second client first performs a validity check. This typically involves checking the integrity, signature, or encryption status of the audio packet to ensure the security and authenticity of the audio data. If the validity check passes, the second client will use its audio player to play the received audio packet. This involves audio decoding, format conversion, and playback control. The validity check may involve security mechanisms such as digital signature verification and the application of encryption / decryption algorithms. The audio player may need to be configured and adjusted according to parameters such as the audio data's encoding format and sampling rate to ensure correct playback.

[0061] By establishing separate session connections between the SIP client and TCP client and the server, interconnection of devices under different communication protocols is achieved, enhancing the compatibility and flexibility of the communication system. Both instant messaging needs based on the SIP protocol and data transmission needs based on the TCP protocol can be met. Raw audio data is collected by the sound sensor of the first client and preprocessed through format conversion and compression, effectively improving the quality and transmission efficiency of the audio data. Format conversion ensures the compatibility of audio data under different devices and network environments, while compression reduces the bandwidth required for data transmission, lowering communication costs. The preprocessed audio data is encoded into voice packets of a specified format and transmitted and processed by the server, achieving efficient and reliable voice communication. This encoding method not only ensures the integrity and consistency of voice data but also reduces the complexity of data transmission, improving the overall performance of the communication system. The second client performs legality checks on the received audio packets, effectively preventing the intrusion of illegal data and malicious attacks, ensuring the security and reliability of the communication system. Simultaneously, by playing the processed audio packets through the audio player on the second client, users can hear voice information from the first client in real time, further improving the real-time performance and accuracy of communication. This method is particularly suitable for scenarios requiring efficient, real-time communication, such as command and dispatch platforms. By achieving seamless communication between SIP and TCP clients, and rapid acquisition, processing, and transmission of audio data, it can significantly improve the efficiency and accuracy of command and dispatch, providing decision-makers with timely and accurate information support.

[0062] Figure 2 is a schematic diagram illustrating the principle of the converged communication method based on SIP and half-duplex TCP protocols implemented on the command and dispatch platform disclosed in this application. As shown in Figure 2, after receiving a SIP request from a SIP client, the SIP & TCP service (i.e., the server) processes the request through the SIP and TCP services and sends audio data or messages to the corresponding TCP terminal. Similarly, after receiving a TCP request from a TCP client, the SIP & TCP service processes the request through the TCP and SIP services and sends audio data or messages to the SIP terminal.

[0063] Optionally, the preprocessing of the original audio data to obtain audio data includes:

[0064] The original audio data is converted into first audio data in ALAW format, and the first audio data is converted into second audio data in PCM format;

[0065] The second audio data is merged and compressed to convert it into audio data with a preset sampling rate and sampling bit depth.

[0066] The raw audio data is read, typically stored in linear PCM format, where each sample value represents the instantaneous amplitude of the audio signal, ranging from a minimum negative value to a maximum positive value (e.g., -32768 to 32767 for 16-bit PCM). The ALAW compression algorithm is applied to convert each PCM sample value into its corresponding ALAW encoded value. This conversion process is non-linear and aims to reduce variations in low-amplitude signals, thus saving storage space and reducing transmission bandwidth requirements. The result is first audio data encoded in ALAW format. The ALAW decoding algorithm is then applied to convert each ALAW encoded value back to its corresponding linear PCM sample value. This decoding process, the reverse of the previous compression process, aims to restore the dynamic range and detail of the original audio signal. The result is second audio data encoded in PCM format, which is now consistent in amplitude with the original audio data but may have undergone requantization or sample rate conversion (if required by subsequent steps). If the sample rate or bit depth of the audio data does not match the desired format, resampling or bit depth conversion is performed. Resampling involves changing the sample rate of the audio data to match the processing capabilities of the target device or network bandwidth limitations. Bit conversion involves changing the number of bits per sample (e.g., from 16-bit to 8-bit) to further reduce the data volume. In some cases, it may also be necessary to merge the audio data, such as combining multiple audio channels into a single mono signal or splicing multiple audio segments into a continuous file. Finally, an appropriate compression algorithm (such as lossless or lossy compression, depending on the trade-off between sound quality and file size) is applied to further reduce the data volume and optimize storage and transmission efficiency. The result is audio data that conforms to the preset sampling rate and bit depth, which is now ready for subsequent encoding, transmission, or processing steps.

[0067] Raw audio data may use different encoding formats, which can lead to compatibility issues across different devices and environments. Converting the raw audio data to ALAW format ensures compatibility during transmission and storage, as ALAW is a widely used audio compression format suitable for various communication systems and devices. Further converting the first ALAW audio data to a second PCM format fully leverages the audio quality advantages of PCM. PCM (Pulse Code Modulation) is an uncompressed digital audio format that provides higher quality audio signals, suitable for applications with high sound quality requirements. After converting the audio data from ALAW to PCM, merging and compression may be necessary. This step aims to reduce redundancy and improve storage efficiency. Merging multiple audio data segments reduces data fragmentation and improves data access speed. Simultaneously, compression further reduces the storage space occupied by the audio data, lowering storage costs. During merging and compression, the audio data can be converted to a preset sampling rate and bit depth. Sampling rate and bit depth are key factors determining audio quality. Controlling these two parameters ensures consistent sound quality across different devices and environments. For example, in some applications, it may be necessary to set the sampling rate of audio data to a specific value (such as 44.1kHz or 48kHz) to ensure the accuracy and stability of the audio signal.

[0068] Optionally, encoding the audio data into a voice packet of a specified format and sending it to the server includes:

[0069] The audio data is converted into Opus format audio packets using Opus encoding, and then the Opus format audio packets are encapsulated into RTP voice packets using the RTP protocol. The RTP voice packets are then sent to the server at fixed time intervals.

[0070] Opus (Opus Interactive Audio Codec) is an open, royalty-free audio codec format designed to provide low-latency, high-quality audio transmission and storage solutions. The core principle of the Opus codec blends multiple audio encoding and decoding technologies, including Linear Predictive Coding (LPC), MDCT transform, vector quantization, and entropy coding. The raw audio signal undergoes preprocessing steps, including filtering, resampling, and audio gain adjustment, to meet the encoder's requirements. Framing: The audio signal is divided into short time segments, typically 20 to 60 milliseconds. Each time segment is called a frame, used for subsequent processing and encoding. Feature extraction: Features are extracted from each frame; common features include short-time spectra, cepstral coefficients, and linear prediction coefficients, used for acoustic modeling and encoding. Multiple encoding techniques are employed to encode the extracted features. Opus uses a hybrid encoding approach, including MDCT transform, vector quantization, and residual coding. During encoding, appropriate encoding algorithms and parameters are selected based on the characteristics of the audio signal. Opus supports multiple encoding modes, including VoIP mode (suitable for real-time communication), music mode (suitable for music and high-fidelity audio), and voice mode (suitable for speech and speech recognition). Entropy encoding is applied to the encoded data to reduce the number of bits required for data representation. Opus uses various entropy encoding techniques, such as arithmetic coding and Huffman coding. The encoded and entropy-encoded data is packaged into audio frames, including audio data, control information, and frame synchronization flags. The packaged data forms Opus-formatted audio packets, which can be transmitted or stored. Real-time Transport Protocol (RTP) is a transport protocol for multimedia data streams over the Internet. RTP is designed to provide timing information and achieve stream synchronization. RTP typically runs on top of User Datagram Protocol (UDP), with both working together to perform transport layer functions. RTP packets consist of an RTP header and payload data. The RTP header contains information such as data type and encoding method, data packet sequence number, data transmission timestamp, and data source identifier. The receiving end can correctly reconstruct the original signal based on this information. Opus-formatted audio packets are used as the payload data of RTP packets, and corresponding RTP header information is added to encapsulate them into RTP voice packets. An RTP socket is created at the sending end to transmit RTP voice packets. A fixed time interval is set according to actual needs, for example, one RTP voice packet is sent every 20 milliseconds. Within the fixed time interval, the encapsulated RTP voice packets are sent to the server through the RTP socket. After receiving the RTP voice packets, the server can perform corresponding decoding and processing.

[0071] Opus is an open, royalty-free audio codec format designed to provide low-latency, high-quality audio transmission and storage solutions. It delivers high-quality audio transmission while maintaining low latency, making it ideal for real-time communication applications. Opus supports a wide range of bitrates, from very low (e.g., 6kbps) to very high (e.g., 512kbps), making it suitable for diverse application needs, from low-bandwidth network environments to high-quality audio storage. Opus boasts excellent compression performance, providing high-quality audio at lower bitrates, thus saving transmission bandwidth and storage space. Through Opus encoding, audio data can be efficiently compressed and encoded, reducing redundant information and improving transmission efficiency. The Opus encoder also supports mixing and splitting multiple audio streams, which is extremely useful for multi-channel audio transmission and processing, such as multiple participants in an audio conference. Encapsulating Opus-formatted audio packets into RTP voice packets using the RTP protocol ensures real-time transmission of audio data. RTP voice packets contain information such as the sequence number and timestamp of the audio data, aiding the receiving end in packet reassembly, synchronization, and error detection. By sending RTP voice packets at fixed time intervals, the continuity and real-time nature of audio data can be maintained, reducing transmission latency and jitter. Opus has a certain degree of fault tolerance, and can still provide good audio quality even in the event of network packet loss or partial data loss. It uses error correction coding and forward error correction techniques, and recovers lost audio data through methods such as resampling, interpolation, and hiding lost data.

[0072] Optionally, the step of encapsulating Opus format audio packets into RTP voice packets using the RTP protocol includes:

[0073] Add a data length field of length 1 number of bytes and a data type field of length 2 number of bytes to the front of the Opus format audio packet to form a third number of bytes RTP data packet header, wherein the third number is the sum of the first number and the second number;

[0074] The maximum allowable size of each RTP voice packet is set according to network bandwidth, latency requirements, and terminal device processing capabilities.

[0075] The Opus format audio packets with RTP headers are divided into multiple RTP voice packets according to the maximum allowed size, and each RTP voice packet is marked with a sequence number.

[0076] The data length field indicates the length of data carried in the RTP packet, and the data type field indicates the data type carried in the RTP packet. For Opus audio, it has a specific payload type code, which is negotiated and determined during the initial phase of the RTP session via the RTCP protocol or other non-RTP mechanisms. The receiving end uses this field to identify and correctly process the received data. The total length of the RTP header consists of the data length field and the data type field, among others. In this embodiment, the 4-byte data length field and the 4-byte data type field together constitute the 8-byte packet header. The maximum allowed size of the RTP voice packet ensures that the RTP packet can be effectively transmitted and processed under different network conditions. If the packet is too large, it may lead to increased transmission delay, increased packet loss rate, or inability of the terminal device to process it. Therefore, when encapsulating Opus audio data, a reasonable maximum packet size needs to be set based on these factors. According to the set maximum allowed size, the Opus format audio packet with the RTP packet header is segmented. This usually means dividing a long Opus audio frame into multiple smaller segments, each of which can be transmitted as a separate RTP packet. During packet segmentation, it's crucial to ensure that each segment is part of a complete Opus audio frame (or at least a segment that can be correctly reassembled by the receiver). Furthermore, appropriate sequence numbers and timestamps need to be added to the header of each RTP packet to help the receiver correctly reassemble and play the audio data. The sequence number identifies the order of RTP packets within the session. The receiver uses this sequence number to detect issues such as packet loss, reordering, and duplicate packets, and accordingly performs error recovery or discards redundant data. Each RTP packet is assigned a unique sequence number (typically an integer incrementing from an initial value). This sequence number is sequential throughout the RTP session and is unique to each sender.

[0077] Encapsulating Opus audio data using the RTP protocol leverages RTP's low latency and high bandwidth to achieve efficient audio data transmission. This helps ensure the real-time performance and integrity of audio data, improving user experience. By setting the maximum allowed size of each RTP voice packet, the audio data transmission strategy can be flexibly adjusted according to actual conditions. This helps maintain smooth audio data transmission even when network bandwidth is limited or terminal device processing power is insufficient. Sequence numbering each RTP voice packet helps the receiving end reassemble and sort the received audio data. This simplifies the audio data processing flow and reduces management complexity.

[0078] Optionally, the step of processing the audio packet through the server and sending it to the second client includes:

[0079] Remove the packet header and RTP header from the RTP voice packet to restore it to the Opus format voice packet, and convert the Opus format voice packet into a PCM voice packet with the preset sampling rate and number of samples.

[0080] The PCM voice packets are converted into ALAW format voice packets according to a preset offset. The ALAW format voice packets are then sorted by data concatenation and sent to the second client.

[0081] When the server receives an RTP voice packet, it first removes its packet header and RTP header. The packet header typically contains information such as data length and data type, while the RTP header contains RTP protocol-related information such as sequence number, timestamp, and Synchronization Source Identifier (SSRC). After removing these headers, the original Opus format voice packet is obtained. After removing the headers, the server restores the remaining byte stream into an Opus format voice packet according to previously set encoding rules or protocols. This process usually involves parsing and reassembling the byte stream to ensure that the restored Opus voice packet is consistent with the original audio data. Before format conversion, the server sets a preset sampling rate and number of samples according to requirements. The sampling rate determines the sampling frequency of the audio data, while the number of samples determines the bit depth of each sample point. The selection of these parameters affects the quality and size of the converted PCM voice packet. Appropriate audio processing tools or libraries (such as FFmpeg, SoX, etc.) are then used to convert the Opus format voice packet into a PCM format voice packet. This process involves decoding, resampling, and encoding audio data to ensure that the converted PCM audio packets meet the preset sampling rate and sample count requirements. Before converting PCM to ALAW, the server may set a preset offset. This offset is typically used to adjust the numerical range of the PCM audio packets to better suit the requirements of ALAW encoding. Using relevant functions or methods from audio processing tools or libraries, the PCM audio packets are converted to ALAW format audio packets. This process involves non-linear quantization of each sample point in the PCM audio packet to generate an ALAW-encoded audio packet. Before sending the ALAW format audio packet to the second client, the server may perform data concatenation. Data concatenation typically involves merging multiple ALAW audio packets according to chronological order or other rules to generate a complete audio data stream. Finally, the server sends the concatenated ALAW format audio packet to the second client via an appropriate network protocol (such as TCP, UDP, etc.). During transmission, the server may consider factors such as network bandwidth, latency, and packet loss rate to ensure the real-time performance and integrity of the audio data.

[0082] By removing the RTP header and RTP header from the RTP voice packets, the server can accurately reconstruct the original Opus format voice packets. Subsequently, the Opus format voice packets are converted into PCM voice packets with a preset sampling rate and number of samples. This step ensures that the audio data format matches the decoding capabilities of the second client, thus guaranteeing smooth audio playback. Converting the PCM voice packets to ALAW format enhances audio data compatibility. By converting to ALAW format, the server can ensure that the audio data is widely supported on the second client, improving the reliability of audio communication. Sequencing the ALAW format voice packets through data concatenation ensures the sequential order of the audio data. In real-time audio communication, the sequential order of audio data is crucial, affecting the continuity and fluency of the audio. The ordered voice packets ensure that the second client receives and plays audio in the correct order, thus avoiding audio interruptions and confusion. Although ALAW format voice packets may have some compression loss compared to PCM format, this loss is within an acceptable range, resulting in smaller data packet size and higher transmission efficiency. Especially when network bandwidth is limited, using the ALAW format can reduce the amount of data transmitted, thereby reducing network latency and packet loss rate, and improving the quality of audio communication.

[0083] Optionally, the step of removing the packet header and RTP header from the RTP voice packet to restore it to the Opus format voice packet, and converting the Opus format voice packet into a PCM voice packet with the preset sampling rate and number of samples includes:

[0084] Remove the third and fourth bytes of the RTP header from the received RTP voice packet to recover the Opus format audio data segment.

[0085] The Opus format audio data segment is decoded using the Opus decoding algorithm to convert it into uncompressed PCM format audio data.

[0086] Based on a preset sampling rate and sampling bit depth, the PCM audio data is resampled and bit-decreased to obtain a PCM voice packet.

[0087] The RTP packet header contains crucial information such as sequence numbers, timestamps, and synchronization source identifiers (SSRC), used for data transmission, synchronization, and reassembly. According to the RTP protocol specification, the header length is fixed, but the specific number of bytes may vary depending on the implementation. Typically, the RTP packet header length will not exceed a certain range (e.g., 12 to 20 bytes, including the IP, UDP, and RTP headers themselves). Here, "the third number of bytes" refers to the specific RTP packet header length, which needs to be determined based on the actual situation. In addition to the RTP packet header, Opus audio data may also include an additional encapsulation format header (such as OGG encapsulation format) when encapsulated into RTP packets. This header information is not necessary for the decoder and therefore needs to be removed before decoding. Here, "the fourth number of bytes" refers to the length of the RTP payload header or encapsulation format header, which also needs to be determined based on the actual situation. After removing the RTP packet header and the RTP header, the original Opus format audio data segments are obtained. These data segments are the output of the Opus encoder and contain a compressed representation of the audio. The Opus decoder can be a software library (such as opusdec in opus-tools) or a hardware accelerator. This embodiment uses opusdec from opus-tools as the decoder. The recovered Opus-format audio data segment is input into the Opus decoder. The decoder outputs uncompressed PCM-format audio data. This data is the raw audio signal and can be directly used for playback, processing, or storage. PCM (Pulse Code Modulation) is a digital representation method for analog signals. In digital audio processing, PCM audio data typically has a fixed sampling rate and bit depth. The preset sampling rate and bit depth are determined based on the application scenario and requirements. For example, some audio processing algorithms may require specific sampling rates and bit depths as input. If the sampling rate of the PCM audio data does not match the preset sampling rate, resampling is required. Resampling refers to changing the sampling rate of audio data through methods such as interpolation or decimation. Dedicated audio processing libraries (such as FFmpeg, Libresample, etc.) can be used to perform resampling operations. These libraries provide efficient resampling algorithms and interfaces. If the sampling bit depth of the PCM audio data does not match the preset sampling bit depth, a bit depth conversion operation is required. Bit depth conversion refers to changing the quantization bit depth of the audio data (e.g., from 16-bit to 8-bit or from 8-bit to 16-bit). Bit depth conversion can be achieved through simple bit manipulation or more complex quantization algorithms. Similarly, dedicated audio processing libraries can be used to perform bit depth conversion operations. After resampling and bit depth conversion, a PCM voice packet that meets the preset requirements can be obtained. These voice packets can be directly used for subsequent audio processing or storage operations.

[0088] By deleting the RTP packet header and RTP header (occupying the third and fourth bytes respectively), the server can accurately recover the Opus format audio data segment from the RTP voice packet. This process ensures the integrity of the audio data and avoids audio quality degradation due to data loss or corruption. The Opus format audio data segment is decoded using the Opus decoding algorithm, converting it into uncompressed PCM format audio data. The Opus decoding algorithm is known for its efficiency and low latency, ensuring the decoding speed and quality of audio data to meet the needs of real-time audio communication. The PCM audio data is resampled and bit-converted according to the preset sampling rate and bit depth. This process allows the server to flexibly adjust the sampling rate and bit depth of the PCM voice packet according to the decoding capabilities of the second client or the needs of specific application scenarios. This flexibility helps ensure the compatibility of audio data across different devices and network environments. Through accurate decoding and flexible sampling rate and bit depth conversion, the server can generate high-quality PCM voice packets. These voice packets, after being transmitted to the second client, present a clear and coherent audio effect, enhancing the user's listening experience. When decoding and converting data on the server side, processing latency can be reduced through techniques such as optimized algorithms and hardware acceleration. This helps ensure real-time transmission and playback of audio data, reducing audio interruptions or delays caused by processing latency.

[0089] Optionally, converting the PCM voice packet into an ALAW format voice packet according to a preset offset includes:

[0090] Each audio sample value in the PCM voice packet is mapped to an ALAW format encoded value according to a preset conversion rule;

[0091] The converted ALAW encoded values ​​are packaged to form a continuous ALAW format voice packet.

[0092] Each audio sample value in a PCM speech packet is typically stored in linear PCM format, meaning the sample value directly represents the amplitude of the audio signal. However, ALAW (A-law algorithm) is an audio compression algorithm that achieves compression by mapping linear PCM sample values ​​to a finite set of encoded values. In this process, the server maps each audio sample value in the PCM speech packet to an ALAW format encoded value according to a preset conversion rule. This conversion rule is usually a lookup table or mathematical function that maps the PCM sample value to a specific value in the ALAW encoding space based on its magnitude. Once all PCM sample values ​​have been converted to ALAW encoded values, the server packages these encoded values ​​into a continuous ALAW format speech packet. In ALAW format, each encoded value typically occupies 8 bits (i.e., 1 byte), so the packaging process involves arranging these encoded values ​​sequentially into a continuous byte stream. It's important to note that ALAW format speech packets may also include additional metadata, such as packet headers and timestamps, which are used to identify the structure and timing information of the speech packet. However, this metadata is not part of the PCM to ALAW conversion process but is added as needed during the packaging process.

[0093] While PCM (Pulse Code Modulation) offers excellent sound quality, its large data volume hinders storage and transmission. ALAW format, on the other hand, uses non-uniform quantization to retain more data in low-volume segments and less in high-volume segments, achieving highly efficient audio compression. This compression method not only reduces storage and transmission costs but also preserves sufficient audio detail, ensuring sound quality. Converting PCM voice packets to ALAW format facilitates audio data interoperability across different devices and network environments. Furthermore, the standardization of ALAW format simplifies audio data processing and reduces system integration complexity. Compared to PCM, ALAW format has a smaller data volume, meaning more audio data can be stored or transmitted with the same storage space or transmission bandwidth. This is undoubtedly a significant advantage for applications requiring large amounts of audio data processing (such as speech recognition and speech synthesis). Compression and decompression are two crucial steps in audio data processing. ALAW format's compression algorithm is relatively simple, and its decompression speed is fast, which helps reduce processing latency and improve the real-time performance of audio data. This is particularly important for real-time audio communication applications. Although ALAW is a lossy compression format, its non-uniform quantization method, which performs fine quantization on low-volume parts, allows it to preserve audio detail and sound quality as much as possible while maintaining compression efficiency. This makes ALAW superior to some other lossy compression formats in terms of audio quality. Converting PCM audio packets to ALAW simplifies subsequent audio processing. For example, ALAW-encoded audio data can be used directly during storage, transmission, and playback without additional decoding. This reduces system complexity and improves processing efficiency.

[0094] Figure 3 is a flowchart illustrating the process of a SIP client acting as the initiator according to an embodiment of this application. As shown in Figure 3, the process includes: S301, the SIP client performs authentication; S302, an error message is displayed when authentication fails; S303, a session is established when authentication succeeds; S304, raw audio data (Alaw format) is collected; S305, the SIP service is accessed; S306, the raw audio data is converted into PCM audio data; S307, Opus compression is performed; S308, voice packets are concatenated and sorted; S309, the TCP service is accessed; and S310, the data is transmitted to the target TCP client.

[0095] Figure 4 is a flowchart illustrating the TCP client as the initiator in an embodiment of this application. As shown in Figure 4, the process includes: S401, the TCP client performs authentication; S402, an error message is displayed when authentication fails; S403, a session is established when authentication succeeds; S404, raw audio data (PCM format) is collected; S405, Opus compression is performed; S406, voice packets are concatenated and sorted; S407, the TCP service is entered; S408, Opus is decoded into PCM audio packets; S409, the PCM audio packets are converted into Alaw audio packets; S410, the SIP service is entered; S411, the data is transmitted to the target SIP client.

[0096] Figure 5 is a schematic diagram of the process of a TCP client acting as a receiver for voice transmission and playback according to an embodiment of this application. As shown in Figure 5, the process includes: S501, the SIP client performs authentication; S502, an error message is displayed when authentication fails; S503, a session is established when authentication succeeds; S504, microphone access is obtained; S505, an error message is displayed when access acquisition fails; S506, an Alaw audio stream is captured when access acquisition succeeds; S507, the SIP service is entered; S508, Alaw audio data is converted into PCM audio data; S509, audio data is merged and compressed; S510, audio data is packetized; S511, Opus compression is performed; S512, voice packets are encapsulated in RTP format; S513, the data is sent to the TCP client, and the TCP client plays it.

[0097] Figure 6 is a schematic diagram of the process of a SIP client acting as a receiver for voice transmission and playback according to an embodiment of this application. As shown in Figure 6, the process includes: S601, TCP client performs authentication; S602, error message is displayed when authentication fails; S603, session is established when authentication succeeds; S604, microphone permission is obtained; S605, error message is displayed when permission acquisition fails; S606, PCM audio stream is captured when permission acquisition succeeds; S607, audio data is merged and compressed; S608, audio data is packetized; S609, Opus compression; S610, voice packets are encapsulated in RTP format; S611, TCP service is entered; S612, RTP voice packets are restored; S613, Opus decompression; S614, PCM audio data is converted to Alaw audio data; S615, audio data is packetized; S616, the data is sent to the SIP client, and the SIP client plays it.

[0098] This embodiment also discloses a converged communication system based on SIP protocol and half-duplex TCP protocol. Figure 7 is a schematic diagram of the modules of the converged communication system based on SIP protocol and half-duplex TCP protocol disclosed in this application embodiment. As shown in Figure 7, the system includes a connection module 701, a data acquisition module 702, an execution module 703, and a playback module 704, wherein:

[0099] The connection module 701 is configured to establish session connections between the SIP client, the TCP client, and the server, respectively, and to obtain access permissions for the sound sensors of the SIP client and the TCP client.

[0100] Acquisition module 702 is configured to acquire raw audio data through the sound sensor of the first client and preprocess the raw audio data to obtain audio data. The first client is one of the SIP client and the TCP client. The preprocessing includes format conversion and merging compression.

[0101] The execution module 703 is configured to encode the audio data into a voice packet of a specified format and send it to the server. The server processes the audio packet and then sends it to a second client. The second client is one of the SIP client and the TCP client, and the second client is different from the first client.

[0102] The playback module 704 is configured to perform a validity check on the processed audio package through the second client, and play the audio package through the audio player of the second client after the check is passed.

[0103] Optionally, the acquisition module 702 is configured to:

[0104] The original audio data is converted into first audio data in ALAW format, and the first audio data is converted into second audio data in PCM format;

[0105] The second audio data is merged and compressed to convert it into audio data with a preset sampling rate and sampling bit depth.

[0106] Optionally, the execution module 703 is configured to:

[0107] The audio data is converted into Opus format audio packets using Opus encoding, and then the Opus format audio packets are encapsulated into RTP voice packets using the RTP protocol. The RTP voice packets are then sent to the server at fixed time intervals.

[0108] Optionally, the execution module 703 is configured to:

[0109] Add a data length field of length 1 number of bytes and a data type field of length 2 number of bytes before the Opus format audio packet to form a third number of bytes RTP data packet header, wherein the third number is the sum of the first number and the second number;

[0110] The maximum allowable size of each RTP voice packet is set according to network bandwidth, latency requirements, and terminal device processing capabilities.

[0111] The Opus format audio packets with RTP headers are divided into multiple RTP voice packets according to the maximum allowed size, and each RTP voice packet is marked with a sequence number.

[0112] Optionally, the execution module 703 is configured to:

[0113] Remove the packet header and RTP header from the RTP voice packet to restore it to the Opus format voice packet, and convert the Opus format voice packet into a PCM voice packet with the preset sampling rate and number of samples.

[0114] The PCM voice packets are converted into ALAW format voice packets according to a preset offset. The ALAW format voice packets are then sorted by data concatenation and sent to the second client.

[0115] Optionally, the execution module 703 is configured to:

[0116] Remove the third and fourth bytes of the RTP header from the received RTP voice packet to recover the Opus format audio data segment.

[0117] The Opus format audio data segment is decoded using the Opus decoding algorithm to convert it into uncompressed PCM format audio data.

[0118] Based on a preset sampling rate and sampling bit depth, the PCM audio data is resampled and bit-decreased to obtain a PCM voice packet.

[0119] Optionally, the execution module 703 is configured to:

[0120] Each audio sample value in the PCM voice packet is mapped to an ALAW format encoded value according to a preset conversion rule;

[0121] The converted ALAW encoded values ​​are packaged to form a continuous ALAW format voice packet.

[0122] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided above belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0123] This embodiment also discloses an electronic device. Referring to FIG8, the electronic device may include: at least one processor 801, at least one communication bus 802, user interface 803, network interface 804, and at least one memory 805.

[0124] The communication bus 802 is used to enable communication between these components.

[0125] The user interface 803 may include a display screen and a camera. Optionally, the user interface 803 may also include a standard wired interface and a wireless interface.

[0126] The network interface 804 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0127] The processor 801 may include one or more processing cores. The processor 801 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 805, and by calling data stored in the memory 805. Optionally, the processor 801 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 801 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 801 and may be implemented as a separate chip.

[0128] The memory 805 may include random access memory (RAM) or read-only memory. Optionally, the memory 805 may include a non-transitory computer-readable storage medium. The memory 805 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 805 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 805 may also be at least one storage device located remotely from the aforementioned processor 801. As shown in FIG8, the memory 805, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program based on a converged communication method using the SIP protocol and the half-duplex TCP protocol.

[0129] In the electronic device shown in Figure 8, the user interface 803 is mainly used to provide an input interface for the user and obtain the user input data; while the processor 801 can be used to call the application program stored in the memory 805 that is based on the converged communication method of SIP protocol and half-duplex TCP protocol. When executed by one or more processors 801, the electronic device executes one or more methods as described in the above embodiments.

[0130] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0131] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0133] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0135] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 805 and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory 805 includes various media capable of storing program code, such as a USB flash drive, external hard drive, magnetic disk, or optical disk.

[0136] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the disclosure in this specification. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A converged communication method based on SIP protocol and half-duplex TCP protocol, characterized in that, Applied to a command and dispatch platform, the command and dispatch platform includes a server, a SIP client, and a TCP client, the method includes: Establish session connections between the SIP client, TCP client, and server respectively, and obtain access permissions for the sound sensors of the SIP client and TCP client; Raw audio data is acquired through the sound sensor of the first client, and the raw audio data is preprocessed to obtain audio data. The first client is either the SIP client or the TCP client. The preprocessing includes format conversion and merging / compression. The audio data is encoded into a voice packet of a specified format and sent to the server. The server processes the audio packet and then sends it to the second client. The second client is either the SIP client or the TCP client, and the second client is different from the first client. The second client performs a validity check on the processed audio package. If the check passes, the audio package is played through the audio player on the second client.

2. The converged communication method based on SIP protocol and half-duplex TCP protocol according to claim 1, characterized in that, The preprocessing of the original audio data to obtain the audio data includes: The original audio data is converted into first audio data in ALAW format, and the first audio data is converted into second audio data in PCM format; The second audio data is merged and compressed to convert it into audio data with a preset sampling rate and sampling bit depth.

3. The converged communication method based on SIP protocol and half-duplex TCP protocol according to claim 2, characterized in that, Encoding the audio data into a voice packet of a specified format and sending it to the server includes: The audio data is converted into Opus format audio packets using Opus encoding, and then the Opus format audio packets are encapsulated into RTP voice packets using the RTP protocol. The RTP voice packets are then sent to the server at fixed time intervals.

4. The converged communication method based on SIP protocol and half-duplex TCP protocol according to claim 3, characterized in that, The process of encapsulating Opus format audio packets into RTP voice packets using the RTP protocol includes: Add a data length field of length 1 number of bytes and a data type field of length 2 number of bytes to the front of the Opus format audio packet to form a third number of bytes RTP data packet header, wherein the third number is the sum of the first number and the second number; The maximum allowable size of each RTP voice packet is set according to network bandwidth, latency requirements, and terminal device processing capabilities. The Opus format audio packets with RTP headers are divided into multiple RTP voice packets according to the maximum allowed size, and each RTP voice packet is marked with a sequence number.

5. The converged communication method based on SIP protocol and half-duplex TCP protocol according to claim 3, characterized in that, The step of processing the audio packet through the server and sending it to the second client includes: Remove the packet header and RTP header from the RTP voice packet to restore it to the Opus format voice packet, and convert the Opus format voice packet into a PCM voice packet with the preset sampling rate and number of samples. The PCM voice packets are converted into ALAW format voice packets according to a preset offset. The ALAW format voice packets are then sorted by data concatenation and sent to the second client.

6. The converged communication method based on SIP protocol and half-duplex TCP protocol according to claim 5, characterized in that, The step of removing the header and RTP header from the RTP voice packet to restore it to the Opus format voice packet, and converting the Opus format voice packet into a PCM voice packet with the preset sampling rate and number of samples includes: Remove the third and fourth bytes of the RTP header from the received RTP voice packet to recover the Opus format audio data segment. The Opus format audio data segment is decoded using the Opus decoding algorithm to convert it into uncompressed PCM format audio data. Based on a preset sampling rate and sampling bit depth, the PCM audio data is resampled and bit-decreased to obtain a PCM voice packet.

7. The converged communication method based on SIP protocol and half-duplex TCP protocol according to claim 5, characterized in that, The step of converting the PCM voice packet into an ALAW format voice packet according to a preset offset includes: Each audio sample value in the PCM voice packet is mapped to an ALAW format encoded value according to a preset conversion rule; The converted ALAW encoded values ​​are packaged to form a continuous ALAW format voice packet.

8. A converged communication system based on SIP protocol and half-duplex TCP protocol, characterized in that, It includes a connection module, a data acquisition module, an execution module, and a playback module, among which: The connection module is configured to establish session connections between the SIP client, the TCP client, and the server, respectively, and to obtain access permissions for the sound sensors of the SIP client and the TCP client. The acquisition module is configured to acquire raw audio data through the sound sensor of the first client and preprocess the raw audio data to obtain audio data. The first client is one of the SIP client and the TCP client. The preprocessing includes format conversion and merging compression. The execution module is configured to encode the audio data into a voice packet of a specified format and send it to the server. The server processes the audio packet and then sends it to a second client. The second client is either a SIP client or a TCP client, and the second client is different from the first client. The playback module is configured to perform a validity check on the processed audio package through the second client, and play the audio package through the audio player of the second client after the check is passed.

9. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are both used to communicate with other devices. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-7.

Images