A method for restoring an offline media voice stream

Abstract

The application discloses a kind of off-line media voice stream restoration methods, including off-line media voice stream analysis algorithm, RTP-SDP dynamic mapping algorithm and voice extraction restoration algorithm.Media voice stream analysis algorithm, first parse off-line media voice stream file header information, separate out SDP description information and RTP data packet, and parse SDP and RTP information according to protocol;RTP-SDP dynamic mapping algorithm, the encoder name, clock rate and encoder parameter parsed out by SDP are stored into hash table, then when processing the data carried by each packet RTP, corresponding decoder is matched from high-speed hash by RTP header load type, so multiple media data decoders can be matched in this stage;Voice extraction restoration algorithm, according to the media format transmission protocol, the complete media format data of a session is extracted and restored to the decoder that has been matched.

Description

Technical Field

[0001] This invention belongs to the field of speech encoding and decoding technology, and specifically relates to a method for restoring offline media speech streams. Background Technology

[0002] With the continuous evolution and popularization of mobile internet technology, the types of voice media formats transmitted in IMS networks are increasing, the media quality is becoming higher and higher, and the datasets are becoming larger and larger. Parsing the SDP media negotiation protocol, extracting media data carried by RTP, identifying media types, and decoding and restoring various media data for ultra-large dataset voice media streams is a complex and time-consuming problem.

[0003] Current media audio stream restoration algorithms are mainly optimized for real-time media streams. The main process is to convert the media data carried by RTP frame by frame according to the media transmission format based on the SDP description information, thereby restoring the audio media file with the specified encoding format.

[0004] The problems with this method are as follows: 1. It can only extract and restore media audio data in a single format. If the encoding format of the media stream data changes, the restoration of the media stream data may fail. 2. Due to the possibility of loss, retransmission, out-of-order delivery, or corruption of media stream data during data transmission, it cannot correctly handle abnormal data frames, which may result in noise or incorrect duration in the restored audio file. 3. When new media formats appear, they cannot be decoded in a timely manner, which may cause data loss. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides a method for restoring offline media audio streams, including an offline media audio stream parsing algorithm, an RTP-SDP dynamic mapping algorithm, and a voice extraction and restoration algorithm. The media audio stream parsing algorithm first parses the header information of the offline media audio stream file, separating the SDP description information and RTP data packets, and then parses the SDP and RTP information according to the protocol. The RTP-SDP dynamic mapping algorithm stores the encoder name, clock rate, and encoder parameters parsed from the SDP into a hash table. Then, when processing the data carried by each RTP packet, it matches the corresponding decoder from the high-speed hash table based on the RTP packet header payload type. Therefore, multiple media data decoders can be matched at this stage. The voice extraction and restoration algorithm, for the matched decoders, extracts and restores the complete media format data of a session according to the media format transmission protocol.

[0006] The technical solution adopted by this invention to solve its technical problem includes the following steps:

[0007] Step 1: Classify offline media voice stream data packets;

[0008] Read offline media audio stream data files, parse the data packet headers, and classify the data packets into SDP data packets or RTP data packets based on the data packet header information;

[0009] Step 2: Media mapping relationship identification;

[0010] Read multiple lines of media information from the SDP data packet obtained in step 1, and extract the mapping relationship between the bearer type identifier value and the bearer media type name, as well as other parameters required for media transcoding;

[0011] Step 3: RTP packet parsing;

[0012] Each RTP packet contains an RTP header and a payload; the RTP header consists of 12 bytes and contains various control information; RTP maintains synchronization between different media streams through synchronization signals, and each media stream has a unique SSRC identifier to distinguish different senders and receivers; the receiver uses SSRC to associate received packets with specific media streams to ensure synchronized playback of multiple media streams.

[0013] The steps for parsing RTP packets are as follows:

[0014] Step 3-1: Separate the RTP header: Extract the first 12 bytes from the received data packet, i.e., the RTP header;

[0015] Step 3-2: Parse the RTP header. The structure of the RTP header is as follows:

[0016] - Version: 2 digits, used to indicate the version of the RTP protocol;

[0017] - Padding bit: occupies 1 bit and is used to indicate whether there are additional padding bytes;

[0018] - Extension bit: occupies 1 bit and is used to indicate whether there is an extension portion of the RTP header;

[0019] -CSRC Count: 4 bits, indicates the number of CSRC identifiers;

[0020] - Flag bit: occupies 1 bit and is defined by the application.

[0021] - Load type: 7 bits, used to indicate the load type;

[0022] - Sequence number: 16 bits, used to identify the order of data packets;

[0023] - Timestamp: 32 bits, used to synchronize playback speed on the receiving end;

[0024] - Synchronization signal: 32 bits, used to identify the sender;

[0025] -CSRC identifier: 32 bits, used to identify other sources participating in synchronization;

[0026] Step 3-3: Extract payload data: Extract the payload data portion based on the information in the RTP header; the payload data is the actual content of the media.

[0027] Step 4: Reassembling and sorting voice data packets

[0028] Sort the RTP packets; the Seq field in the RTP header information indicates the sequence number of the packet, and sort each RTP in the list in ascending order based on this field;

[0029] Step 5: Dynamic mapping of RTP-SDP;

[0030] SDP is used to describe the parameters of the session, while RTP is used to actually transmit media data. When a call is established, SDP is used to exchange session description information between the participants. After the session parameters are determined, RTP is used to transmit audio and video data during the call. The description information in SDP specifies RTP as the transmission protocol for media data.

[0031] Restore the content carried by RTP, and use the parameters in SDP in combination with the RTP protocol for parsing and processing;

[0032] Design a dynamic RTP-SDP mapping processing procedure: First, before the call begins, extract information related to media parameters from the SDP description information to construct the media stream payload type, construct the dynamic RTP-SDP mapping high-speed hash, and extract the encoder name, clock rate, and encoder parameters to initialize the corresponding decoder; then, when restoring each packet of RTP data, parse the RTP packet header payload type, match the corresponding decoder from the high-speed hash for decoding processing, and restore the media file of the corresponding format.

[0033] Step 6: Abnormal data frame detection;

[0034] Before extracting media frame data carried by RTP, the system first determines whether the frame data is corrupted based on the frame header attributes. If it is corrupted, the frame data is replaced with a silent frame. When the payload format is AMR or AMRWB, the complete payload includes a payload header, a payload directory, and voice data representing one or more speech frame blocks.

[0035] Step 7: Insert silence frames into the voice data;

[0036] During the voice reconstruction process, the transcoding program adds the corresponding silence data packets as needed;

[0037] After sorting the RTP information by Seq, calculate the relative timestamp of two adjacent RTPs, and divide the difference by the standard interval value to determine how many frames of silence data need to be added.

[0038] Step 8: Speech extraction and restoration;

[0039] Extract and restore the media data carried by RTP; the voice media data transmission formats include six types: AMR-NB, AMR-WB, EVS, PCMA, PCMU, and G729. Design the initialization, extraction, and closing interfaces for each type and encapsulate them into a unified media data extraction interface.

[0040] When processing streaming media files, the identified media type is used as a parameter to initialize the multimedia extractor. When extracting subsequent audio data, the extraction interface is called to extract audio data, obtaining one frame of data or multiple frames of data including silence frames each time. After all processing is completed, the extraction is turned off.

[0041] Preferably, the data packet header information includes a data packet type flag and a data packet length. A data packet header type flag of 1 indicates an RTP data packet, while a flag of 0 indicates an SDP data packet.

[0042] Preferably, the basic structure of the SDP data packet consists of a series of descriptive fields, each with specific semantics, as follows:

[0043] v-version: Specifies the version of the SDP description information;

[0044] o-Owner / Creator: Describes the owner or creator of the session; it is an identifier and a timestamp of the session's creation.

[0045] s-session name: Specifies the name or title of the session;

[0046] t-time: Defines the valid time range of the session;

[0047] m-media: Describes a media stream, including media type, transport address, and port;

[0048] c-connection: Specifies the connection information for the media stream, including the network protocol and address;

[0049] a-Attributes: Define various attributes of a session or media stream, including codec information, bandwidth requirements, and flow control;

[0050] A complete SDP description contains multiple media streams, each described using independent m, c, and a fields. SDP descriptions are exchanged during session establishment via SIP or other protocols. The sender sends its described session parameters to the receiver, which uses this information to configure its local session environment for consistent multimedia communication. Information related to media parameters in the SDP description constructs the payload type of the media stream and hashes of the encoder name, clock rate, and encoder parameters. The SDP parsing module extracts the correspondence between media type identifier values ​​and the carrying media types from the lines "a=rtpmap" and "a=fmtp" to identify the media mapping relationship.

[0051] Preferably, the payload type includes an audio codec or a video codec.

[0052] Preferably, the session description information includes media type, codec, and network address.

[0053] Preferably, in step 7:

[0054] A frame lasts for 20 milliseconds. The timestamp interval between two consecutive frames is 160. If the difference is n times 160, n-1 silent frames are inserted.

[0055] For the AMR-NB format, a complete payload includes a payload header, a payload directory, and speech data representing one or more speech frame blocks;

[0056] In bandwidth-efficient mode, the payload header consists of only a 4-bit codec mode request: CMR: indicates the codec mode request of the speech encoder sent to this payload receiver site; the value of the CMR field is set to the frame type index of the requested corresponding speech mode, the frame type index of AMR is 0-7, and the frame type index of AMR-WB is 0-8; the payload directory consists of a list of payload entries, each entry representing a speech frame.

[0057] In bandwidth efficiency mode, payload entries are formatted as follows: F, 1 bit: If set to 1, it indicates that this frame is followed by another voice frame in this payload; if set to 0, it indicates that this frame is the last frame in this payload; FT, 4 bits: Frame type index, indicating the AMR or AMR-WB voice coding mode or comfort noise SID mode of the corresponding frame carried in this payload; Q, 1 bit: Frame quality indicator. If set to 0, it indicates that the corresponding frame is severely corrupted, and the receiver should set RX_type to SPEECH_BAD or SID_BAD depending on the frame type.

[0058] The beneficial effects of this invention are as follows:

[0059] This invention designs an offline media voice stream parsing algorithm, an RTP-SDP dynamic mapping algorithm, and a voice extraction and restoration algorithm. The media voice stream parsing algorithm first separates the SDP and RTP data packets from the media voice stream and parses the SDP and RTP information according to the protocol. The RTP-SDP dynamic mapping algorithm stores the encoder name, clock rate, and encoder parameters parsed from the SDP into a hash table. When processing each packet of RTP data, it automatically matches the corresponding decoder from the high-speed hash table, thus achieving adaptive matching capability for various media data decoders at this stage. The voice extraction and restoration algorithm, for the matched decoder, extracts and restores the complete media format data of a session according to the media format transmission protocol, mainly solving the problem of unified restoration of various encoded voice data and increasing the universality of voice data restoration. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the data packet header information of the present invention.

[0061] Figure 2 This is a schematic diagram of the RTP header of the present invention.

[0062] Figure 3 This is a schematic diagram of the RTP-SDP dynamic mapping of the present invention.

[0063] Figure 4 The software flowchart for implementing this invention is shown below. Detailed Implementation

[0064] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0065] This invention relies on media stream data. For a complete session, it saves the media stream data to a proprietary format file according to the received time sequence for offline processing. First, the offline media data is parsed and categorized into SDP and RTP data. Then, the media type mapping relationship is parsed from the SDP description. Next, deduplication and global sorting operations are performed based on the timestamps and sequence numbers in the RTP data. Finally, the corresponding decoder is called based on the media type in the RTP data packet. Through data frame verification, replacement, and supplementation operations, the corresponding media format data is restored and output to the media file. For newly appearing undecoded media data, since an offline restoration method is used, the data will be restored again once a corresponding decoder is developed to ensure no data loss.

[0066] This invention is mainly aimed at media voice stream data generated during a session, from which various media format voice files are extracted and restored. It mainly includes offline media voice stream data packet classification, media mapping relationship identification, RTP data packet parsing, data packet reassembly and sorting, RTP-SDP dynamic mapping, abnormal data frame detection, insertion of silence frames into voice data, and voice extraction and restoration, which can be specifically described as the following 8 points.

[0067] 1) Classification of offline media voice stream data packets

[0068] The main task is to read offline media audio stream data files, parse the packet headers, and classify the packets into SDP or RTP packets based on the header information. Figure 1 As shown, the data packet header information includes the data packet type flag and the data packet length. A data packet header type flag of 1 indicates an RTP data packet, while a flag of 0 indicates an SDP data packet.

[0069] 2) Media mapping relationship identification

[0070] The main task is to extract the mapping relationship between the bearer type identifier value and the bearer media type name, as well as other parameters required for media transcoding, from the multi-line information of the SDP. The basic structure of the SDP description information consists of a series of description fields, each with specific semantics. Below are some common SDP description fields:

[0071] v(version): Specifies the version of the SDP description information.

[0072] o(Owner / Creator): Describes the owner or creator of the session, usually an identifier and a timestamp indicating the session's creation.

[0073] s(session name): Specifies the name or title of the session.

[0074] t(time): Defines the valid time range of the session.

[0075] m (media): Describes a media stream, including media type (e.g., audio, video), transport address, and port.

[0076] c(connection): Specifies the connection information for the media stream, including the network protocol and address.

[0077] a(attributes): Defines various attributes of a session or media stream, such as codec information, bandwidth requirements, flow control, etc.

[0078] A complete SDP description may contain multiple media streams, each described using independent m, c, and a fields. SDP descriptions are typically exchanged during session establishment via SIP (Session Initiation Protocol) or other protocols. The sender can send its described session parameters to the receiver, which uses this information to configure its local session environment for consistent multimedia communication. The media parameter-related information in the SDP description constructs hashes of the media stream's payload type, encoder name, clock rate, and encoder parameters, playing a crucial role in massive data processing and high-performance computing. The SDP parsing module extracts the correspondence between media type identifier values ​​and the carrying media types from the "a=rtpmap" and "a=fmtp" lines to identify media mapping relationships.

[0079] 3) RTP packet parsing

[0080] RTP divides data into a series of packets, each containing an RTP header and a payload. For example... Figure 2 As shown, the RTP header consists of 12 bytes and contains various control information, such as version number, payload type, sequence number, timestamp, and synchronization signals. RTP uses the Synchronization Source Identifier (SSRC) to maintain synchronization between different media streams. Each media stream has a unique SSRC identifier to distinguish between different senders and receivers. The receiver can use the SSRC to associate received data packets with a specific media stream to ensure synchronized playback of multiple media streams.

[0081] The steps for parsing RTP packets are as follows:

[0082] a. Separate the RTP header: Extract the first 12 bytes from the received data packet; this is the length of the RTP header. The RTP header contains various control information.

[0083] b. Parsing the RTP header: The structure of the RTP header is as follows:

[0084] - Version: 2 digits, used to indicate the version of the RTP protocol.

[0085] - Padding: 1 bit, used to indicate whether there are extra padding bytes.

[0086] - Extension bit: occupies 1 bit and is used to indicate whether there is an extension portion of the RTP header.

[0087] -CSRC Count: 4 bits, indicating the number of CSRC identifiers.

[0088] - Marker: occupies 1 bit and can be customized by the application.

[0089] - Payload Type: 7 bits, used to indicate the type of payload, such as audio codec or video codec.

[0090] - Sequence Number: 16 bits, used to identify the order of data packets.

[0091] - Timestamp: 32 bits, used to synchronize playback speed on the receiving end.

[0092] - Synchronization Source Identifier (SSRC): 32 bits, used to identify the sender.

[0093] -CSRC Identifiers (Contributing Source Identifiers): These occupy 32 bits and can be used to identify other sources participating in the synchronization.

[0094] c. Extract payload data: Extract the payload data portion based on the information in the RTP header. The payload data is the actual content of the media, such as audio or video.

[0095] d. Further processing of load data: The load data is parsed and processed according to the load type. Depending on the application scenario and load type, specific decoders or processing algorithms may be required to process the load data.

[0096] 4) Voice data packet reassembly and sorting

[0097] The received voice data frames may be out of order, so RTP packets need to be sorted before media parsing and reconstruction. The Seq field in the RTP header indicates the sequence number of the packet, and the internal sorting algorithm sorts each RTP packet in the list in ascending order based on this field. Since the maximum value of Seq is 65535, the sequence number may increase to the maximum value and then reset to 0 when the call duration is long, thus affecting the sorting algorithm. To solve this problem, a wraparound sorting algorithm was designed to ensure the order of RTP data.

[0098] 5) RTP-SDP dynamic mapping

[0099] SDP is used to describe session parameters, while RTP is used for the actual transmission of media data. When a call is established, SDP is used to exchange session description information between participants, including media type, codec, and network address. Once the session parameters are determined, RTP is used to transmit audio and video data during the call. The description information in SDP can specify RTP as the transport protocol for media data.

[0100] To reconstruct the content carried by RTP, parameters from SDP must be used in conjunction with the RTP protocol for appropriate parsing and processing. The RTP packet header contains fields describing the media type, the most important of which is the Payload Type field. The Payload Type field is located in the fixed header of the RTP packet and occupies an 8-bit field. It indicates the media type carried in the RTP packet, and the mapping between specific Payload Type values ​​and media types is usually determined by information carried by SDP.

[0101] Because media format changes may occur during transmission due to device switching, a dynamic RTP-SDP mapping handler needs to be designed. For example... Figure 3 As shown, in the implementation of RTP-SDP dynamic mapping technology, before the call starts, information related to media parameters in the SDP description information is extracted to construct the media stream payload type and the RTP-SDP dynamic mapping high-speed hash. The encoder name, clock rate and encoder parameters are extracted to initialize the corresponding decoder. Then, when restoring each packet of RTP data, the RTP packet header payload type is parsed, and the corresponding decoder is matched from the high-speed hash for decoding processing to restore the media file of the corresponding format.

[0102] 6) Abnormal data frame detection

[0103] Before extracting media frame data carried by RTP, the frame header attributes are used to determine if the frame data is corrupted. If corrupted, the frame data is replaced with a silent frame. When the payload format is AMR or AMRWB, the complete payload includes a payload header, payload directory, and speech data representing one or more speech frame blocks. F (1 bit): If set to 1, it indicates that this frame is followed by another speech frame in this payload; if set to 0, it indicates that this frame is the last frame in this payload. FT (4 bits): Frame type index, indicating the AMR or AMR-WB speech coding mode or comfort noise (SID) mode of the corresponding frame carried in this payload. The frame type index for AMR can be 0-7, and the frame type index for AMR-WB can be 0-8. If it exceeds the range, it is considered a silent frame. Q (1 bit): Frame quality indicator. If set to 0, it indicates that the corresponding frame is severely corrupted, and the speech in that frame is treated as a silent frame.

[0104] 7) Insert silence frames into voice data

[0105] During voice calls, there may be issues with discontinuous data transmission or data packet loss. If one party remains silent for an extended period, no data will be transmitted through the transmission channel. If only media data from the data packets is extracted, it may affect the call duration, causing the local and remote audio to become out of sync during playback. Therefore, during the voice reconstruction process, the transcoding program adds appropriate silence data packets as needed.

[0106] The following is an example of two consecutive RTP data streams:

[0107]

[0108] After sorting the RTP information by Seq, the relative timestamps of two adjacent RTPs are calculated. The difference is divided by the standard interval to determine how many frames of silence data need to be added. Normally, a frame is 20 milliseconds long, and the timestamp interval between two consecutive frames is 160 (the interval varies depending on the media format, generally 160 or 320). If the difference is n times 160, n-1 silence frames are inserted.

[0109] 8) Speech extraction and restoration

[0110] The main task is to extract and restore the media data carried by RTP. Currently, there are six main audio media data transmission formats on the network: AMR-NB, AMR-WB, EVS, PCMA, PCMU, and G729. The data extraction methods for each format are quite different. Therefore, it is necessary to design the initialization, extraction, and shutdown interfaces for each media extractor according to the relevant transmission protocols, and encapsulate them into a unified media data extraction interface.

[0111] Taking AMR-NB as an example, a complete payload includes a payload header, a payload directory, and voice data representing one or more speech frame blocks. In bandwidth-efficient mode, the payload header consists of only a 4-bit codec mode request: CMR (4 bits): Indicates the codec mode request of the speech encoder sent to this payload receiver site. The value of the CMR field is set to the frame type index of the requested corresponding speech mode; the frame type index is 0-7 for AMR and 0-8 for AMR-WB. The payload directory consists of a list of payload entries, each representing a speech frame. In bandwidth-efficient mode, payload entries are formatted as follows: F (1 bit): If set to 1, it indicates that this frame is followed by another speech frame in this payload; if set to 0, it indicates that this frame is the last frame in this payload. FT (4 bits): Frame type index, indicating the AMR or AMR-WB speech coding mode or comfort noise (SID) mode of the corresponding frame carried in this payload. Q (1 bit): Frame quality indicator. If set to 0, it indicates that the corresponding frame is severely corrupted, and the receiver should set the RX_ type to SPEECH_BAD or SID_BAD according to the frame type (FT).

[0112] When processing a streaming media file, the identified media type is used as a parameter to initialize the multimedia extractor. When extracting subsequent audio data, the extraction interface is called to extract the audio data, obtaining one frame of data or multiple frames of data including silence frames each time. After all processing is completed, the extraction is turned off.

[0113] The method of this invention can be implemented by... Figure 4 The software flowchart is used.

Claims

1. A method for restoring offline media audio streams, characterized in that, Includes the following steps: Step 1: Classify offline media voice stream data packets; Read offline media audio stream data files, parse the data packet headers, and classify the data packets into SDP data packets or RTP data packets based on the data packet header information; Step 2: Media mapping relationship identification; Read multiple lines of media information from the SDP data packet obtained in step 1, and extract the mapping relationship between the bearer type identifier value and the bearer media type name, as well as other parameters required for media transcoding; Step 3: RTP packet parsing; Each RTP packet contains an RTP header and a payload; the RTP header consists of 12 bytes and contains various control information; RTP maintains synchronization between different media streams through synchronization signals, and each media stream has a unique SSRC identifier to distinguish different senders and receivers; the receiver uses SSRC to associate received packets with specific media streams to ensure synchronized playback of multiple media streams. The steps for parsing RTP packets are as follows: Step 3-1: Separate the RTP header: Extract the first 12 bytes from the received data packet, i.e., the RTP header; Step 3-2: Parse the RTP header. The structure of the RTP header is as follows: - Version: 2 digits, used to indicate the version of the RTP protocol; - Padding bit: occupies 1 bit and is used to indicate whether there are additional padding bytes; - Extension bit: occupies 1 bit and is used to indicate whether there is an extension portion of the RTP header; -CSRC Count: 4 bits, indicates the number of CSRC identifiers; - Flag bit: occupies 1 bit and is defined by the application. - Load type: 7 bits, used to indicate the load type; - Sequence number: 16 bits, used to identify the order of data packets; - Timestamp: 32 bits, used to synchronize playback speed on the receiving end; - Synchronization signal: 32 bits, used to identify the sender; -CSRC identifier: 32 bits, used to identify other sources participating in synchronization; Step 3-3: Extract payload data: Extract the payload data portion based on the information in the RTP header; the payload data is the actual content of the media. Step 4: Reassembling and sorting voice data packets Sort the RTP packets; the Seq in the RTP header information indicates the sequence number of the packet, and sort each RTP in the list in ascending order according to the field; Step 5: Dynamic mapping of RTP-SDP; SDP is used to describe the parameters of the session, while RTP is used to actually transmit media data; when a call is established, SDP is used to exchange session description information between the participants. Once the session parameters are determined, RTP is used to transmit audio and video data during the call; the description information in the SDP specifies RTP as the transport protocol for media data. Restore the content carried by RTP, and use the parameters in SDP in combination with the RTP protocol for parsing and processing; Design a dynamic RTP-SDP mapping process: First, extract information related to media parameters from the SDP description information before the call starts, construct the payload type of the media stream, construct the dynamic RTP-SDP mapping high-speed hash, and extract the encoder name, clock rate, and encoder parameters to initialize the corresponding decoder. Then, when restoring each packet of RTP data, the RTP packet header payload type is parsed, and the corresponding decoder is matched from the high-speed hash for decoding processing to restore the media file of the corresponding format. Step 6: Abnormal data frame detection; Before extracting media frame data carried by RTP, the system first determines whether the frame data is corrupted based on the frame header attributes. If it is corrupted, the frame data is replaced with a silent frame. When the payload format is AMR or AMRWB, the complete payload includes a payload header, a payload directory, and voice data representing one or more speech frame blocks. Step 7: Insert silence frames into the voice data; During the voice reconstruction process, the transcoding program adds the corresponding silence data packets as needed; After sorting the RTP information by Seq, calculate the relative timestamp of two adjacent RTPs, and divide the difference by the standard interval value to determine how many frames of silence data need to be added. Step 8: Speech extraction and restoration; Extract and restore the media data carried by RTP; the voice media data transmission formats include six types: AMR-NB, AMR-WB, EVS, PCMA, PCMU, and G729. Design the initialization, extraction, and closing interfaces for each type and encapsulate them into a unified media data extraction interface. When processing streaming media files, the identified media type is used as a parameter to initialize the multimedia extractor. When extracting subsequent audio data, the extraction interface is called to extract audio data, obtaining one frame of data or multiple frames of data including silence frames each time. After all processing is completed, the extraction is turned off.

2. The method for restoring offline media audio streams according to claim 1, characterized in that, The data packet header information includes a data packet type flag and a data packet length. A data packet type flag of 1 indicates an RTP data packet, while a data packet of 0 indicates an SDP data packet.

3. The method for restoring offline media audio streams according to claim 1, characterized in that, The basic structure of the SDP data packet consists of a series of descriptive fields, each with specific semantics, as follows: v-version: Specifies the version of the SDP description information; o-Owner / Creator: Describes the owner or creator of the session; it is an identifier and a timestamp of the session's creation. s-session name: Specifies the name or title of the session; t-time: Defines the valid time range of the session; m-media: Describes a media stream, including media type, transport address, and port; c-connection: Specifies the connection information for the media stream, including the network protocol and address; a-Attributes: Define various attributes of a session or media stream, including codec information, bandwidth requirements, and flow control; A complete SDP description contains multiple media streams, each described using independent m, c, and a fields. SDP descriptions are exchanged during session establishment via SIP or other protocols. The sender sends its described session parameters to the receiver, which uses this information to configure its local session environment for consistent multimedia communication. Information related to media parameters in the SDP description constructs the payload type of the media stream and hashes of the encoder name, clock rate, and encoder parameters. The SDP parsing module extracts the correspondence between media type identifier values ​​and the carrying media types from the "a=rtpmap" and "a=fmtp" lines to identify the media mapping relationship.

4. The method for restoring offline media audio streams according to claim 1, characterized in that, The payload type includes audio codecs or video codecs.

5. The method for restoring offline media audio streams according to claim 1, characterized in that, The session description information includes media type, codec, and network address.

6. The method for restoring offline media audio streams according to claim 1, characterized in that, In step 7: A frame lasts for 20 milliseconds. The timestamp interval between two consecutive frames is 160. If the difference is n times 160, n-1 silent frames are inserted. For the AMR-NB format, a complete payload includes a payload header, a payload directory, and speech data representing one or more speech frame blocks; In bandwidth-efficient mode, the payload header consists of only a 4-bit codec mode request: CMR: indicates the codec mode request of the speech encoder sent to this payload receiver site; the value of the CMR field is set to the frame type index of the requested corresponding speech mode, the frame type index of AMR is 0-7, and the frame type index of AMR-WB is 0-8; the payload directory consists of a list of payload entries, each entry representing a speech frame. In bandwidth efficiency mode, the payload entry is formatted as follows: F, 1 bit: If set to 1, it indicates that this frame is followed by another voice frame in this payload; if set to 0, it indicates that this frame is the last frame in this payload; FT, 4 bits: Frame type index, indicating the AMR or AMR-WB voice coding mode or comfort noise SID mode of the corresponding frame carried in this payload; Q, 1 bit: Frame quality indicator; if set to 0, it indicates that the corresponding frame is severely corrupted, and the receiver should set RX_type to SPEECH_BAD or SID_BAD depending on the frame type.

Images