Multimedia data stream transmission method, device, computer equipment and storage medium

Abstract

The present application relates to the field of image data processing technology, and specifically, to a multimedia data stream transmission method, device, computer equipment and storage medium. The multimedia data stream transmission method includes: encapsulating the program stream information of the multimedia data to be processed to obtain a program mapping table data structure; encoding the multimedia data to be processed; configuring the basic code stream data according to a preset code stream standard format; dividing and packaging the program mapping table data structure, the basic code stream data and the code stream format information according to the preset configured data transmission stream format to form a multimedia stream data packet; parsing the multimedia stream data according to the program mapping table data structure, the basic code stream data and the code stream format information to obtain the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data. A multimedia data stream transmission method that can carry complete standard information and can achieve correct decoding is provided.

Description

Technical Field

[0001] The present application relates to the technical field of image data processing, and in particular, to a multimedia data stream transmission method, apparatus, computer equipment and storage medium. Background Art

[0002] MPEG-TS (a standard data container format for transmitting and storing audio, video, program and system information protocol data, used in digital broadcasting systems, hereinafter referred to as TS) is one of the most widely used and longest-standing real-time media data formats. It is widely used in multimedia systems and devices for transmitting multimedia data streams. However, with the development of many new multimedia encoding and decoding technologies, TS has been difficult to meet actual technical requirements. For example, TS needs to use an 8-bit field to describe the stream type in the stream information of PMT (Program Map Table, program map table data structure). With the rapid development of multimedia technology and the occupation of its segments by branch standards in different countries and regions, its definition method can no longer meet the needs of new media stream types. In order to reduce resource usage, some encoders do not carry complete standard information in their stream data structure, and need to be encapsulated and described in the media stream. If the TS format is used directly for transmission, the receiving end will not be able to configure and decode correctly. These problems hinder the further development of multimedia streaming technology.

[0003] Therefore, there is an urgent need for a multimedia data stream transmission method that can carry complete format information and achieve correct decoding. Summary of the invention

[0004] The embodiments of the present application provide a multimedia data stream transmission method, apparatus, computer equipment and storage medium.

[0005] A first aspect of an embodiment of the present application provides a multimedia data stream transmission method, including:

[0006] Encapsulating the program stream information of the multimedia data to be processed to obtain a program mapping table data structure; wherein the program stream information includes: at least one of the number of code streams, code stream numbers and code stream types of the multimedia data to be processed;

[0007] Encoding the multimedia data to be processed to form basic code stream data;

[0008] The basic code stream data is configured according to a preset code stream standard format to obtain the code stream format information;

[0009] Divide and package the program mapping table data structure, the basic code stream data and the code stream format information according to a preset data transmission stream format to form a multimedia stream data packet;

[0010] The multimedia stream data is parsed according to the program mapping table data structure, the basic code stream data and the code stream format information to obtain original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data.

[0011] In an optional embodiment of the present application, encapsulating the multimedia data to be processed with program stream information to obtain the program mapping table data structure includes:

[0012] For each channel of code stream data of the to-be-processed multimedia data, if the code stream type in the program stream information is empty, configure the code stream type of the code stream data according to a pre-configured registration descriptor format to obtain a code stream descriptor of the code stream data;

[0013] The code stream number, the code stream type and the code stream descriptor are configured into an initial program mapping table data structure to obtain the program mapping table data structure.

[0014] In an optional embodiment of the present application, configuring the basic code stream data according to a preset code stream format to obtain the code stream format information includes:

[0015] Generating multimedia standard information of the basic code stream data according to the format data of the multimedia data to be processed;

[0016] The multimedia standard information, the timestamp of the basic code stream data and the data length of the basic code stream data are encapsulated into the code stream format information.

[0017] In an optional embodiment of the present application, the program mapping table data structure, the basic code stream data and the code stream format information are divided, packaged and encapsulated according to a preset data transmission stream format to form a multimedia stream data packet, including:

[0018] Packing the program mapping table data structure and the basic code stream data into a load part of a preset multimedia stream data packet;

[0019] The code stream format information is packaged into a packet header part of a preset multimedia stream data packet, wherein the packet header part includes an expandable private data field.

[0020] In an optional embodiment of the present application, the multimedia stream data is parsed according to the program mapping table data structure, the basic code stream data and the code stream format information to obtain the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data, including:

[0021] Parsing the program mapping table data structure in the multimedia stream data packet, reading the table items in the program mapping table data structure, and obtaining the code stream type and media stream identifier of each code stream data;

[0022] The code stream data of each channel in the media stream data packet is extracted by using the media stream identifier and the code stream type, so as to obtain the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data.

[0023] In an optional embodiment of the present application, before extracting each stream data in the media stream data packet by using the media stream identifier and the stream type to obtain the original stream data corresponding to the basic stream data and the stream format information of the basic stream data, the method further includes:

[0024] Extracting the code stream type and the media stream identifier in the program map table data structure;

[0025] A stream type table is constructed based on the stream type and the media stream identifier; wherein the stream type table includes the stream type of each stream data and the stream format information corresponding to the stream data.

[0026] In an optional embodiment of the present application, extracting each stream data in the media stream data packet by using the media stream identifier and the stream type to obtain the original stream data corresponding to the basic stream data and the stream format information of the basic stream data includes:

[0027] Based on the code stream type table, the TS data packets in each code stream data are parsed to obtain the private data of the private data field of the TS packet header and the TS payload data;

[0028] Reorganize the private data according to a pre-configured order to generate the code stream format information of the basic code stream data;

[0029] The TS payload data is reorganized to form the original code stream data corresponding to the basic code stream data.

[0030] A second aspect of an embodiment of the present application provides a multimedia data stream transmission device, including:

[0031] A first encapsulation module is used to encapsulate program stream information of the multimedia data to be processed to obtain a program mapping table data structure; wherein the program stream information includes: at least one of the number of code streams, code stream numbers and code stream types of the multimedia data to be processed;

[0032] An encoding module, used for encoding the multimedia data to be processed to form basic code stream data;

[0033] A configuration module, used to configure the basic code stream data according to a preset code stream format to obtain the code stream format information;

[0034] A second encapsulation module is used to divide, package and encapsulate the program mapping table data structure, the basic code stream data and the code stream format information according to a preset data transmission stream format to form a multimedia stream data packet;

[0035] The parsing module is used to parse the multimedia stream data according to the program mapping table data structure, the basic code stream data and the code stream format information to obtain the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data.

[0036] According to a third aspect of an embodiment of the present application, a computer device is provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0037] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0038] The embodiment of the present application provides a multimedia data stream transmission method, which encapsulates the program stream information of the multimedia data to be processed to obtain a program mapping table data structure; encodes the multimedia data to be processed to form basic code stream data; configures the basic code stream data according to a preset code stream format to obtain the code stream format information; divides, packages and encapsulates the program mapping table data structure, the basic code stream data and the code stream format information according to the preset data transmission stream format to form a multimedia stream data packet; standardizes the basic code stream data through the code stream format information in the multimedia stream data packet, and carries complete standard information through the code stream format information, including but not limited to the basic code stream data of the multimedia data to be processed and other description information, so as to facilitate the decoding stage of the multimedia stream data according to the program mapping table data structure, the basic code stream data and the code stream format information to obtain the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data, so as to realize the correct decoding of the multimedia data stream. In summary, the embodiment of the present application provides a multimedia data stream transmission method that can carry complete standard information and can realize correct decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 A flowchart of a multimedia data stream transmission method provided by an embodiment of the present application;

[0041] Figure 2 A flowchart of interaction between an encoding end and a decoding end in a multimedia data stream transmission method provided by an embodiment of the present application;

[0042] Figure 3 A flowchart of a multimedia data stream transmission method provided by an embodiment of the present application;

[0043] Figure 4 An interactive flow chart of constructing a program mapping table data structure in a multimedia data stream transmission method provided in one embodiment of the present application;

[0044] Figure 5 A flowchart of a multimedia data stream transmission method provided by an embodiment of the present application;

[0045] Figure 6 An interactive flow chart of generating code stream format information in a multimedia data stream transmission method provided in one embodiment of the present application;

[0046] Figure 7 A flowchart of a multimedia data stream transmission method provided by an embodiment of the present application;

[0047] Figure 8 A schematic diagram of the encapsulation structure of a multimedia stream data packet in a multimedia data stream transmission method provided in one embodiment of the present application;

[0048] Fig. 9 A flowchart of a multimedia data stream transmission method provided by an embodiment of the present application;

[0049] Fig.10 An interactive flow chart of constructing a stream type table in a multimedia data stream transmission method provided in one embodiment of the present application;

[0050] Fig.11 A flowchart of a multimedia data stream transmission method provided by an embodiment of the present application;

[0051] Fig.12 An interactive flow chart for determining original bitstream data and bitstream format information in a multimedia data stream transmission method provided by an embodiment of the present application;

[0052] Fig.13A schematic diagram of the structure of a multimedia data stream transmission device provided in one embodiment of the present application;

[0053] Fig.14 A schematic diagram of the computer device structure provided for one embodiment of the present application. DETAILED DESCRIPTION

[0054] In the process of implementing the present application, the inventors discovered that there is an urgent need for a multimedia data stream transmission method that can carry complete format information and can achieve correct decoding.

[0055] To solve the above problems, the embodiments of the present application provide a multimedia data stream transmission method, apparatus, computer equipment and storage medium. The solutions in the embodiments of the present application can be implemented in various computer languages, such as object-oriented programming language Java and interpreted scripting language JavaScript.

[0056] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0057] The following multimedia data stream transmission method provided in the embodiment of the present application is applied to a multimedia data stream transmission system, which includes an encoding end and a decoding end, wherein the encoding end refers to a device or terminal including an encoder for encoding and encapsulating multimedia data, and the decoding end refers to a device or terminal including a decoder for decoding multimedia data to obtain original code stream data. The encoding end and the decoding end can be configured in the same terminal or electronic device, or respectively configured in different terminals or electronic devices, and the terminal or electronic device can be a computer, a server, a laptop, a numerical control device, a wearable device, etc. The embodiment of the present application does not make any limitation and can be flexibly adjusted according to actual conditions.

[0058] See also Figure 1 and Figure 2 The following embodiments take the above-mentioned multimedia data stream transmission system as the execution subject, and apply the multimedia data stream transmission method provided in the embodiment of the present application to the above-mentioned multimedia data stream transmission system to perform multimedia data stream transmission as an example for specific description. The multimedia data stream transmission method provided in the embodiment of the present application includes the following steps 101 to 105:

[0059] Step 101: The encoding end encapsulates program stream information of the multimedia data to be processed to obtain a program mapping table data structure;

[0060] The program stream information includes: at least one of the number of code streams, code stream numbers and code stream types of the multimedia data to be processed; the code stream data refers to the number of code stream data, such as how many channels of data there are, the code stream numbers can be flexibly adjusted according to actual conditions, for example, they are numbered 1, 2, 3, 4, etc., and the code stream types can be flexibly configured according to actual conditions, for example, including audio code streams, video code streams, image code streams, etc.

[0061] The program mapping table data structure in the embodiment of the present application refers to the PMT (Program Map Table, program mapping table data structure) table. The embodiment of the present application encapsulates the program stream information of the multimedia data to be processed, and can be serialized according to the data structure defined in Table 2-41 of ISO / IEC 13818-1 2023 (not shown in this application), wherein the PMT includes a fixed header field and multiple stream type description table items, and each stream data corresponds to a table item, and each table item mainly includes an 8-bit encoding type, an 8-bit stream number, a 12-bit descriptor length and a descriptor content. Of course, this is only an example, and does not constitute a limitation on the process and form of forming the program mapping table data structure.

[0062] The embodiment of the present application encapsulates program stream information by generating a program mapping table data structure PMT, where the program stream information includes: at least one of the number of code streams, code stream numbers and code stream types of the multimedia data to be processed. In the subsequent decoding process, the basic code stream data in the multimedia data to be processed can be correctly identified and extracted by parsing the program mapping table data structure PMT.

[0063] Step 102: The encoding end encodes the multimedia data to be processed to form basic code stream data;

[0064] The encoding end encodes the multimedia data to be processed, and encapsulates the encoded stream to form basic code stream data. The basic code stream data refers to the code stream data after the multimedia data to be processed is encoded, and contains various basic information of the multimedia data, such as: the original code stream data output by the encoder and the decoding timestamp and other information. It can be organized according to the data syntax structure defined in Table 2-21 of "ISO / IEC13818-1 2023", and it is necessary to record the complete basic code stream data PES length information in the code stream format information. The code stream number of the basic code stream data PES can also be indexed to the corresponding table item in the program mapping table data structure PMT. The basic code stream data in the embodiment of the present application refers to PES (Packetized Elementary Streams, packaged basic code stream), which can be a PES stream (digital television) audio, video and digital signal. After data compression by encoders such as MPEG-2, the basic code stream is formed, such as ES stream, and the ES stream is then packaged to form a code stream with a packet header.

[0065] Step 103: The encoding end configures the basic bitstream data according to a preset bitstream format to obtain the bitstream format information;

[0066] Generate code stream format information, and encapsulate the format and code stream description information of the multimedia data to be processed through the code stream format information. The preset code stream format includes information such as image resolution, image color space, image block encoding method, frame rate, audio sampling rate, audio sampling bit number, etc. The code stream format information can correctly handle decoding and rendering in the subsequent decoding process. The preset code stream format can be, for example, a binary data format serialization format. The embodiment of the present application does not impose any limitation on the specific form of the preset code stream format, and can be flexibly adjusted according to actual conditions, and only needs to facilitate the purpose of storing and transmitting code stream data in a fixed format.

[0067] Step 104: The encoding end divides, packages and encapsulates the program mapping table data structure, the basic code stream data and the code stream format information according to the preset data transmission stream format to form a multimedia stream data packet;

[0068] The data transmission stream format can be flexibly adjusted according to actual conditions. For example, it can be packaged according to the TS packet structure. The embodiment of the present application does not impose any limitations, and only needs to satisfy the function of synchronously storing and transmitting the program mapping table data structure, the basic code stream data and the code stream format information.

[0069] Step 105: The decoding end parses the multimedia stream data according to the program mapping table data structure, the basic code stream data and the code stream format information to obtain the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data.

[0070] The embodiment of the present application provides a multimedia data stream transmission method, which encapsulates the program stream information of the multimedia data to be processed to obtain a program mapping table data structure; encodes the multimedia data to be processed to form basic code stream data; configures the basic code stream data according to a preset code stream format to obtain the code stream format information; divides, packages and encapsulates the program mapping table data structure, the basic code stream data and the code stream format information according to the preset data transmission stream format to form a multimedia stream data packet; standardizes the basic code stream data through the code stream format information in the multimedia stream data packet, and carries complete standard information through the code stream format information, including but not limited to the basic code stream data of the multimedia data to be processed and other description information, so as to facilitate the decoding stage of the multimedia stream data according to the program mapping table data structure, the basic code stream data and the code stream format information to obtain the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data, so as to realize the correct decoding of the multimedia data stream. In summary, the embodiment of the present application provides a multimedia data stream transmission method that can carry complete standard information and can realize correct decoding.

[0071] See also Figure 3 and Figure 4 In an optional embodiment of the present application, the above step 101, the encoding end encapsulates the program stream information of the multimedia data to be processed to obtain the program mapping table data structure, includes the following steps 301-302:

[0072] Step 301: for each bitstream data of the to-be-processed multimedia data, if the bitstream type in the program stream information is empty, configure the bitstream type of the bitstream data according to a pre-configured registration descriptor format to obtain a bitstream descriptor of the bitstream data;

[0073] See also Figure 4, read one channel of stream data. Usually, one channel of program contains multiple channels of audio stream data and video stream data. It is necessary to record the stream number and stream type of each channel of audio stream data and video stream data into the program mapping table data structure PMT. If the stream type already exists, go directly to the next step 302; if the stream type does not exist, determine the stream descriptor according to this step to characterize the stream type through the stream descriptor. At the same time, the stream type field in the program mapping table data structure PMT table item can be fixedly set to 254. The pre-configured registration descriptor format in the embodiment of the present application is configured according to the registration descriptor format of "ISO / IEC 13818-1 2023" (not shown in this application), and the registration descriptor data syntax structure can be as shown in Table 1:

[0074] Table 1

[0075]

[0076] Among them, in Table 1: the registration stream descriptor type (descriptor_tag) can be configured to 5, the content length (descriptor_length) can be configured to 4, and the identifier (descriptor_length) can be configured to the stream type represented by the four-byte code, for example, the value of 'suvc' can be set to 0x73757663. Of course, this is only an example and does not constitute a limitation on the registration descriptor format and the specific content of the stream descriptor. The registration descriptor format and the stream descriptor can be flexibly adjusted and set according to actual conditions, and the embodiments of this application do not impose any limitations.

[0077] At the same time, the stream type parameter value can also be set, which is determined by the execution results of other parameters such as the stream descriptor and the stream type.

[0078] Step 302: configure the code stream number, the code stream type and the code stream descriptor into an initial program mapping table data structure to obtain the program mapping table data structure.

[0079] If the code stream type exists, there is no need to determine the code stream descriptor, and the program map table data structure PMT table item does not need to carry the registered code stream descriptor. The data syntax structure of the program map table data structure PMT table item is shown in Table 2 below:

[0080] Table 2

[0081]

[0082] In Table 2, the stream type (stream_type); stream number (elementary_PID); stream descriptor content length (ES_info_length) are in bytes. Each table entry can contain multiple descriptors (descriptors), and each descriptor contains a fixed 8-bit flag and 8-bit length field.

[0083] After the table entry data is generated, a program mapping table data structure PMT header is generated, and data such as the byte length of the table entry is recorded in the program mapping table data structure PMT header. Then, the header and table entries of the program mapping table data structure PMT are cached, and can be directly reused when the multimedia stream data packets are packaged multiple times in the future.

[0084] In the embodiment of the present application, for each stream data of the multimedia data to be processed, if the stream type in the program stream information is empty, the stream type of the stream data is configured according to the pre-configured registration descriptor format to obtain the stream descriptor of the stream data. The stream descriptor of the stream data can quickly expand the MPEG-TS stream content format, and there is no need to perform complex type standard definition registration when adding stream types, and no additional channel description media stream format information is required. At the same time, it is compatible with the existing transmission system and transmission equipment of the multimedia stream data packet TS, adapts to different multimedia stream technologies and encoding and decoding technologies, and uses the reserved private data segment in the adaptive data of the multimedia stream data packet TS to carry the stream format information; further, the embodiment of the present application configures the stream number, the stream type and the stream descriptor to the initial program mapping table data structure to obtain the program mapping table data structure, and can use the extensible stream descriptor in the stream information of the program mapping table data structure PMT to carry any media stream type format, thereby achieving the purpose of format expansion and flexibly carrying more format information.

[0085] See also Figure 5 and Figure 6 In an optional embodiment of the present application, in the above step 103, the encoding end configures the basic bitstream data according to a preset bitstream format to obtain the bitstream format information, including steps 501-502:

[0086] Step 501: Generate multimedia format information of the basic code stream data according to the format data of the multimedia data to be processed;

[0087] Step 502: Encapsulate the multimedia standard information, the timestamp of the basic code stream data, and the data length of the basic code stream data into the code stream format information.

[0088] See also Figure 6, the multimedia data to be processed may include: video data and audio data, and correspondingly, the multimedia format information includes video format information and audio format information. The bitstream format information in the embodiment of the present application may include video format size, frame type (key frame, reference frame), audio sampling rate, number of channels, time code, camera parameters and other information, which are mainly carried by the encoder in the output bitstream in the TS stream defined in "ISO / IEC 13818-1 2023", but not all encoding formats support the encapsulation of these key information. The embodiment of the present application describes these bitstream format information independently in the same format (according to the preset bitstream format), and by using the private data segment of the TS packet to carry it, it can adapt to more bitstream types. In this case, the bitstream output by the encoder is no longer required to contain format information, and the decoder does not need to adapt to different bitstream designs. It can parse the format information in a unified format and configure the decoder, which is more efficient and has a wider range of applications.

[0089] Taking the above-mentioned multimedia stream data to be processed including audio data and video data as an example, the specific process of generating the code stream format information can be found in Figure 6 First, according to the format data of video and audio, the video standard information and audio standard information in the multimedia standard information are generated, and then the multimedia standard information, the timestamp output by the encoder and the PES length of the basic code stream data are encapsulated together as the code stream format information. The data syntax structure of the code stream format information can be shown in Table 3, the data syntax structure of the video standard information is shown in Table 4 below, and the data syntax structure of the audio standard information is shown in Table 5 below.

[0090] Table 3

[0091]

[0092] In Table 3, the data syntax structure version (version) is used to indicate the version number of the bitstream format information content. When adding new format information, the version number needs to be changed, and it is currently fixed to 1; the bitstream format information type (type), when type is equal to 1, it indicates video format information, and when type is equal to 2, it indicates audio format information; the format information data block number (index), the size of each TS is fixed to 188, if the length of the format information content is greater than a single TS, it is split into multiple data segments for storage, and each data segment is assigned a continuous number, and the flag of the last segment (last_flag) must be set to 1; the reserved field (reserved) is used for subsequent expansion; the encoding class Type four-byte code (fourcc), the encoding type represented by the big-endian four-byte code, for example, 'suvc' should be set to 0x73757663; frame number (frame_number), a frame of video or audio encoded data has a number, which is accumulated by 1 for each frame; basic code stream data PES length (PES_length), the complete basic code stream data PES length, used for subsequent decapsulation verification, the maximum value is 4294967295, which can meet the PES length description requirements of any frame of basic code stream data; clock count (tick), video is counted at a frequency of 90000Hz, and audio is counted at a sampling rate; timestamp (time) records the generation time of the video and audio frames in microseconds.

[0093] Table 4

[0094]

[0095] In Table 4, the frame rate is expressed as a rational number, and the numerator (frame_rate_num) and the denominator (frame_rate_den) are both 32-bit unsigned integers. For example, for 49.97fps, frame_rate_num is equal to 50000, and frame_rate_den is equal to 1001; the video resolution width (width) and height (height) represent the image pixel width and height; the image display width (sar_width) and display height (sar_height) represent the aspect ratio; the image chroma sampling format (chroma_format), 0 represents grayscale, 1 represents 4:2:0 sampling, 2 represents 4:2:2 sampling, and 3 represents 4:4:4 sampling; the single sample data bit depth (bit_depth) ranges from 8 to 16; the code stream encoding type (frame_type), 1 represents key frame, 2 represents reference frame; the video interlaced attribute (frame_field), 0 represents the video is interlaced row, 1 indicates the top layer of the video, 2 indicates the bottom layer of the video; image rotation angle (ration), 0 indicates no rotation, 1 indicates the image is rotated 90 degrees clockwise, 2 indicates the image is rotated 180 degrees clockwise, and 3 indicates the image is rotated 270 degrees clockwise; color space (color_space), including the RGB / YUV conversion matrix of the video, and the sampling range configuration parameters; block encoding type (slice_mode), 0 indicates that the encoder is not divided into multiple blocks, 1 indicates that there are multiple blocks, and multiple blocks of the same frame code stream can be transmitted in multiple basic code stream data PESs. Each basic code stream data PES needs to be transmitted together with the format information; the total number of block encodings (slice_total), indicates the total number of blocks divided by the encoder for the current frame; the sum of the number of blocks (slice_count) and the starting block number (slice_index) carried in the current basic code stream data PES must be less than or equal to the total number of block encodings, and the blocks carried in each PES should be continuous.

[0096] Table 5

[0097]

[0098] In Table 5, the sampling rate (sample_rate) indicates the sampling frequency of the audio LPCM data; the sampling type (bit_depth) indicates how many bits of data each sample occupies, 8 indicates an 8-bit integer, 16 indicates a 16-bit integer, 24 indicates a 24-bit integer, and 32 indicates a 32-bit floating point; the audio channel information (channels) indicates the number of channels contained in the audio and the channel spatial layout configuration, 1 indicates mono, and 2 indicates stereo.

[0099] See also Figure 7 and Figure 8In an optional embodiment of the present application, in the above step 104, the encoding end divides, packages and encapsulates the program mapping table data structure, the basic code stream data and the code stream format information according to the preset data transmission stream format to form a multimedia stream data packet, including the following steps 701-702:

[0100] Step 701: Pack the program mapping table data structure and the basic code stream data into the payload part of a preset multimedia stream data packet;

[0101] Step 702: Pack the code stream format information into a header portion of a preset multimedia stream data packet, wherein the header portion includes an expandable private data field.

[0102] See also Figure 8 , taking the data syntax structure organization of the multimedia stream data packet TS as an example, each TS packet has a fixed length of 188 bytes, which is divided into a header part and a payload part. The payload part is used to store the basic code stream data PES. The TS header defines an extensible private data field for extending and identifying the multimedia stream data packet TS. The code stream format information must be stored in this area to facilitate the expansion of the code stream format. Each frame of the code stream or each code stream block output by the encoder corresponds to a format information, which is packaged into the TS header part. The data start field of the TS header part must be set to 1. If it cannot be completely encapsulated into a TS header, the remaining data is encapsulated into the next TS header part. After the format information is encapsulated, the remaining space of the TS packet is filled with empty data, or it can be filled with basic code stream data PES. Finally, the remaining basic code stream data PES is encapsulated into multiple TS packets, and the packet sequence number is guaranteed to be continuous. The private data in the TS header can be organized as described in Appendix D of DVB-ETSI TS 101 154 V2.6.1 (not shown in this application). Each private data field definition must contain an 8-bit type, an 8-bit data length and data content. The private data type value corresponding to the code stream format information is fixed to 192.

[0103] The program mapping table data structure PMT data is encapsulated separately into the payload part of the TS packet and is periodically packaged into the TS stream so that the decoder can randomly access it during transmission. The TS packet header contains the data stream number (PID). The program mapping table data structure PMT and each audio and video stream have a unique PID to distinguish different types of stream data. For example, the PID of the PAT (Program Association Table) is fixed to 0. The PAT records the PMT and the PID (ProgramID, media stream identifier) ​​used by each media stream. The decoder constructs the PID mapping relationship between the PMT and the media stream through the PAT. Usually, the PAT and PMT are periodically encapsulated into the TS stream at the same time.

[0104] See also Figure 8 and Fig. 9 In an optional embodiment of the present application, the above step 105, the decoding end parses the multimedia stream data according to the program mapping table data structure, the basic code stream data and the code stream format information to obtain the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data, including the following steps 901-904:

[0105] Step 901: Parse the program mapping table data structure in the multimedia stream data packet, read the table items in the program mapping table data structure, and obtain the code stream type and media stream identifier of each code stream data;

[0106] Step 902: extract the code stream type and the media stream identifier in the program mapping table data structure;

[0107] Step 903: construct a stream type table based on the stream type and the media stream identifier;

[0108] The code stream type table includes the code stream type of each code stream data and the code stream format information corresponding to the code stream data.

[0109] Step 904: extract each stream data in the media stream data packet by using the media stream identifier and the stream type, and obtain the original stream data corresponding to the basic stream data and the stream format information of the basic stream data.

[0110] The multimedia stream data decapsulation process is the process of extracting the original code stream data and information from the media stream data packet. The embodiment of the present application obtains the code stream number and the corresponding code stream type in the media stream data packet by constructing a code stream type table, and then extracts the code stream data and code stream format information, distinguishes different program code streams through the media stream identifier PID in the TS packet, thereby restoring the basic code stream data PES and code stream format data, extracting the correct original code stream data and code stream format information, without missing the format information, ensuring the integrity of the code stream information transmission, and having higher reliability.

[0111] The number, number and type of code streams are recorded in TS through PMT data. TS packets distinguish PMT data streams through media stream identifiers PID, and then extract PMT data in the program map table data structure from the payload of TS packets. The media stream identifier PID of the program map table data structure PMT is recorded by the program association table PAT data. The media stream identifier PID of the program association table PAT is fixed to 0. After parsing the program map table data structure PMT, PMT data can be identified and extracted in subsequent TS packets.

[0112] The process of constructing the code stream type table can be found in Fig.10 :First, the version of the program mapping table data structure PMT data is checked. The version number of the data is recorded in the program mapping table data structure PMT data. If the version number is greater than the cached data version, the new program mapping table data structure PMT version is recorded; then the table item of the program mapping table data structure PMT is read. If the version number is less than or equal to the cached data version, the current program mapping table data structure PMT data is ignored. The program mapping table data structure PMT data in the TS packet can be sent repeatedly periodically. The version check can avoid repeated data parsing and improve the overall efficiency. Read the stream type field in the program mapping table data structure PMT table item, and determine whether the stream type field matches the preset field. If it matches, for example, it is equal to 254, then read the subsequent registration descriptor information, and identify the descriptor field from the four-byte code of the descriptor according to the data structure definition in Table 1 above to obtain the stream type. If it is not equal to 254, match it through other set tables, such as the type value defined in Table 2-34 of ISO / IEC 13818-1 2023 (not shown in this application). Finally, record the bitstream type and the media stream identifier PID, and cache the bitstream type and the media stream identifier PID as table items in the initial bitstream type table to obtain the above bitstream type table. Subsequently, the bitstream type table can be indexed by the media stream identifier PID of the TS packet header to summarize and extract each audio and video bitstream data and format information, which is more convenient and efficient.

[0113] See also Fig.11 and Fig.12 In an optional embodiment of the present application, the above step 904, extracting each stream data in the media stream data packet by using the media stream identifier and the stream type to obtain the original stream data corresponding to the basic stream data and the stream format information of the basic stream data, includes the following steps 1101-1103:

[0114] Step 1101: Based on the code stream type table, parse the TS data packet in each code stream data to obtain the private data of the TS packet header private data field and the TS payload data;

[0115] Step 1102: reorganize the private data according to a pre-configured order to generate the bitstream format information of the basic bitstream data;

[0116] Step 1103: reorganize the TS payload data to form the original code stream data corresponding to the basic code stream data.

[0117] Extracting the original code stream data and code stream format information is to distinguish different code stream data by the media stream identifier PID in the TS packet, and then parse the private data in the TS packet header data and reassemble it in order to generate the code stream format information, and then reassemble the TS payload data to generate the original code stream data. Read the media stream identifier PID field of the TS packet and query it in the code stream type table. If the media stream identifier PID matches, continue parsing. If not, read the subsequent TS packets. When there is extended data in the TS packet header, extract the private data (S402), and then parse the private data type according to the private data format described in Appendix D of "DVB - ETSI TS 101 154 V2.6.1".

[0118] When the private data type is equal to 192 and the start marker field of the TS packet header is equal to 1, the format information is parsed, otherwise the private data is not parsed. The format information is parsed according to the data structure defined in Table 3 above, and the length of the format information is checked to be at least 2 bytes. The PMT version and type value are extracted. The type value equal to 1 indicates video format information, and equal to 2 indicates audio format information. Then, the video and audio formats are parsed according to the data structure defined in Table 4 and Table 5 above.

[0119] For the TS packet payload part, extract the basic code stream data PES, and splice the content payload part of the TS packet in sequence to obtain the basic code stream data PES until the length of the basic code stream data PES is equal to the PES length field in the format information. If the start field of the next TS packet header is equal to 1, the current basic code stream data PES extraction is completed.

[0120] Please continue to see Fig.12 In an optional embodiment of the present application, the basic code stream data PES is further verified, and the verification process is as follows:

[0121] If the TS packet number is discontinuous in the PES parsing of the basic code stream data, the verification is wrong. If it is continuous, continue to execute; the length field carried in the PES packet header data plus 6 is compared with the actual extracted PES length of the basic code stream data. If they are not equal, the verification is wrong. If they are equal, continue to the next step; if the length field of the PES header data is equal to 0, the PES length value of the code stream format information is used to compare with the actual extracted PES length of the basic code stream data. If they are not equal, the verification is wrong. If they are equal, continue to the next step; after the data verification is successful, the original code stream data and code stream format information are delivered.

[0122] In an optional embodiment of the present application, if the multimedia data to be processed is image data, the decoding end parses the multimedia stream data according to the program mapping table data structure, the basic code stream data and the code stream format information in the above step 105 to obtain the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data, and then generates a multimedia image based on the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data, and optimizes the multimedia image, and the optimization process includes at least the following steps:

[0123] The multimedia image to be optimized is obtained and converted into HSI color space, the brightness component in HSI color space is extracted and brightness is enhanced; the contrast of the enhanced brightness component is adjusted, and the brightness area is divided and the dynamic range is optimized; the edge of the optimized brightness component is enhanced and recombined to generate an enhanced HSI image, and the enhanced HSI image is converted back into RGB color space to obtain the optimized multimedia image.

[0124] In an optional embodiment of the present application, the obtaining of the multimedia image to be optimized and performing HSI color space conversion refers to extracting original image data from a multimedia device, converting the obtained image format into a standard format, and performing noise smoothing on the image by a mean filter;

[0125] Normalize the color channel data of the preprocessed original image, and calculate the brightness component I, saturation S and hue H of the image based on the normalized color channel data;

[0126] Get the converted HSI color space and the corresponding HSI image.

[0127] In an optional embodiment of the present application, extracting the brightness component in the HSI color space and performing brightness enhancement refers to extracting the brightness component I from the HSI color space and using it as an independent channel, normalizing the extracted brightness component I, and then smoothing the brightness component through a bilateral filter, and outputting the final brightness component after completing edge smoothing and normalization.

[0128] Apply a brightness conversion function to the brightness component to enhance the contrast and depth of the original brightness:

[0129]

[0130] Where A is the pixel brightness value after brightness conversion, a(x) is the brightness enhancement adjustment parameter, and I(x) is the brightness component;

[0131] A Gaussian pyramid is constructed for the brightness component after brightness conversion. The pyramid is set to n layers to generate images at multiple resolution levels. The initial layer directly takes the converted brightness component. From the second layer onwards, the previous layer image is Gaussian blurred and downsampled layer by layer to generate low-resolution images at each layer until the nth layer is reached.

[0132] After reaching the nth layer, a multi-layer Gaussian pyramid is obtained, each layer represents the low-frequency information of brightness at different spatial scales;

[0133] Construct a Laplacian pyramid based on the Gaussian pyramid, extract high-frequency information from each layer of the Gaussian pyramid image, upsample the Gaussian pyramid image of the l+1th layer by bilinear interpolation, and calculate the Laplacian pyramid image of the lth layer Ll(x,y) using the following formula:

[0134]

[0135] Where Gl(x, y) is the lth layer image of the Gaussian pyramid, Gl+1(x, y) is the l+1th layer image of the Gaussian pyramid, U() is the upsampling operation, which enlarges Gl+1(x, y) to the same resolution as the lth layer Gl(x, y), and method represents the interpolation method used for upsampling;

[0136] The calculated Stored as the high-frequency component of the l-th layer of the Laplacian pyramid;

[0137] The last layer of the Gaussian pyramid is used as the lowest frequency information of the Laplace pyramid, and the high frequency information of each layer is integrated with the low frequency basic brightness information to form a complete Laplace pyramid structure. is the multi-scale brightness component;

[0138] The brightness component of each layer is input into the initial convolution layer of DCNN to capture the basic brightness features through convolution operation, and the convolution structure is processed using the ReLU activation function to extract the basic details of brightness;

[0139] In the middle layer of DCNN, the output of the initial convolutional layer H(z) is processed by multiple layers of convolution, and each layer of convolution is based on the output of the previous layer:

[0140]

[0141] In the formula, represents the brightness feature of the k-th layer output, is the k-th convolution kernel weight, is the brightness feature output from the k-1th layer, is the bias item of the current layer;

[0142] The output of each convolution layer is batch normalized and then the ReLU activation function is used:

[0143]

[0144] In the formula, Represents the brightness feature output after the k-th layer of convolution, batch normalization and activation processing. BN() is the batch normalization operation;

[0145] After multiple layers of convolution, batch normalization and activation function, the deep extracted brightness features are finally obtained;

[0146] Based on the brightness component I, at each pixel x in the image, take N The local brightness mean and standard deviation of pixel x are calculated for the pixels in the N windows, and the local brightness mean and local standard deviation of all pixels are combined to generate the final illumination estimation map T(x). The weight matrix W(x) is generated based on the illumination estimation map.

[0147] The multi-scale brightness components and deep brightness features are adjusted to the same resolution through upsampling and downsampling methods;

[0148] The multi-channel depth brightness features are merged using the mean of all channels and converted into single-channel brightness features. After the format is unified, weighted fusion is performed through the weight matrix to obtain the fused brightness component. Starting from the lowest frequency layer, the Laplacian pyramid components of each layer are upsampled and superimposed in turn to gradually reconstruct the complete brightness component to obtain the enhanced brightness component. .

[0149] In an optional embodiment of the present application, adjusting the contrast of the enhanced brightness component refers to obtaining the enhanced brightness component and performing normalization processing, defining the solution population size of the water cycle algorithm as P, and each solution represents a set of contrast parameters (s, d), where s refers to the contrast scaling factor and d refers to the brightness offset;

[0150] Initialize the river solution and the stream solution, and simulate the process of water flow to find the optimal solution;

[0151] According to the parameters (s, d) of the current solution, the contrast of the normalized brightness component is adjusted:

[0152]

[0153] In the formula, is the adjusted brightness value, is the normalized brightness value;

[0154] Calculate the objective function value J for the adjusted brightness component to evaluate the optimization effect of the current solution:

[0155]

[0156] Where M and N are the width and height of the image, representing the total range of pixels. is the local brightness mean, x represents the horizontal position of the pixel, and y represents the vertical position of the pixel;

[0157] Update the river solution and the stream solution according to the objective function value;

[0158] Set the maximum number of iterations until the maximum number of iterations is reached and stop the iteration. After the water cycle algorithm completes the iteration, select the solution with the smallest objective function value and use its parameters as the final optimal solution. Use the optimal solution to adjust the brightness component and perform range clipping on the optimized brightness value to obtain the brightness component after contrast adjustment. .

[0159] In an optional embodiment of the present application, the brightness region division and dynamic range optimization refers to dividing the brightness component Divide into bright, dark and mid-tone areas, set the threshold H and , if the brightness value , then the pixel is classified as a brightness area. If the brightness value , then the pixel is classified as a dark area. If the brightness value , the pixel is classified as a mid-tone area; for the brightness area, a nonlinear compression function is used to reduce the output of the brightness of the bright part; for the dark area, a logarithmic transformation is used to enhance the dark details; for the mid-tone area, a linear adjustment is performed to balance the brightness transition; the adjustment results of the bright, dark, and mid-tone areas are combined into the final dynamic range optimized brightness component.

[0160] In an optional embodiment of the present application, the edge enhancement of the optimized brightness component and the recombining to generate the enhanced HSI image refers to using the Laplacian operator to enhance the edge details of the brightness component after dynamic range optimization, keeping the brightness component consistent with the hue and saturation in the original HSI image, and The brightness channel in the original HSI image is replaced, and the enhanced brightness component is recombined with the hue and saturation components in the HSI color space to form the final HSI image.

[0161] In an optional embodiment of the present application, converting the enhanced HSI image back to the RGB color space to obtain an optimized multimedia image means converting the HSI image into an RGB image through the image processing library OpenCV to obtain an optimized multimedia image, saving the generated multimedia image in a standard format and performing data backup.

[0162] It should be understood that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0163] See also Fig.13 An embodiment of the present application provides a multimedia data stream transmission device 1300, including:

[0164] The first encapsulation module 1310 is used to encapsulate the program stream information of the multimedia data to be processed to obtain a program mapping table data structure; wherein the program stream information includes: at least one of the number of code streams, code stream numbers and code stream types of the multimedia data to be processed;

[0165] The encoding module 1320 is used to encode the multimedia data to be processed to form basic code stream data;

[0166] The configuration module 1330 is used to configure the basic code stream data according to a preset code stream format to obtain the code stream format information;

[0167] The second encapsulation module 1340 is used to divide, package and encapsulate the program mapping table data structure, the basic code stream data and the code stream format information according to a preset data transmission stream format to form a multimedia stream data packet;

[0168] The parsing module 1350 is used to parse the multimedia stream data according to the program mapping table data structure, the basic code stream data and the code stream format information to obtain the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data.

[0169] In an optional embodiment of the present application, the first encapsulation module 1310 is specifically used to, for each channel of code stream data of the multimedia data to be processed, if the code stream type in the program stream information is empty, configure the code stream type of the code stream data according to a pre-configured registration descriptor format to obtain the code stream descriptor of the code stream data; configure the code stream number, the code stream type and the code stream descriptor to the initial program mapping table data structure to obtain the program mapping table data structure.

[0170] In an optional embodiment of the present application, the configuration module 1330 is specifically used to generate multimedia standard information of the basic code stream data according to the format data of the multimedia data to be processed; and encapsulate the multimedia standard information, the timestamp of the basic code stream data and the data length of the basic code stream data into the code stream format information.

[0171] In an optional embodiment of the present application, the second encapsulation module 1340 is specifically used to package the program mapping table data structure and the basic code stream data into the payload part of a preset multimedia stream data packet; and package the code stream format information into the header part of a preset multimedia stream data packet, wherein the header part includes an expandable private data field.

[0172] In an optional embodiment of the present application, the configuration module 1330 is specifically used to parse the program mapping table data structure in the multimedia stream data packet, read the table items in the program mapping table data structure, and obtain the stream type and media stream identifier of each stream data; extract each stream data in the media stream data packet through the media stream identifier and the stream type, and obtain the original stream data corresponding to the basic stream data and the stream format information of the basic stream data.

[0173] In an optional embodiment of the present application, the parsing module 1350 is also used to extract the bitstream type and the media stream identifier in the program mapping table data structure; construct a bitstream type table based on the bitstream type and the media stream identifier; wherein the bitstream type table includes the bitstream type of each bitstream data and the bitstream format information corresponding to the bitstream data.

[0174] In an optional embodiment of the present application, the parsing module 1350 is specifically used to parse the TS data packets in each code stream data based on the code stream type table to obtain the private data of the TS packet header private data field and the TS payload data; reorganize the private data in a pre-configured order to generate the code stream format information of the basic code stream data; and reorganize the TS payload data to form the original code stream data corresponding to the basic code stream data.

[0175] For the specific definition of the multimedia data stream transmission device 1300, please refer to the definition of the multimedia data stream transmission method above, which will not be repeated here. Each module in the multimedia data stream transmission device 1300 can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the modules above.

[0176] In one embodiment, a computer device is provided, the internal structure diagram of which can be as follows: Fig.14 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a multimedia data stream transmission method as described above is implemented. It includes: a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, any step in the multimedia data stream transmission method as described above is implemented.

[0177] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any step in the above multimedia data stream transmission method can be implemented.

[0178] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0179] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0180] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0181] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0182] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0183] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A multimedia data stream transmission method, characterized in that: include: Encapsulating the program stream information of the multimedia data to be processed to obtain a program mapping table data structure; wherein the program stream information includes: at least one of the number of code streams, code stream numbers and code stream types of the multimedia data to be processed; Encoding the multimedia data to be processed to form basic code stream data; The basic code stream data is configured according to a preset code stream standard format to obtain code stream format information; Divide and package the program mapping table data structure, the basic code stream data and the code stream format information according to a preset data transmission stream format to form a multimedia stream data packet; Parsing the multimedia stream data according to the program mapping table data structure, the basic code stream data and the code stream format information to obtain original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data; The step of encapsulating the program stream information of the multimedia data to be processed to obtain the program mapping table data structure includes: For each channel of code stream data of the to-be-processed multimedia data, if the code stream type in the program stream information is empty, configure the code stream type of the code stream data according to a pre-configured registration descriptor format to obtain a code stream descriptor of the code stream data; Configuring the code stream number, the code stream type and the code stream descriptor into an initial program mapping table data structure to obtain the program mapping table data structure; The configuring the basic code stream data according to a preset code stream format to obtain the code stream format information includes: Generating multimedia standard information of the basic code stream data according to the format data of the multimedia data to be processed; Encapsulating the multimedia standard information, the timestamp of the basic code stream data, and the data length of the basic code stream data into the code stream format information; The program mapping table data structure, the basic code stream data and the code stream format information are divided, packaged and encapsulated according to a preset data transmission stream format to form a multimedia stream data packet, including: Packing the program mapping table data structure and the basic code stream data into a load part of a preset multimedia stream data packet; Packing the code stream format information into a header part of a preset multimedia stream data packet, wherein the header part includes an expandable private data field; The multimedia data to be processed includes: video data and audio data. Correspondingly, the multimedia format information includes video format information and audio format information; the bitstream format information includes the video format size, frame type, audio sampling rate, number of channels, time code, and camera parameter information. These bitstream format information are independently described in the same format and carried by using the private data segment of the TS packet to adapt to more bitstream types.

2. The multimedia data stream transmission method according to claim 1, characterized in that: The step of parsing the multimedia stream data according to the program mapping table data structure, the basic code stream data and the code stream format information to obtain the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data includes: Parsing the program mapping table data structure in the multimedia stream data packet, reading the table items in the program mapping table data structure, and obtaining the code stream type and media stream identifier of each code stream data; The code stream data of each channel in the media stream data packet is extracted by using the media stream identifier and the code stream type, so as to obtain the original code stream data corresponding to the basic code stream data and the code stream format information of the basic code stream data.

3. The multimedia data stream transmission method according to claim 2, characterized in that: Before extracting each stream data in the media stream data packet by using the media stream identifier and the stream type to obtain the original stream data corresponding to the basic stream data and the stream format information of the basic stream data, the method further includes: Extracting the code stream type and the media stream identifier in the program map table data structure; A stream type table is constructed based on the stream type and the media stream identifier; wherein the stream type table includes the stream type of each stream data and the stream format information corresponding to the stream data.

4. The multimedia data stream transmission method according to claim 3, characterized in that: The extracting each stream data in the media stream data packet by using the media stream identifier and the stream type to obtain the original stream data corresponding to the basic stream data and the stream format information of the basic stream data includes: Based on the code stream type table, TS data packets in each code stream data are parsed to obtain private data in the private data field of the TS packet header and TS payload data; Reorganize the private data according to a pre-configured order to generate the code stream format information of the basic code stream data; The TS payload data is reorganized to form the original code stream data corresponding to the basic code stream data.

5. A computer device comprising: The method comprises a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of the method according to any one of claims 1 to 4 when executing the computer program.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images