Encoding methods and electronic devices

Abstract

Embodiments of this application provide an encoding method and an electronic device. The method comprises a first electronic device acquiring a signal to be encoded and then generating a first bitstream, the first bitstream comprising a second bitstream and a model identifier, the second bitstream being acquired by encoding the signal to be encoded based on an encoding model, the value of the model identifier being a pre-configured identifier value, the pre-configured identifier value instructing the second electronic device to acquire model information for a decoding model, the decoding model corresponding to an encoding model. After the first bitstream has been transmitted to the second electronic device in this manner, the second electronic device may acquire model information for a decoding model based on the pre-configured identifier value to synchronously update the decoding model of the second electronic device and the encoding model of the first electronic device, or synchronously switch between the decoding model used for decoding by the second electronic device and the encoding model used for encoding by the first electronic device.

Description

Technical Field

[0001] This application claims the priority of Chinese Patent Application No. 202310630393.3, titled "Coding Method and Electronic Device", which was filed with the State Intellectual Property Office of China on May 30, 2023 and is incorporated herein by reference in its entirety.

[0002] Embodiments of this application relate to the field of encoding and decoding, and in particular, to encoding methods and electronic devices.

Background Art

[0003] With the development of high-definition audio and video as well as three-dimensional (3D) audio and video, people have increasingly high requirements for the quality of audio and video. As a result, the amount of audio and video data in media applications has increased rapidly. This also makes audio and video compression an essential part of media applications such as media communication applications and media broadcast applications.

[0004] In conventional audio and video compression technologies, the original audio and video data are compressed based on the basic principles of signal processing and the correlation between signals in time and space, reducing the amount of data and facilitating the transmission or storage of data. With the continuous development and maturity of artificial intelligence (AI) technology, AI technology has also been introduced into the field of audio and video encoding and decoding to improve encoding performance.

[0005] After the coding standard is standardized, related AI models are usually not updated or iterated. This affects the long-term evolution and performance optimization of the coding standard.

Summary of the Invention

[0006] In view of this, this application provides a coding method and an electronic device. [Means for solving the problem]

[0007] According to a first aspect, one embodiment of the present application provides an encoding method applicable to a first electronic device. The method includes the steps of first acquiring a signal to be encoded, and then generating a first bitstream, the first bitstream comprising a second bitstream and a model identifier, the second bitstream being acquired by encoding the signal to be encoded based on an encoding model, the value of the model identifier being a pre-configured identifier value, the pre-configured identifier value instructing the second electronic device to acquire model information for a decoding model, the decoding model corresponding to the encoding model.

[0008] In this way, the first electronic device sets the value of the model identifier in the first bitstream to a preset identifier value in order to instruct the second electronic device to obtain model information for the decoding model. After the first bitstream is transmitted to the second electronic device, the second electronic device may obtain the model information for the decoding model based on the preset identifier value and synchronously update the decoding model of the second electronic device and the encoding model of the first electronic device, or synchronously switch the decoding model used for decoding by the second electronic device and the encoding model used for encoding by the first electronic device. Thus, the long-term evolution of coding standards and the optimization of coding performance are facilitated.

[0009] For example, both the encoding and decoding models may be implemented using neural networks. Therefore, the encoding model may be called an AI encoding model, and the decoding model may be called an AI decoding model.

[0010] In one possible scheme, the coding model and the decoding model are two independent models. In this case, the coding model and the decoding model may be trained together. In another possible scheme, the coding model and the decoding model are two parts of the model.

[0011] It should be understood that when the encoding model and the decoding model are two parts of a model, a pre-configured identifier value may instruct a second electronic device to retrieve the model information of the encoding model and the model information of the decoding model. In this way, the second electronic device can reconstruct the encoding model based on the model information of the encoding model and reconstruct the decoding model based on the model information of the decoding model. This application is illustrated by using an example in which the encoding model and the decoding model are two independent models.

[0012] In one possible scheme, an identifier (i.e., a flag) or field may be added to the first bitstream as a model identifier. In another possible scheme, an existing field in the first bitstream may be reused, or an existing field may be used as a model identifier. This is not limited to the present application. When an existing field in the first bitstream is reused, or an existing field may be used as a model identifier, the present application does not require a change in the decoding syntax, i.e., the model update or switch is performed while maintaining backward compatibility.

[0013] For example, the signals to be encoded may include, but are not limited to, audio signals, image signals, and video signals.

[0014] According to the first embodiment, the first bitstream further includes a third bitstream, the third bitstream being generated based on model information of a decoding model. A pre-configured identifier value may indicate that the first bitstream includes a third bitstream. In this case, the pre-configured identifier value may instruct a second electronic device to obtain model information of a decoding model from the first bitstream. In this way, the first electronic device can transmit model information of a decoding model to the second electronic device, which then reconstructs a decoding model to be used for decoding based on the model information of the decoding model.

[0015] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the method further includes the step of generating a third bitstream based on model information of a decoding model, wherein the step of generating the first bitstream includes the step of encapsulating the second bitstream and the third bitstream in order to obtain the first bitstream.

[0016] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the first bitstream further includes index information corresponding to model information of the decryption model. A pre-configured identifier value indicates that the first bitstream includes index information corresponding to model information of the decryption model. In this case, the pre-configured identifier value may instruct the second electronic device to obtain the model information of the decryption model from the third electronic device based on the index information parsed from the first bitstream.

[0017] In this way, the first electronic device only needs to transmit index information corresponding to the model information of the decoding model. Transmitting only index information corresponding to the model information of the decoding model has lower bitrate overhead and can reduce bandwidth requirements compared to transmitting the model information of the decoding model.

[0018] For example, the index information may be a Uniform Resource Locator (URL). This is not limited to the present application.

[0019] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the step of generating a first bitstream includes the step of obtaining the first bitstream based on a second bitstream and index information corresponding to model information of a decoding model.

[0020] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the pre-configured identifier value further indicates that the model information of the decryption model is stored in a pre-configured storage location within the third device. In this case, the pre-configured identifier value may instruct the second electronic device to retrieve the model information of the decryption model from the pre-configured storage location within the third electronic device, and the pre-configured storage location may be a location predetermined by the first and second electronic devices.

[0021] In this way, the first electronic device does not need to transmit index information corresponding to the model information of the decoding model, nor does it need to transmit the model information of the decoding model. Therefore, the bitrate overhead is lower and the bandwidth requirements are lower.

[0022] It should be noted that the method by which the second electronic device obtains model information from the first bitstream is assumed to be called the first method, the method by which the second electronic device obtains model information from the first bitstream based on index information corresponding to the model information is called the second method, and the method by which the second electronic device obtains model information from a pre-configured storage location in the third electronic device is called the third method. In one possible method, the first and second electronic devices may agree in advance on the method by which the second electronic device obtains model information for the decoded model. In the encoding and decoding process, once the first and second electronic devices agree on the first method, the first electronic device may generate a third bitstream based on the model information for the decoded model and encapsulate the second and third bitstreams in order to obtain the first bitstream. In the encoding and decoding process, if the first and second electronic devices agree on a second scheme, the first electronic device may generate a first bitstream based on the second bitstream and index information corresponding to the model information of the decoding model. In the encoding and decoding process, if the first and second electronic devices agree on a third scheme, the first electronic device may generate a first bitstream based on the second bitstream.

[0023] In one possible scheme, during the encoding and decoding process, the first electronic device may select one of the following schemes as the target scheme: the first scheme, the second scheme, and the third scheme. When the target scheme is the first scheme, the first electronic device may generate a third bitstream based on model information of the decoding model and encapsulate the second and third bitstreams to obtain the first bitstream. When the target scheme is the second scheme, the first electronic device may generate the first bitstream based on the second bitstream and index information corresponding to the model information of the decoding model. When the target scheme is the third scheme, the first electronic device may generate the first bitstream based on the second bitstream.

[0024] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the model identifier is an Elementary Stream (ES) field.

[0025] For example, an ES encapsulation format may contain multiple fields, and an ES field may be a field included in the ES encapsulation format. In other words, an ES packet may be obtained by encapsulating data according to an ES encapsulation format. An ES packet may contain multiple fields, and an ES field may be a field within an ES packet.

[0026] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, when the target bitstream type of the first bitstream is the type of bitstream encapsulated according to the target container format or the type of bitstream encapsulated according to the target transport protocol, the model identifier is a field in the target container format. In this way, the second electronic device can obtain the value of the model identifier without parsing the ES packet.

[0027] For example, the target container format includes, but is not limited to, a transport stream (TS) format, an MP4 format, etc. This is not limited in this application.

[0028] For example, the target container format may include a plurality of fields, or the fields within the target container format may be fields included in the target container format. In other words, the encapsulated packet may be obtained by encapsulating data according to the target container format. The encapsulated packet may include a plurality of fields, or the fields within the target container format may be fields within the encapsulated packet obtained by encapsulation according to the target container format. For example, when the target container format is the TS format / MP4 format, the fields within the target container format may be fields within the TS packet.

[0029] For example, the target transport protocol may include a plurality of types, such as HTTP Live Streaming (HLS) and Real-time Transport Protocol (RTP). This is not limited in this application.

[0030] According to any one of the first aspect or the foregoing implementation forms of the first aspect, when the target bitstream type of the first bitstream is the type of the bitstream encapsulated according to the target transport protocol, the model identifier is a field within the target transport protocol. In this way, the second electronic device can obtain the value of the model identifier without decrypting the first bitstream according to the target container format and without analyzing the ES packet.

[0031] For example, the encapsulation format of the target transport protocol may contain multiple fields, and the fields within the target transport protocol may be fields included in the encapsulation format of the target transport protocol. In other words, an encapsulated packet may be obtained by encapsulating data according to the encapsulation format of the target transport protocol. The encapsulated packet may contain multiple fields, and the fields within the target transport protocol may be fields within the encapsulated packet obtained by encapsulation according to the target transport protocol.

[0032] According to either the first embodiment or the aforementioned implementation of the first embodiment, the method further includes the steps of determining whether the decoding model is an updated decoding model, and if the decoding model is an updated decoding model, setting the value of the model identifier to a pre-configured identifier value. In this case, the pre-configured identifier value instructs a second electronic device to retrieve model information for the updated decoding model. In this case, the encoding model may not be updated, or only the decoding model may be updated (for example, the decoding model may be optimized). In this case, the updated decoding model corresponds to the encoding model.

[0033] In one possible case, both the encoding model and the decoding model are updated. In this case, the method further includes the step of determining whether both the encoding model and the decoding model have been updated, and the step of encoding the signal to be encoded based on the encoding model in order to obtain a second bitstream, if it is determined that both the encoding model and the decoding model have been updated, the step of encoding the signal to be encoded based on the updated encoding model in order to obtain a second bitstream. In this case, a pre-configured identifier value instructs a second electronic device to obtain model information for the updated decoding model, and the updated decoding model corresponds to the updated encoding model.

[0034] In one possible case, the first electronic device includes multiple encoding models and multiple decoding models (with one-to-one correspondence between encoding models and decoding models). The method further includes the steps of determining whether to switch the encoding model used for encoding, and, if it is determined that the encoding model used for encoding has been switched, encoding the signal to be encoded based on the switched encoding model in order to obtain a second bitstream. In this case, a pre-configured identifier value instructs the second electronic device to obtain model information for the switched decoding model, where the switched decoding model corresponds to the switched encoding model.

[0035] It should be understood that the first electronic device may set the value of the model identifier in the first bitstream to a preset identifier value, regardless of whether the encoding and decoding models in the first electronic device are updated (or whether the decoding model is updated), and whether the encoder in the first electronic device switches the encoding model used for encoding. In this case, the preset identifier value may represent the type of decoding model (or encoding model). Correspondingly, after obtaining the preset identifier value, the second electronic device determines whether a decoding model of the type corresponding to the preset identifier value exists in the second electronic device, and if no decoding model of the type corresponding to the preset identifier value exists, it obtains the model information of the decoding model.

[0036] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the step of encapsulating the second bitstream and the third bitstream in order to obtain a first bitstream includes the step of encapsulating the second bitstream and the third bitstream according to the target bitstream type in order to obtain the first bitstream.

[0037] Note that in one possible configuration, the first and second electronic devices may agree in advance on the target bitstream type. In this manner, the first electronic device encapsulates the second and third bitstreams according to the target bitstream type, and the second electronic device decapsulates the first bitstream according to the target bitstream type.

[0038] In one possible scheme, during the encoding process, the first electronic device may, if necessary, select a bitstream type from a group of bitstream types as the target bitstream type. Correspondingly, the second electronic device identifies the bitstream type of the first bitstream (i.e., the target bitstream type) and then decapsulates the first bitstream according to the target bitstream type.

[0039] According to the first embodiment or any of the aforementioned implementations of the first embodiment, the target bitstream type is an ES type. The step of encapsulating the second and third bitstreams according to the target bitstream type in order to obtain the first bitstream includes the step of encapsulating the third and second bitstreams in a first ES packet and using the first ES packet as the first bitstream. That is, the first bitstream is an ES stream, and the second and third bitstreams are encapsulated in the same ES packet for transmission. In this way, after the ES packet is obtained, the ES packet can be transmitted directly without encapsulation according to the target container format or target transport protocol, thereby reducing bitrate overhead and bandwidth requirements. In addition, since the second and third bitstreams are encapsulated in the same ES packet, the second and third bitstreams can be transmitted on the same channel, thereby reducing transmission complexity.

[0040] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the target bitstream type is the type of bitstream encapsulated according to the target container format. The step of encapsulating the second and third bitstreams according to the target bitstream type in order to obtain the first bitstream includes the steps of encapsulating the third and second bitstreams in a second ES packet and encapsulating the second ES packet according to the target container format in order to obtain the first bitstream. The ES packet may be divided into a plurality of subpackets, and the subpackets are encapsulated according to the target container format. This facilitates bitstream distribution and synchronous audio and video control of the bitstream distribution device.

[0041] For example, when the target container format is TS format, the first bitstream is a TS stream, the second and third bitstreams may be encapsulated in the same ES packet, and the ES packet is encapsulated in one TS stream for transmission. When the target container format is MP4 format, the first bitstream is an MP4 stream, the second and third bitstreams are encapsulated in the same ES packet, and the ES packet is encapsulated in one MP4 TS stream.

[0042] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the target bitstream type is the type of bitstream encapsulated according to the target container format. The step of encapsulating the second and third bitstreams according to the target bitstream type in order to obtain the first bitstream includes the steps of encapsulating the third bitstream in a third ES packet and using the second bitstream as a fourth ES packet, and encapsulating the third and fourth ES packets according to the target container format in order to obtain the first bitstream, setting the type of the third ES packet to a pre-configured type and setting the type of the fourth ES packet to a media type. That is, the second and third bitstreams are encapsulated in different ES packets for transmission. In this way, compared to encapsulating the second and third bitstreams in the same ES packet, the second electronic device does not need to decompose the ES packets to analyze the second and third bitstreams, thereby reducing the complexity of the analysis process.

[0043] For example, media types may include, but are not limited to, audio types, video types, and image types.

[0044] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the preconfigured type is a metadata type. In other words, the third bitstream is encapsulated as metadata.

[0045] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the target bitstream type is the type of bitstream encapsulated according to the target container format. The step of encapsulating the second and third bitstreams according to the target bitstream type in order to obtain the first bitstream includes the steps of encapsulating the third bitstream in a fifth ES packet and using the second bitstream as a sixth ES packet, encapsulating the fifth ES packet according to the target container format and setting the type of the fifth ES packet to a pre-configured type in order to obtain the fourth bitstream, encapsulating the sixth ES packet according to the target container format and setting the type of the sixth ES packet to a media type in order to obtain the fifth bitstream, and combining the fourth and fifth bitstreams in order to obtain the first bitstream. That is, the second and third bitstreams are encapsulated separately according to the target container format. For example, the second and third bitstreams are encapsulated in two TS streams. In this way, compared to encapsulating the second and third bitstreams in the same TS stream, the second electronic device does not need to decompose the TS stream to analyze the second and third bitstreams, thereby reducing the complexity of the analysis process.

[0046] It should be noted that combining the fourth bitstream and the fifth bitstream may be understood as combining only the fourth bitstream and the fifth bitstream.

[0047] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the target bitstream type is the type of bitstream encapsulated according to the target transport protocol. The step of encapsulating the second and third bitstreams according to the target bitstream type in order to obtain the first bitstream includes the steps of encapsulating the third bitstream in a seventh ES packet and using the second bitstream as an eighth ES packet; encapsulating the seventh ES packet according to the target container format and setting the type of the seventh ES packet to a pre-configured type in order to obtain the sixth bitstream; encapsulating the eighth ES packet according to the target container format and setting the type of the eighth ES packet to a media type in order to obtain the seventh bitstream; and encapsulating the sixth and seventh bitstreams according to the target transport protocol in order to obtain the first bitstream. In other words, after the second and third bitstreams are separately encapsulated according to the target container format, the encapsulation is performed according to the target transport protocol. A second bitstream having a different reconstruction accuracy (which may be understood as the accuracy of the reconstructed signal obtained by decoding the second bitstream) may be generated. In this way, the bitstream distribution device distributes the first bitstream containing the second bitstream having a different reconstruction accuracy based on bandwidth, thereby ensuring the smooth reproduction of the reconstructed signal.

[0048] It should be noted that the first bitstream obtained by combining the fourth bitstream and the fifth bitstream contains only the fourth bitstream and the fifth bitstream. The first bitstream obtained by encapsulating the fourth bitstream and the fifth bitstream according to the target transport protocol, in addition to the sixth bitstream and the seventh bitstream, further includes fields contained in the encapsulation format of the target transport protocol.

[0049] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the first bitstream further includes an index file, the index file includes index information for a sixth bitstream and index information for a seventh bitstream, the index information for the sixth bitstream in the index file precedes the index information for the seventh bitstream. In this way, the second electronic device can first obtain the model information used to reconstruct the decoding model, and as a result, the delay in decoding the second bitstream corresponding to each frame of the signal can be reduced.

[0050] For example, the target bitstream type is the type of bitstream encapsulated according to HLS, and the first bitstream may be an HLS stream.

[0051] According to either the first embodiment or the aforementioned implementation of the first embodiment, the first bitstream further includes an index file, the attribute information of the initialization tags in the index file includes the index information of the sixth bitstream. This ensures that the decoder can decode the second bitstream after the decoding model has been reconstructed and it has been determined that the decoding model has been read (i.e., the decoder has been initialized).

[0052] For example, the target bitstream type is the type of bitstream encapsulated according to HLS, and the first bitstream may be an HLS stream.

[0053] According to either the first embodiment or the aforementioned implementation of the first embodiment, the signal to be encoded comprises M frames, has M second bitstreams, model information comprises N information groups, the third bitstream comprises N subbitstreams, the N information groups correspond one-to-one with the N subbitstreams, and the first ES packet comprises N ninth ES packets and MN tenth ES packets, where M and N are positive integers and M is greater than or equal to N. The step of encapsulating the third bitstream and the second bitstream in a first ES packet comprises encapsulating the N subbitstreams and N second bitstreams in N ninth ES packets and using the other MN second bitstreams as MN tenth ES packets, where the ninth ES packet comprises one subbitstream and one second bitstream, and the tenth ES packet comprises one second bitstream.

[0054] A second bitstream is obtained by encoding one frame of the signal to be encoded based on the encoding model. N may be determined based on the amount of data in the model information (or the encoded data of the model information), the network bandwidth, and the encoded data of the audio signal. In this way, the smoothness of the reconstructed signal playback by the second electronic device can be guaranteed.

[0055] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the first bitstream further includes an index file, the index file includes index information corresponding to the model information of the decoding model.

[0056] For example, the target bitstream type is the type of bitstream encapsulated according to HLS, and the first bitstream may be HLS.

[0057] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the step of obtaining a first bitstream based on a second bitstream and index information corresponding to model information of a decoding model includes the step of generating an eighth bitstream based on index information corresponding to model information of a decoding model, and the step of encapsulating the second bitstream and the eighth bitstream in order to obtain the first bitstream. This is similar to the method of encapsulating the second bitstream and the third bitstream in order to obtain the first bitstream. Further details are not described here again.

[0058] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the third bitstream includes model information description information and model information, or the third bitstream includes model information description information and encoded data of model information.

[0059] According to the first embodiment or any one of the aforementioned implementations of the first embodiment, the model information includes model structure information or model parameters.

[0060] The model structure information may be the network structure information of a neural network, and the network structure information may include the connection relationships between network layers. The decoding model may include multiple types of network layers (e.g., convolutional layers, fully connected layers, recurrent neural network (RNN) layers, and activation layers), and one or more network layers of each type may be present.

[0061] For example, model parameters may be network parameters of a neural network, and network parameters may be weight matrices and bias matrices corresponding to network layers. The weight matrix contains weight vectors for multiple network layers, and the bias matrix contains bias vectors for multiple network layers. Each network layer may contain one or more neurons. One value in the weight vector of a network layer represents the weight value of one neuron in the network layer, and one value in the bias vector of a network layer represents the bias value of one neuron in the network layer.

[0062] In one possible approach, the model information for the decoding model may be a model file for the decoding model, i.e., it may include connections between all network layers of the decoding model and the network parameters of all network layers.

[0063] In one possible approach, the model information of the decoding model may be the connections between the updated network layer and / or the network parameters of the updated network layer of the decoding model.

[0064] According to either the first embodiment or the aforementioned implementation of the first embodiment, the pre-configured identifier value further represents the type of decoding model. In this way, the second electronic device learns whether it needs to acquire model information of the decoding model and perform model reconstruction.

[0065] According to a second aspect, one embodiment of the present application provides a decoding method. The decoding method includes the steps of: first receiving a first bitstream, the first bitstream comprising a model identifier and a second bitstream, the second bitstream being generated based on an encoded signal; obtaining model information when the value of the model identifier is a pre-set identifier value; then performing a model reconstruction based on the model information to obtain a decoded model; and then decoding the second bitstream based on the decoded model to obtain a reconstructed signal.

[0066] The decoding method may also be applied to a second electronic device. In this way, when it is determined that the model identifier is a pre-configured identifier value, the second electronic device retrieves the model information, reconstructs the model, and synchronously updates the decoding model and the encoding model of the first electronic device, or synchronously switches the decoding model used for decoding and the encoding model used by the first electronic device for encoding. Thus, the long-term evolution of the coding standard and the optimization of coding performance are facilitated.

[0067] For example, the reconstructed signal may include, but is not limited to, a reconstructed audio signal, a reconstructed image, or a reconstructed video signal.

[0068] According to a second embodiment, when the value of the model identifier is a pre-set identifier value, the first bitstream further includes a third bitstream, the third bitstream is generated based on model information of the decoding model, and the step of obtaining the model information includes the step of analyzing the third bitstream from the first bitstream and the step of obtaining model information of the decoding model based on the third bitstream.

[0069] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, when the value of the model identifier is a pre-configured identifier value, the first bitstream further includes index information corresponding to the model information of the decryption model, and the step of obtaining the model information includes parsing the index information corresponding to the model information of the decryption model from the first bitstream, and obtaining the model information of the decryption model based on the index information corresponding to the model information of the decryption model.

[0070] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the pre-configured identifier value indicates that the model information of the decryption model is stored in a pre-configured storage location within the remote electronic device, and the step of obtaining the model information includes the step of obtaining the model information of the decryption model from the pre-configured storage location within the remote electronic device.

[0071] The remote electronic device may also be the third electronic device mentioned above.

[0072] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the model identifier is an ES field, and the method further includes the step of parsing the field corresponding to the model identifier in the first bitstream in order to obtain the value of the model identifier.

[0073] According to the second aspect or any one of the aforementioned implementations of the second aspect, the target bitstream type of the first bitstream is a type of bitstream encapsulated according to a target container format or a type of bitstream encapsulated according to a target transport protocol, the model identifier is a field in the target container format, and the method further includes the steps of decapsulating the first bitstream according to the target container format in order to obtain a field corresponding to the model identifier, and parsing the field corresponding to the model identifier in order to obtain a value of the model identifier.

[0074] According to the second aspect or any one of the aforementioned implementations of the second aspect, the target bitstream type of the first bitstream is the type of bitstream encapsulated according to the target transport protocol, the model identifier is a field in the target transport protocol, and the method further includes the steps of decapsulating the first bitstream according to the target transport protocol to obtain the field corresponding to the model identifier, and parsing the field corresponding to the model identifier to obtain the value of the model identifier.

[0075] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the model identifier represents the type of decoding model, the method includes the step of determining whether there is a decoding model of the type corresponding to the pre-configured identifier value when the value of the model identifier is a pre-configured identifier value, and the step of reconstructing the model based on the model information in order to obtain a decoding model includes the step of reconstructing the model based on the model information when there is no decoding model of the type corresponding to the pre-configured identifier value in order to obtain a decoding model of the type corresponding to the pre-configured identifier value.

[0076] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the step of parsing a third bitstream from a second bitstream includes the step of parsing the first bitstream according to the target bitstream type of the first bitstream in order to obtain the third bitstream.

[0077] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the target bitstream type is an ES type, the first bitstream further comprises a first ES packet, the first ES packet comprises a second bitstream, a third bitstream, and a model identifier, and the step of parsing the first bitstream according to the target bitstream type of the first bitstream in order to obtain the third bitstream comprises the step of reading the third bitstream from the first ES packet, and the method further comprises the step of reading the second bitstream from the first ES packet.

[0078] For example, this application does not restrict the execution order in which the second bitstream is analyzed from the first bitstream and the model identifier value is analyzed from the first bitstream, nor does it restrict the execution order in which the second bitstream is analyzed from the first bitstream and the third bitstream is analyzed from the first bitstream.

[0079] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the target bitstream type is a type of bitstream encapsulated according to a target container format, and the step of parsing the first bitstream according to the target bitstream type of the first bitstream in order to obtain a third bitstream includes the step of decapsulating the first bitstream according to the target container format in order to obtain a second ES packet, and the step of reading the third bitstream from the second ES packet, and the method further includes the step of reading the second bitstream from the second ES packet.

[0080] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the target bitstream type is the type of bitstream encapsulated according to the target container format, and the step of parsing the first bitstream according to the target bitstream type of the first bitstream in order to obtain a third bitstream includes the step of decapsulating the first bitstream according to the target container format in order to obtain a third ES packet, a fourth ES packet, the type of the third ES packet, and the type of the fourth ES packet, and the step of reading the third bitstream from the third ES packet, wherein the type of the third ES packet is a pre-configured type, and the method further includes the step of reading the second bitstream from the fourth ES packet, wherein the type of the fourth ES packet is a media type.

[0081] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the preconfigured type is a metadata type.

[0082] According to the second aspect or any one of the aforementioned implementations of the second aspect, the target bitstream type is the type of bitstream encapsulated according to the target container format, the first bitstream further includes a fourth bitstream and a fifth bitstream, the fifth bitstream includes a second bitstream, the fourth bitstream includes a third bitstream, the model identifier is placed within the fourth bitstream or the fifth bitstream, and the step of parsing the first bitstream according to the target bitstream type of the first bitstream in order to obtain the third bitstream is: The method includes the steps of decapsulating a fourth bitstream according to a target container format in order to obtain a fifth ES packet and the type of the fifth ES packet, and reading a third bitstream from the fifth ES packet, wherein the type of the fifth ES packet is a pre-configured type, and further includes the steps of decapsulating a fifth bitstream according to a target container format in order to obtain a sixth ES packet and the type of the sixth ES packet, and reading a second bitstream from the sixth ES packet, wherein the type of the sixth ES packet is a media type.

[0083] According to the second aspect or any one of the aforementioned implementations of the second aspect, the target bitstream type is the type of bitstream encapsulated according to the target transport protocol, the first bitstream further includes a sixth bitstream and a seventh bitstream, the seventh bitstream includes a second bitstream, the sixth bitstream includes a third bitstream, the model identifier is placed within the sixth bitstream or the seventh bitstream, and the step of parsing the first bitstream according to the target bitstream type of the first bitstream in order to obtain the third bitstream is to parsing the first bitstream according to the target transport protocol in order to obtain the sixth bitstream The method includes the steps of decapsulating a bitstream, decapsulating a sixth bitstream according to a target container format to obtain a seventh ES packet and the type of the seventh ES packet, and reading a third bitstream from the seventh ES packet, wherein the type of the seventh ES packet is a pre-configured type. The method further includes the steps of decapsulating a seventh bitstream decapsulated from a first bitstream according to a target container format to obtain an eighth ES packet and the type of the eighth ES packet, and reading a second bitstream from the eighth ES packet, wherein the type of the eighth ES packet is a media type.

[0084] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the first bitstream further includes an index file, the index file includes index information for a sixth bitstream and index information for a seventh bitstream, the index information for the sixth bitstream precedes the index information for the seventh bitstream, and the step of decapsulating the first bitstream in accordance with the target transport protocol in order to obtain the sixth bitstream includes the step of decapsulating the first bitstream in accordance with the target transport protocol in order to obtain the index file and a plurality of bitstreams, and the step of reading the index information for the sixth bitstream from the index file and determining the sixth bitstream from the plurality of bitstreams based on the index information for the sixth bitstream.

[0085] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the first bitstream further includes an index file, the attribute information of the initialization tags in the index file includes index information of the sixth bitstream, and the step of decapsulating the first bitstream in accordance with the target transport protocol in order to obtain the sixth bitstream includes the step of decapsulating the first bitstream in accordance with the target transport protocol in order to obtain the index file and a plurality of bitstreams, and the step of reading the index information of the sixth bitstream from the attribute information of the initialization tags in the index file, and determining the sixth bitstream from the plurality of bitstreams based on the index information of the sixth bitstream.

[0086] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the reconstructed signal comprises M frames, M first ES packets, each of the M first ES packets comprises N ninth ES packets and MN tenth ES packets, each ninth ES packet comprises one sub-bitstream and one second bitstream, each tenth ES packet comprises one second bitstream, M and N are positive integers, M is greater than or equal to N, and the step of reading a third bitstream from a first ES packet comprises the step of reading N sub-bitstreams from N ninth ES packets and the step of joining the N sub-bitstreams to obtain a third bitstream, and the step of reading a second bitstream from a first ES packet comprises the step of reading N second bitstreams from N ninth ES packets and using MN tenth ES packets as MN second bitstreams.

[0087] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the first bitstream further includes an index file, the index file includes index information corresponding to model information of a decoding model, and the step of parsing the index information corresponding to model information of a decoding model from the first bitstream includes the step of decapsulating the first bitstream according to a target transport format in order to obtain the index file, and the step of reading the index information corresponding to model information of a decoding model from the index file.

[0088] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the step of determining model information based on a third bitstream includes the step of analyzing model information from the third bitstream, or the step of analyzing encoded model information from the third bitstream and decoding the encoded model information in order to obtain model information.

[0089] According to the second embodiment or any one of the aforementioned implementations of the second embodiment, the model information includes model structure information or model parameters.

[0090] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect, respectively. For technical effects corresponding to the second aspect and any implementation of the second aspect, please refer to the technical effects corresponding to the first aspect and any implementation of the first aspect. Further details will not be explained here.

[0091] According to a third aspect, the present application provides an encoding device used in a first electronic device. The device is A signal acquisition module configured to acquire a signal to be encoded, An encoding module configured to generate a first bitstream, the first bitstream comprising a second bitstream and a model identifier, the second bitstream being obtained by encoding information to be encoded based on an encoding model, the value of the model identifier being a pre-configured identifier value, the pre-configured identifier value instructing a second electronic device to obtain model information of a decoding model, the decoding model being an encoding module corresponding to the encoding model and Includes.

[0092] It should be understood that the encoding device may perform the steps in the first embodiment and any implementation of the first embodiment. Further details are not described here.

[0093] According to a fourth aspect, the present application provides a decoding device. The device is A bitstream receiving module configured to receive a first bitstream, wherein the first bitstream includes a model identifier and a second bitstream, the second bitstream being generated based on an encoded signal, and A decapsulation module configured to retrieve model information when the value of the model identifier is a pre-configured identifier value, The decapsulation module is further configured to perform model reconstruction based on model information in order to obtain a decoded model, To obtain the reconstructed signal, a decoder configured to decode a second bitstream based on a decoding model is used. Includes.

[0094] For example, the decoding device may be used in a second electronic device.

[0095] It should be understood that the decoding device may perform the steps in the second embodiment and any implementation of the second embodiment. Further details are not described here.

[0096] According to a fifth aspect, one embodiment of the present application provides an electronic device including a memory and a processor. The memory is coupled to the processor. The memory stores program instructions. When the program instructions are executed by the processor, the electronic device is enabled to perform an encoding method performed by the first electronic device in the first aspect or any possible implementation of the first aspect.

[0097] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. For technical effects corresponding to the fifth aspect and any implementation of the fifth aspect, please refer to the technical effects corresponding to the first aspect and any implementation of the first aspect. Further details will not be explained here.

[0098] According to a sixth aspect, one embodiment of the present application provides an electronic device including a memory and a processor. The memory is coupled to the processor. The memory stores program instructions. When the program instructions are executed by the processor, the electronic device is enabled to perform a decoding method performed by the second electronic device in the second aspect or in any possible implementation of the second aspect.

[0099] The sixth aspect and any one of its implementations correspond to the second aspect and any implementation of the second aspect, respectively. For technical effects corresponding to the sixth aspect and any implementation of the sixth aspect, please refer to the technical effects corresponding to the second aspect and any implementation of the second aspect. Further details are not provided here.

[0100] According to a seventh aspect, one embodiment of the present application provides a chip comprising one or more interface circuits and one or more processors. The interface circuit is configured to receive signals from the memory of an electronic device and transmit signals to the processor. The signals include computer instructions stored in the memory. When the processor executes a computer instruction, a step of the encoding method in the first aspect or any possible implementation of the first aspect is performed.

[0101] The seventh aspect and any implementation of the seventh aspect correspond to the first aspect and any implementation of the first aspect, respectively. For technical effects corresponding to the seventh aspect and any implementation of the seventh aspect, please refer to the technical effects corresponding to the first aspect and any implementation of the first aspect. Further details will not be explained here.

[0102] According to the eighth aspect, one embodiment of the present application provides a chip including one or more interface circuits and one or more processors. The one or more processors receive or transmit data through the one or more interface circuits. When the one or more processors execute a computer instruction, the steps of the decoding method in the second aspect or any possible implementation of the second aspect are performed.

[0103] The eighth aspect and any implementation of the eighth aspect correspond to the second aspect and any implementation of the second aspect, respectively. For technical effects corresponding to the eighth aspect and any implementation of the eighth aspect, please refer to the technical effects corresponding to the second aspect and any implementation of the second aspect. Further details will not be explained here.

[0104] According to the ninth aspect, one embodiment of the present application provides a computer-readable storage medium for storing a computer program. When the computer program is run on a computer or processor, the computer or processor is enabled to perform an encoding method according to the first aspect or any possible implementation thereof.

[0105] The ninth aspect and any implementation of the ninth aspect correspond to the first aspect and any implementation of the first aspect, respectively. For technical effects corresponding to the ninth aspect and any implementation of the ninth aspect, please refer to the technical effects corresponding to the first aspect and any implementation of the first aspect. Further details will not be explained here.

[0106] According to a tenth aspect, one embodiment of the present application provides a computer-readable storage medium for storing a computer program. When the computer program is run on a computer or processor, the computer or processor is enabled to perform a decoding method according to a second aspect or any possible implementation of the second aspect.

[0107] The tenth aspect and any implementation of the tenth aspect correspond to the second aspect and any implementation of the second aspect, respectively. For technical effects corresponding to the tenth aspect and any implementation of the tenth aspect, please refer to the technical effects corresponding to the second aspect and any implementation of the second aspect. Further details will not be explained here.

[0108] According to the eleventh aspect, one embodiment of the present application provides a computer program product. The computer program product includes computer instructions. When the computer instructions are executed by a computer or processor, the computer or processor is enabled to perform an encoding method according to the first aspect or any possible implementation of the first aspect.

[0109] The eleventh aspect and any implementation of the eleventh aspect correspond to the first aspect and any implementation of the first aspect, respectively. For technical effects corresponding to the eleventh aspect and any implementation of the eleventh aspect, please refer to the technical effects corresponding to the first aspect and any implementation of the first aspect. Further details will not be explained here.

[0110] According to the twelfth aspect, one embodiment of the present application provides a computer program product. The computer program product includes computer instructions. When the computer instructions are executed by a computer or processor, the computer or processor is enabled to perform a decoding method according to the second aspect or any possible implementation of the second aspect.

[0111] The twelfth aspect and any implementation of the twelfth aspect correspond to the second aspect and any implementation of the second aspect, respectively. For technical effects corresponding to the twelfth aspect and any implementation of the twelfth aspect, please refer to the technical effects corresponding to the second aspect and any implementation of the second aspect. Further details will not be explained here.

[0112] According to a thirteenth aspect, one embodiment of the present application provides a bitstream generation method. The bitstream may be generated according to an encoding method in the first aspect and any implementation of the first aspect.

[0113] The 13th aspect and any implementation of the 13th aspect correspond to the 1st aspect and any implementation of the 1st aspect, respectively. For technical effects corresponding to the 13th aspect and any implementation of the 13th aspect, please refer to the technical effects corresponding to the 1st aspect and any implementation of the 1st aspect. Further details will not be explained here.

[0114] According to a fourteenth aspect, one embodiment of the present application provides a bitstream storage device. The device includes a receiver and at least one storage medium. The receiver is configured to receive a bitstream. The at least one storage medium is configured to store a bitstream. The bitstream is a first bitstream generated according to a first aspect and any implementation of a first aspect.

[0115] The 14th aspect and any implementation of the 14th aspect correspond to the first aspect and any implementation of the first aspect, respectively. For technical effects corresponding to the 14th aspect and any implementation of the 14th aspect, please refer to the technical effects corresponding to the first aspect and any implementation of the first aspect. Further details will not be explained here.

[0116] According to a fifteenth aspect, one embodiment of the present application provides a bitstream transmission device. The device includes a transmitter and at least one storage medium. The at least one storage medium is configured to store a bitstream. The bitstream is a first bitstream generated according to a first aspect and any implementation of the first aspect. The transmitter is configured to retrieve the bitstream from the storage medium and transmit the bitstream to a device-side device via the transmission medium.

[0117] The 15th aspect and any implementation of the 15th aspect correspond to the 1st aspect and any implementation of the 1st aspect, respectively. For technical effects corresponding to the 15th aspect and any implementation of the 15th aspect, please refer to the technical effects corresponding to the 1st aspect and any implementation of the 1st aspect. Further details will not be explained here.

[0118] According to the sixteenth aspect, one embodiment of the present application provides a bitstream distribution system. The system includes at least one storage medium configured to store at least one bitstream, wherein the at least one bitstream is a first bitstream generated according to the first aspect and any implementation of the first aspect, and a streaming media device configured to retrieve a target bitstream from the at least one storage medium and transmit the target bitstream to a device-side device, wherein the streaming media device includes a content server or a content distribution server.

[0119] The 16th aspect and any implementation of the 16th aspect correspond to the 1st aspect and any implementation of the 1st aspect, respectively. For technical effects corresponding to the 16th aspect and any implementation of the 16th aspect, please refer to the technical effects corresponding to the 1st aspect and any implementation of the 1st aspect. Further details will not be explained here.

[0120] According to the 17th aspect, one embodiment of the present application provides a compression system, which includes an encoder side and a decoder side.

[0121] The encoder is configured to acquire the signal to be encoded and generate a first bitstream, the first bitstream containing a second bitstream and a model identifier, the second bitstream being acquired by encoding the signal to be encoded based on an encoding model, the model identifier being a pre-configured identifier value which instructs a second electronic device to acquire model information for the decoding model, the decoding model corresponding to the encoding model.

[0122] The decoder is configured to receive a first bitstream, acquire model information when the value of the model identifier is a pre-set identifier value, perform model reconstruction based on the model information to obtain the decoded model, and decode a second bitstream based on the decoded model to obtain a reconstructed signal.

[0123] The 17th aspect and any implementation of the 17th aspect correspond to the 1st aspect and any implementation of the 1st aspect, and to the 2nd aspect and any implementation of the 2nd aspect, respectively. For the technical effects corresponding to the 17th aspect and any implementation of the 17th aspect, please refer to the technical effects corresponding to the 1st aspect and any implementation of the 1st aspect, and to the technical effects corresponding to the 2nd aspect and any implementation of the 2nd aspect. Further details will not be explained here again. [Brief explanation of the drawing]

[0124] [Figure 1A] This is a diagram showing the system framework. [Figure 1B] This is a diagram showing the system framework. [Figure 1C] This diagram shows the structure of the encoding module and the decoding module. [Figure 2] This is a diagram showing the encoding process. [Figure 3] This diagram shows the decryption process. [Figure 4] This diagram shows the audio encoding process. [Figure 5] This diagram shows the audio decoding process. [Figure 6A] This diagram shows the audio encoding process. [Figure 6B] This figure shows the model information bitstream. [Figure 6C] This figure shows the structure of the first bitstream. [Figure 7] This diagram shows the audio decoding process. [Figure 8] This diagram shows the audio encoding process. [Figure 9] This diagram shows the audio decoding process. [Figure 10A] This diagram shows the audio encoding process. [Figure 10B] This figure shows the structure of the first bitstream. [Figure 11] This diagram shows the audio decoding process. [Figure 12A] This diagram shows the audio encoding process. [Figure 12B] This is a diagram showing the metadata encapsulation process. [Figure 13] This diagram shows the audio decoding process. [Figure 14A] This diagram shows the audio encoding process. [Figure 14B] This figure shows the structure of the first bitstream. [Figure 15] This diagram shows the audio decoding process. [Figure 16A] This diagram shows the audio encoding process. [Figure 16B] This figure shows the structure of the first bitstream. [Figure 17] This diagram shows the audio decoding process. [Figure 18] This is a diagram showing an encoding device. [Figure 19] This is a diagram showing a decoding device. [Figure 20] This is a diagram showing the structure of the device. [Modes for carrying out the invention]

[0125] The following clearly and completely describes the technical solutions in the embodiments of this application with reference to the accompanying drawings. It will be apparent that the embodiments described are only a part of, and not all, of the embodiments of this application. All other embodiments that can be obtained by those skilled in the art based on the embodiments of this application without creative effort shall fall within the scope of protection of this application.

[0126] In this specification, the term "and / or" describes only the relationship between the relevant objects, indicating that three relationships may exist. For example, A and / or B may represent the following three cases: that only A exists, that both A and B exist, and that only B exists.

[0127] In the specification and claims of the embodiments of this application, terms such as “first,” “second,” etc., are intended to distinguish different subjects, but not to indicate a particular order of subjects. For example, “first target subject” and “second target subject” are used to distinguish different target subjects, but not to describe a particular order of subjects.

[0128] In embodiments of this application, words such as “example” or “for example” are used to indicate an example, an illustrative example, or to provide an explanation. No embodiment or design described as “example” or “for example” in embodiments of this application is described as being more preferable or having more advantages than another embodiment or design. More precisely, the use of words such as “example” or “for example” is intended to present relative concepts in a specific manner.

[0129] In the description of embodiments of this application, unless otherwise specified, “multiple” means two or more. For example, “multiple processing units” means two or more processing units, and “multiple systems” means two or more systems.

[0130] Figure 1A shows the system framework. Figure 1A shows the encoding and decoding framework for media signals. Media signals may include, but are not limited to, audio signals, video signals, and images.

[0131] Please refer to Figure 1A. For example, the first electronic device may include a first input module, a first encoding module, a first channel encoding module, a first channel decoding module, a first decoding module, and a first playback module. It should be understood that the first electronic device may include more or fewer modules than those shown in Figure 1A. This is not limited to the present application.

[0132] Please refer to Figure 1A. For example, the second electronic device may include a second input module, a second encoding module, a second channel encoding module, a second channel decoding module, a second decoding module, and a second playback module. It should be understood that the second electronic device may include more or fewer modules than those shown in Figure 1A. This is not limited to the present application.

[0133] For example, the process by which a first electronic device encodes a media signal and transmits it to a second electronic device, and the second electronic device decodes and reproduces the media signal, may be as follows: A first capture module may capture the signal and output the media signal to a first encoding module. The first encoding module may then encode the media signal and output the bitstream to a first channel encoding module. The first channel encoding module may then perform channel encoding on the bitstream and transmit the bitstream obtained by channel encoding to a second electronic device via a wireless or wired network communication device. The second channel decoding module of the second electronic device may then perform channel decoding on the received data to obtain the bitstream and output the bitstream to a second decoding module. The second decoding module may then decode the bitstream to obtain a reconstructed signal and output the reconstructed signal to a second playback module, which then reproduces the reconstructed signal.

[0134] It should be understood that the process in which a second electronic device encodes a media signal and transmits it to a first electronic device, and the first electronic device decodes and reproduces the media signal, is similar to the process in which a first electronic device transmits a media signal to a second electronic device, and the second electronic device reproduces the media signal. Further details will not be explained here.

[0135] It should be understood that the first and second electronic devices may be directly connected to each other and communicate with each other without wireless or wired network communication devices. This is not limited to the present application.

[0136] For example, the first electronic device and the second electronic device may include, but are not limited to, personal computers, computer workstations, smartphones, tablet computers, servers, smart cameras, smart cars, other types of mobile phones, media consumption devices, wearable devices, set-top boxes, game consoles, and the like, respectively.

[0137] For example, the encoding and decoding framework in Figure 1A may be applied to any scenario where encoding and decoding are required, such as a VR (Virtual Reality) / AR (Augmented Reality) scenario. In a VR and AR scenario, in one possible scheme, the first electronic device is a server and the second electronic device is a VR / AR device, or in another possible scheme, the second electronic device is a server and the first electronic device is a VR / AR device.

[0138] For example, the first electronic device may be called the encoder side, and the second electronic device may be called the decoder side, or the second electronic device may be called the encoder side, and the first electronic device may be called the decoder side.

[0139] Figure 1B shows the system framework. Figure 1B shows the transcoding framework for media signals.

[0140] See (1) in Figure 1B. For example, the wireless or core network device may include a channel decoding module, another decoding module, an encoding module (i.e., the encoding module in this application), and a channel encoding module. The wireless or core network device may be used for transcoding.

[0141] For example, a specific application scenario of (1) in Figure 1B may be as follows: when the first electronic device has no encoding module but only another encoding module, and the second electronic device has only a decoding module but no other decoding module, the wireless or core network device may be used for transcoding so that the second electronic device can decode and reproduce the media signal encoded by the first electronic device via another encoding module.

[0142] Specifically, the first electronic device encodes the media signal via another encoding module to obtain bitstream 1, performs channel encoding on bitstream 1, and then transmits bitstream 1 to a wireless or core network device. The channel decoding module of the wireless or core network device may then perform channel decoding, or output bitstream 1 obtained by channel decoding to another decoding module. The other decoding module then decodes bitstream 1 to obtain the media signal and outputs the media signal to the encoding module. The encoding module then encodes the media signal to obtain bitstream 2 and outputs bitstream 2 to the channel encoding module. After performing channel encoding on bitstream 2, the channel encoding module transmits bitstream 2 to the second electronic device. In this way, the second electronic device can call the decoding module to decode bitstream 2 obtained by channel decoding to obtain a reconstructed signal, and the reconstructed signal can then be reconstructed.

[0143] See (2) in Figure 1B. For example, the wireless or core network device may include a channel decoding module, a decoding module (i.e., the coding module in this application), another coding module, and a channel coding module. The wireless or core network device may be used for transcoding.

[0144] For example, a specific application scenario of (2) in Figure 1B may be as follows: when the first electronic device comprises only an encoding module and not another encoding module, and the second electronic device comprises only another decoding module and not another decoding module, the wireless or core network device may be used for transcoding so that the second electronic device can decode and reproduce the media signal encoded by the first electronic device via the encoding module.

[0145] Specifically, the first electronic device encodes the media signal via an encoding module to obtain bitstream 1, performs channel encoding on bitstream 1, and then transmits bitstream 1 to a wireless or core network device. The channel decoding module of the wireless or core network device may then perform channel decoding and output bitstream 1 obtained by channel decoding to a decoding module. The decoding module then decodes bitstream 1 to obtain the media signal and outputs the media signal to another encoding module. The other encoding module then encodes the media signal to obtain bitstream 2 and outputs bitstream 2 to a channel encoding module. After performing channel encoding on bitstream 2, the channel encoding module transmits bitstream 2 to the second electronic device. In this way, the second electronic device can call another decoding module to decode bitstream 2 obtained by channel decoding to obtain a reconstructed signal, and the reconstructed signal can then be reconstructed.

[0146] Figure 1C shows the structure of the encoding module and the decoding module. In Figure 1C, the encoding module may be the first encoding module or the second encoding module in Figure 1A, or the encoding module in Figure 1B. In Figure 1C, the decoding module may be the first decoding module or the second decoding module in Figure 1A, or the decoding module in Figure 1B.

[0147] See Figure 1C(1). For example, the encoding module may include an encoder and an encapsulation module, the encoder being configured to encode a media signal based on an encoding model and output a bitstream, and the encapsulation model being configured to encapsulate the bitstream. Please understand that the encoding module in Figure 1C(1) is only an example of the present application. The encoding module in the present application may include more modules than those shown in Figure 1C(1). This is not limited to the present application.

[0148] See Figure 1C(2). For example, the decoding module may include a decoder and a decapsulation module, the decapsulation module being configured to decapsulate and analyze the received bitstream, and the decoder being configured to decode the bitstream based on a decoding model to obtain a reconstructed signal. Please understand that the decoding module in Figure 1C(2) is only an example of the present application. The decoding module in the present application may include more modules than those shown in Figure 1C(2). This is not limited to the present application.

[0149] For example, both encoding and decoding models may be implemented using neural networks. Therefore, encoding models are sometimes called AI (Artificial Intelligence) encoding models, and decoding models are sometimes called AI decoding models.

[0150] In one possible scheme, the coding model and the decoding model are two independent models. In this case, the coding model and the decoding model may be trained together. In another possible scheme, the coding model and the decoding model are two parts of the model.

[0151] Next, we will describe the media signal encoding and decoding process based on Figure 1C.

[0152] Figure 2 shows the encoding process. The encoding step in the embodiment of Figure 2 may be performed by a first electronic device.

[0153] S201: Acquire the signal to be encoded.

[0154] For example, the signal to be encoded may be a media signal, and a media signal includes, but is not limited to, audio signals, image signals, and video signals. The signal to be encoded may contain M frames, and M may be a positive integer. When the signal to be encoded is an image, M is equal to 1.

[0155] S202: Generate a first bitstream, which includes a second bitstream and a model identifier, the second bitstream being obtained by encoding the signal to be encoded based on the encoding model, the value of the model identifier being a pre-configured identifier value, the pre-configured identifier value instructing a second electronic device to obtain model information for the decoding model, the decoding model corresponding to the encoding model.

[0156] For example, an encoder within an encoding module may encode the signal to be encoded based on an encoding model in order to obtain a second bitstream. Note that the second bitstream is the bitstream output by the encoder (which may also be called the raw bitstream), and may contain the encoded data of the signal to be encoded (or the encoded data of the media signal).

[0157] For example, the encoder may perform preprocessing, such as transformation, on the signal to be encoded in order to obtain the first data, and then perform processing such as quantization and entropy coding on the first data in order to output a second bitstream.

[0158] Next, the first bitstream is generated based on the second bitstream. For example, an encapsulation module within an encoding module may generate the first bitstream based on the second bitstream.

[0159] In one possible scenario, if the first electronic device stores only one encoding model and one decoding model, after both the encoding and decoding models in the first electronic device have been updated, the encoding module in the first electronic device may set the value of the model identifier in the first bitstream to a pre-configured identifier value, which may instruct (or guide) the second electronic device to retrieve the model information of the decoding model. This can ensure that the decoding model of the second electronic device is updated synchronously. (It should be understood that in one possible scenario, the encoding model may not be updated, and only the decoding model is updated (e.g., the decoding model is optimized). In this case, the updated decoding model corresponds to the encoding model. In this case, after the decoding model in the first electronic device has been updated, the encoding module in the first electronic device may set the value of the model identifier in the first bitstream to a pre-configured identifier value. This application is illustrated by using an example in which both the encoding and decoding models are updated.)

[0160] In one possible scenario, if the first electronic device stores multiple encoding models and multiple decoding models (with each encoding model corresponding to a decoding model one-to-one), the encoder of the first electronic device may switch the encoding model used for encoding during the process of encoding the signal to be encoded. After the encoder of the first electronic device has switched the encoding model used for encoding (e.g., switching from encoding model 1 to encoding model 2), the encoding module in the first electronic device may set the value of the model identifier in the first bitstream to a pre-configured identifier value, which may instruct the second electronic device to retrieve the model information of the decoding model. This ensures that the decoder of the second electronic device can switch to the decoding model corresponding to the encoding model switched for decoding (e.g., switching from decoding model 1 to decoding model 2).

[0161] In one possible scenario, regardless of whether the encoding and decoding models in the first electronic device are updated, and whether the encoder in the first electronic device switches the encoding model used for encoding, the encoding module in the first electronic device may set the value of the model identifier in the first bitstream to a pre-configured identifier value. In this case, the pre-configured identifier value may represent the type of decoding model (or encoding model). Correspondingly, after obtaining the pre-configured identifier value, the second electronic device determines whether a decoding model of the type corresponding to the pre-configured identifier value exists in the second electronic device, and if no decoding model of the type corresponding to the pre-configured identifier value exists, it obtains the model information of the decoding model.

[0162] It should be understood that when the encoding model and the decoding model are two parts of a model, a pre-configured identifier value may instruct a second electronic device to retrieve the model information of the encoding model and the model information of the decoding model. In this way, the second electronic device can reconstruct the encoding model based on the model information of the encoding model and reconstruct the decoding model based on the model information of the decoding model. This application is illustrated by using an example in which the encoding model and the decoding model are two independent models.

[0163] In one possible scheme, an identifier (i.e., a flag) or field may be added to the first bitstream as a model identifier. In another possible scheme, an existing field in the first bitstream may be reused, or an existing field may be used as a model identifier. This is not limited to the present application. When an existing field in the first bitstream is reused, or an existing field may be used as a model identifier, the present application does not require a change in the decoding syntax, i.e., the model update or switch is performed while maintaining backward compatibility.

[0164] For example, the first electronic device may include a bitstream transmission device. The bitstream transmission device may include a transmitter and at least one storage medium. The at least one storage medium is configured to store the first bitstream generated in the embodiment of Figure 2. The transmitter is configured to retrieve the first bitstream from the storage medium and transmit the first bitstream to the device-side device via the transmission medium.

[0165] For example, after generating a first bitstream, the first electronic device may transmit the first bitstream to a bitstream distribution system. The bitstream distribution system may include at least one storage medium and a streaming media device. The storage medium is configured to store at least one first bitstream generated in the embodiment of Figure 2. The streaming media device is configured to retrieve a target bitstream from at least one storage medium and transmit the target bitstream to a device-side device, and the streaming media device includes a content server or content distribution server.

[0166] Figure 3 shows the decoding process. Figure 3 shows the decoding process corresponding to the encoding process in Figure 2. The decoding step in the embodiment of Figure 3 may be performed by a second electronic device. For example, the second electronic device may include a bitstream storage device. The bitstream storage device may include a receiver and at least one storage medium, wherein the receiver is configured to receive a first bitstream and at least one storage medium is configured to store the first bitstream. Next, referring to S302 to S304, the first bitstream may be decoded.

[0167] S301: Receive the first bitstream.

[0168] For example, after the first electronic device transmits the first bitstream, the second electronic device may receive the first bitstream.

[0169] For example, a decapsulation module within a decoding module of a second electronic device may parse a model identifier value from the first bitstream and then determine, based on the parsed value of the model identifier, whether the first electronic device updates the encoding and decoding models, or whether the encoder within the first electronic device switches the encoding model used for encoding.

[0170] S302: When the value of the model identifier is a pre-configured identifier value, retrieve the model information of the decrypted model.

[0171] For example, when the value of the model identifier is a pre-configured identifier value, it indicates that the first electronic device has updated the encoding and decoding models, or that the encoder of the first electronic device has switched the encoding model used for encoding. In this case, the second electronic device may obtain the model information for the decoding model.

[0172] For example, when the value of the model identifier is a different identifier value, it indicates that the first electronic device has not updated its encoding and decoding models, or that the encoder of the first electronic device has not switched the encoding model used for encoding. In this case, the second electronic device does not need to obtain model information for the decoding model. In this case, the decapsulation module in the decoding module of the second electronic device may parse the second bitstream from the first bitstream, and the decoder may then decode the second bitstream based on a decoding model of the type corresponding to the different identifier value (or a default decoding model) in order to obtain a reconstructed signal.

[0173] For example, regardless of whether the encoding and decoding models of the first electronic device are updated, and whether the encoder of the first electronic device switches the encoding model used for encoding, the first electronic device sets the value of the model identifier to a pre-configured identifier value. When the value of the model identifier parsed by the second electronic device is a pre-configured identifier value, it may be determined whether a decoding model of the type corresponding to the pre-configured identifier value exists in the second electronic device. If a decoding model of the type corresponding to the pre-configured identifier value does not exist in the second electronic device, S303 and S304 are performed. If a decoding model of the type corresponding to the pre-configured identifier value exists in the second electronic device, S303 does not need to be performed. After S304 is performed, the second bitstream may be decoded directly based on a decoding model of the type corresponding to the pre-configured identifier value.

[0174] S303: To obtain the decoding model, the model is reconstructed based on the model information.

[0175] For example, the deencapsulation module may perform model reconstruction based on model information in order to obtain the decryption model.

[0176] S304: To obtain the reconstructed signal, the second bitstream is decoded based on the decoding model.

[0177] For example, the decapsulation module may further analyze the second bitstream from the first bitstream and send the second bitstream to the decoder. It should be understood that the order in which the analysis of the second bitstream from the first bitstream to obtain model information, or the analysis of the model identifier value from the first bitstream, is not limited in this application.

[0178] The decoder may then read the reconstructed decoding model (which may also be called the decoder initialization) and then decode a second bitstream based on the reconstructed decoding model to obtain the reconstructed signal. The decoder can then perform operations such as reproducing the reconstructed signal.

[0179] For example, the decoder may perform operations such as entropy decoding and inverse quantization on the second bitstream based on the reconstructed decoding model to obtain second data, and then perform post-processing such as inverse transform on the second data to obtain a reconstructed signal. The reconstructed signal may include, but is not limited to, a reconstructed audio signal, a reconstructed image, or a reconstructed video signal.

[0180] In one possible approach, if the first electronic device does not update the encoding and decoding models, or does not switch the encoding model used for encoding, the first bitstream may not contain a model identifier. In this way, if the second electronic device fails to parse the model identifier from the first bitstream, the second bitstream parsed from the first bitstream can be decoded based on a default decoding model to obtain a reconstructed signal.

[0181] In this way, the first electronic device sets the value of the model identifier in the first bitstream to a preset identifier value in order to instruct the second electronic device to obtain model information of the decoding model. After the first bitstream has been transmitted to the second electronic device, the second electronic device may obtain model information of the decoding model based on the preset identifier value in order to synchronously update the decoding model or to synchronously switch between the decoding model used for decoding and the encoding model used for encoding.

[0182] In one possible configuration, the first electronic device may transmit model information of the decoding model to the second electronic device. Specifically, a third bitstream may be generated based on the model information of the decoding model. The second and third bitstreams are then encapsulated to obtain the first bitstream. In this case, a pre-configured identifier value may indicate that the first bitstream contains the third bitstream. Thus, the pre-configured identifier value may instruct the second electronic device to obtain the model information of the decoding model from the first bitstream.

[0183] In this application, we will use an example below in which the signal to be encoded is an audio signal.

[0184] In the embodiments shown in Figures 4 to 17, the signal to be encoded is an audio signal, the second bitstream is an audio bitstream, and the third bitstream is a model information bitstream.

[0185] Figure 4 shows the audio encoding process.

[0186] S401: Acquires the audio signal.

[0187] For example, an audio signal to be encoded may be obtained, and the audio signal may contain M frames, where M is a positive integer.

[0188] S402: Encode the audio signal based on the encoding model in order to obtain the audio bitstream.

[0189] For example, an audio encoder may encode an audio signal based on an encoding model in order to obtain an audio bitstream. The audio bitstream is the bitstream output by the audio encoder, and the audio bitstream may contain encoded data of the audio signal.

[0190] For example, an audio encoder may perform audio preprocessing, such as transformation, on an audio signal to obtain a first data, and then perform processing such as quantization and entropy coding on the first data to output an audio bitstream.

[0191] S403: A model information bitstream is generated based on the model information of the decoding model, and the decoding model corresponds to the encoding model.

[0192] S404: To obtain the first bitstream, the audio bitstream and the model information bitstream are encapsulated, the first bitstream containing a model identifier, the value of which is a pre-configured identifier value, and the pre-configured identifier value indicates that the first bitstream contains a model information bitstream.

[0193] In one possible scenario, the first electronic device stores only one encoding model and one decoding model. After both the encoding and decoding models in the first electronic device are updated, to ensure that the decoding model of the second electronic device is updated synchronously, the first electronic device may, after the audio encoder has first encoded an audio signal based on the updated encoding model to obtain an audio bitstream, generate a model information bitstream based on the model information of the updated decoding model, then encapsulate the audio bitstream and the model information bitstream to obtain the first bitstream, set the value of the model identifier in the first bitstream to a preset identifier value (the preset identifier value indicates that the first bitstream contains the model information bitstream), and then transmit the first bitstream to the second electronic device. In this way, the model information can be transmitted to the second electronic device, and the second electronic device reconstructs the decoding model based on the model information to update the decoding model in the second electronic device.

[0194] In one possible scenario, the first electronic device stores multiple encoding models and multiple decoding models (with one-to-one correspondence between encoding models and decoding models). In the process of encoding an audio signal, the first electronic device may switch the encoding model used for encoding. After the first electronic device has switched the encoding model used for encoding (e.g., switching from encoding model 1 to encoding model 2), in order to ensure that the second electronic device can switch to a decoding model corresponding to the switched encoding model for decoding (e.g., switching from decoding model 1 to decoding model 2), the first electronic device may, after encoding the audio signal based on the switched encoding model (e.g., encoding model 2), generate a model information bitstream based on the model information of the decoding model (e.g., decoding model 2) corresponding to the switched encoding model, then encapsulate the audio bitstream and the model information bitstream to obtain the first bitstream, set the value of the model identifier in the first bitstream to a preset identifier value, and then transmit the first bitstream to the second electronic device. In this way, the model information can be transmitted to a second electronic device, which, based on the model information, reconstructs a decoding model (e.g., decoding model 2) corresponding to the switched encoding model.

[0195] In one possible scenario, regardless of whether the coding and decoding models of the first electronic device are updated, and whether the first electronic device switches the coding model used for coding, the first electronic device may generate a model information bitstream based on the model information of the decoding model corresponding to the coding model used by the audio encoder to code the audio signal, then encapsulate the audio bitstream and the model information bitstream to obtain the first bitstream, set the value of the model identifier in the first bitstream to a pre-configured identifier value, and then transmit the first bitstream to the second electronic device. In this case, the pre-configured identifier value may further represent the type of coding model (or decoding model). Furthermore, if the second electronic device determines that a decoding model of the type corresponding to the pre-configured identifier value does not exist locally, it reconstructs the decoding model based on the model information.

[0196] It should be understood that the first electronic device may further generate a model information bitstream of the coding model based on the model information of the updated coding model (or switched coding model), encapsulate the model information bitstream in the first bitstream, and transmit the first bitstream to the second electronic device. In this way, the second electronic device can reconstruct the coding model based on the model information bitstream of the coding model, and as a result, when the second electronic device subsequently performs coding, it can code the audio signal based on the updated coding model (or switched coding model).

[0197] When the encoding model and the decoding model are two parts of a model, it should be understood that the first electronic device may encapsulate the model information bitstream of the encoding model, the model information bitstream of the decoding model, and the audio bitstream into a first bitstream, and then transmit the first bitstream to the second electronic device. The second electronic device may reconstruct the encoding model based on the model information bitstream of the encoding model and reconstruct the decoding model based on the model information bitstream of the decoding model. This application will be illustrated by using an example in which the encoding model and the decoding model are two independent models.

[0198] In one possible scheme, an existing field in the first bitstream may be used as the model identifier. When the first electronic device does not update the encoding and decoding models, or does not switch the encoding model used for encoding, i.e., when the first bitstream does not contain a model information bitstream, the model identifier may be set to a different identifier value. In this case, the different identifier value indicates that the first bitstream does not contain a model information bitstream.

[0199] In addition, if a pre-configured identifier value further represents the type of decoding model (or coding model), another identifier value may also represent the type of decoding model (or coding model). The type of decoding model (or coding model) represented by another identifier value is different from the type of decoding model (or coding model) represented by the pre-configured identifier value.

[0200] In one possible configuration, an additional field may be added to the first bitstream as a model identifier. When the first electronic device does not update the encoding and decoding models, or switches the encoding model used for encoding, i.e., when the first bitstream does not contain a model information bitstream, the additional field does not need to be added.

[0201] For example, the model information of a decoding model may include model structure information and / or model parameters.

[0202] The model structure information may be the network structure information of a neural network, and the network structure information may include the connection relationships between network layers. The decoding model may include multiple types of network layers (e.g., convolutional layers, fully connected layers, recurrent neural network (RNN) layers, and activation layers), and one or more network layers of each type may be present.

[0203] For example, model parameters may be network parameters of a neural network, and network parameters may be weight matrices and bias matrices corresponding to network layers. The weight matrix contains weight vectors for multiple network layers, and the bias matrix contains bias vectors for multiple network layers. Each network layer may contain one or more neurons. One value in the weight vector of a network layer represents the weight value of one neuron in the network layer, and one value in the bias vector of a network layer represents the bias value of one neuron in the network layer.

[0204] In one possible approach, the model information for the decoding model may be a model file for the decoding model, i.e., it may include connections between all network layers of the decoding model and the network parameters of all network layers.

[0205] In one possible approach, the model information of the decoding model may be the connections between the updated network layer and / or the network parameters of the updated network layer of the decoding model.

[0206] Note that S404 may be performed by the encapsulation module, and S403 may be performed by a module within the encoding module other than the audio encoder and the encapsulation module.

[0207] Figure 5 shows the audio decoding process. Figure 5 shows the decoding process corresponding to the audio encoding process in Figure 4.

[0208] S501: Receive the first bitstream.

[0209] For example, after the first electronic device transmits the first bitstream, the second electronic device may receive the first bitstream.

[0210] S502: Analyze the model identifier value from the first bitstream.

[0211] For example, the decapsulation module of the second electronic device may parse the value of a model identifier from the first bitstream and then determine, based on the parsed value of the model identifier, whether the first electronic device updates the encoding and decoding models, or whether the first electronic device switches the encoding model used for encoding.

[0212] S503: When the value of the model identifier is a pre-configured identifier value, the model information bitstream is analyzed from the first bitstream.

[0213] For example, when the value of the model identifier is a pre-configured identifier value, it indicates that the first electronic device has updated the encoding and decoding models, or that the first electronic device has switched the encoding model used for encoding, that is, it indicates that the first bitstream further includes a model information bitstream. In this way, the decapsulation module of the second electronic device can parse the model information bitstream from the first bitstream.

[0214] For example, when the value of the model identifier is a different identifier value, it indicates that the first electronic device has not updated the encoding and decoding models, or that the first electronic device has not switched the encoding model used for encoding, i.e., it indicates that the first bitstream does not contain a model information bitstream. In this case, the second electronic device may parse the audio bitstream from the first bitstream and then decode the audio bitstream based on a decoding model of the type corresponding to the different identifier value (or a default decoding model) in order to obtain a reconstructed audio signal.

[0215] For example, regardless of whether the encoding and decoding models of the first electronic device have been updated, and whether the first electronic device switches the encoding model used for encoding, when the first electronic device transmits a model information bitstream of the decoding model corresponding to the encoding model used by the audio encoder to encode an audio signal to the second electronic device, if the value of the model identifier is a pre-configured identifier value, it may be determined whether a decoding model of the type corresponding to the pre-configured identifier value exists in the second electronic device. If a decoding model of the type corresponding to the pre-configured identifier value does not exist in the second electronic device, steps S504 to S506 are performed. If a decoding model of the type corresponding to the pre-configured identifier value exists in the second electronic device, step S504 does not need to be performed. After step S505 is performed, the audio bitstream may be directly decoded based on the decoding model of the type corresponding to the pre-configured identifier value.

[0216] S504: To obtain the decoding model, the model is reconstructed based on the model information.

[0217] For example, the decapsulation module may determine the model information based on the model information bitstream, and then perform model reconstruction based on the model information to obtain the decoded model. Specifically, the model reconstruction may be performed based on the model structure information and / or model parameters contained in the model information to obtain the decoded model.

[0218] S505: Analyze the audio bitstream from the first bitstream.

[0219] For example, the decapsulation module may further analyze the audio bitstream from the first bitstream and send the audio bitstream to the audio decoder. It should be understood that the execution order of any one of S505 and S502-S504 is not limited in this application.

[0220] S506: Decode the audio bitstream based on the decoding model to obtain the reconstructed audio signal.

[0221] Next, the audio decoder may read out the reconstructed decoding model (which may also be called the audio decoder initialization), and then decode the audio bitstream based on the reconstructed decoding model to obtain the reconstructed audio signal. The second electronic device may then perform processing such as playing the reconstructed audio signal.

[0222] For example, the audio decoder may perform processes such as entropy decoding and inverse quantization on the audio bitstream based on the reconstructed decoding model to obtain a second data, and then perform audio post-processing such as inverse transform on the second data to obtain a reconstructed audio signal.

[0223] In one possible scenario, if the first electronic device does not update the encoding and decoding models, or does not switch the encoding model used for encoding, the first bitstream will not contain a model identifier or a model information bitstream. In this way, if the second electronic device fails to parse the model identifier from the first bitstream, the audio bitstream parsed from the first bitstream can be decoded based on the default decoding model to obtain a reconstructed audio signal.

[0224] For example, the first electronic device may encapsulate the audio bitstream and the model information bitstream according to the target bitstream type (i.e., the expected type of the first bitstream) in order to obtain the first bitstream. Correspondingly, the second electronic device may also parse the first bitstream according to the target bitstream type in order to obtain the model information bitstream, the audio bitstream, and the value of the model identifier.

[0225] For example, the target bitstream type includes one of the first, second, or third types. The first type is an ES type, the second type is a bitstream type encapsulated according to the target container format (e.g., a TS type or MP4 type), and the third type is a bitstream type encapsulated according to the target transport protocol (e.g., an HLS type or RTP type).

[0226] Figure 6A shows the audio encoding process. In the embodiment of Figure 6A, the target bitstream type is a first type. Correspondingly, the first bitstream is sometimes called the ES stream. The model information bitstream and the audio bitstream may be encapsulated in the same ES packet in order to obtain the first bitstream.

[0227] S601: Acquires the audio signal.

[0228] For example, an audio signal containing M frames may be obtained, where M is a positive integer.

[0229] S602: Determine whether both the encoding model and the decoding model have been updated.

[0230] S603: When both the encoding model and the decoding model are updated, encode the audio signal based on the updated encoding model to obtain the audio bitstream.

[0231] For example, an audio encoder may encode an audio signal containing M frames based on an updated encoding model in order to obtain M audio bitstreams.

[0232] S604: A model information bitstream is generated based on the model information of the updated decoding model, and the updated decoding model corresponds to the updated encoding model.

[0233] For example, model information for a decoding model corresponding to an updated coding model (i.e., the updated decoding model) may be obtained, or the model information for the updated decoding model may be divided into N (where N is a positive integer) information groups, and then one sub-bitstream may be generated based on each of the N information groups, or N sub-bitstreams may be obtained, that is, the model information bitstream may contain multiple sub-bitstreams.

[0234] For example, when N is equal to 1, the model information is either not divided, or the model information is divided into one information group. In this case, the model information bitstream may include the model information description information and the model information, or the model information bitstream may include the model information description information and the encoded data of the model information.

[0235] Figure 6B shows the model information bitstream. In (1) of Figure 6B, N=1, the model information description information is the model information length information, and the model information bitstream may include the model information length information and the model information.

[0236] For example, if N is greater than 1, each sub-bitstream may contain descriptive information and information groups for an information group, or each sub-bitstream may contain descriptive information and encoded data for an information group for an information group.

[0237] See (2) in Figure 6B. For example, in (2) in Figure 6B, N is greater than 1, and the descriptive information for each information group may include, but is not limited to, the total number of information groups N, the index of the current information group, and the length information of the current information group. The subbitstream may include, but is not limited to, the total number of information groups N, the index of the current information group, the length information of the current information group, and the information group.

[0238] Furthermore, in order to obtain encoded data of the model information, reversible encoding may be performed on the model information, and in order to obtain encoded data of the information group, reversible encoding may be performed on the information group. This ensures that the decoded model reconstructed by the second electronic device matches the decoded model of the first electronic device.

[0239] S605: The model information bitstream and audio bitstream are encapsulated in a first ES packet, and the first ES packet is used as the first bitstream.

[0240] For example, the model information bitstream and the audio bitstream may be encapsulated in the same ES packet to obtain the first ES packet, which is then used as the first bitstream. In this case, the first bitstream may also be called the ES stream.

[0241] For example, there may be M first ES packets, and these M first ES packets may contain N ninth ES packets and MN tenth ES packets, where M and N are positive integers and M is greater than or equal to N.

[0242] Specifically, N sub-bitstreams and N audio bitstreams may be encapsulated in N 9th ES packets, and other MN audio bitstreams may be used as MN 10th ES packets. One sub-bitstream and one audio bitstream may be encapsulated in one 9th ES packet, i.e., the 9th ES packet may contain one sub-bitstream and one audio bitstream. In addition, one of the other MN audio bitstreams may be used as a 10th ES packet, i.e., the 10th ES packet may contain one audio bitstream. Note that one audio bitstream may be obtained by encoding one frame of the audio signal based on the encoding model.

[0243] Figure 6C shows the structure of the first bitstream.

[0244] See (1) in Figure 6C. For example, when N is equal to 1, the model information bitstream and one audio bitstream may be encapsulated into one ES packet to obtain one ninth ES packet. Specifically, the model information bitstream and the first audio bitstream are encapsulated into one ES packet to obtain one ninth ES packet, and the other M-1 audio bitstreams are used as M-1 tenth ES packets. For example, in the ninth ES packet, the model information bitstream may come after the first audio bitstream.

[0245] See (2) in Figure 6C. For example, when N is greater than 1, one sub-bitstream and one audio bitstream may be encapsulated in one ES packet to obtain one ninth ES packet. Specifically, N sub-bitstreams and N audio bitstreams are encapsulated in N ninth ES packets, and the other MN audio bitstreams are used as MN tenth ES packets. For example, in each ninth ES packet, the sub-bitstream may follow the audio bitstream.

[0246] It should be noted that N may be determined based on the amount of model information (or encoded model information), network bandwidth, and encoded audio signal data in order to ensure the smooth reproduction of the reconstructed audio signal by the second electronic device.

[0247] For example, in the embodiment of Figure 6A, the model identifier may be an ES field. For example, the ES encapsulation format may include multiple fields, and the ES field may be a field included in the ES encapsulation format. In other words, an ES packet may be obtained by encapsulating data according to the ES encapsulation format. The ES packet may include multiple fields, and the ES field may be a field within the ES packet.

[0248] For example, the audio bitstream output by an audio encoder may include a header (i.e., the header of the ES packet) and encoded audio data (sometimes called the audio data payload).

[0249] For example, the header of an audio bitstream may include a field indicating the type of decoding model (or encoding model). If there are unused identifier values ​​in the field, the audio encoder may use the field as a model identifier indicating whether the first bitstream contains a model information bitstream, and may select a pre-configured identifier value from the unused identifier values ​​corresponding to the field.

[0250] For example, in the Audio Vivid standard, the field indicating the type of decoding model (or encoding model) is the nn_type field. The nn_type field may be represented using 3 bits, i.e., the nn_type field corresponds to eight identifier values: "0", "1", "2", "3", "4", "5", "6", and "7". Currently, "0" and "1" are identifier values ​​in use, while "2", "3", "4", "5", "6", and "7" are reserved identifier values, i.e., unused identifier values. The definition of the nn_type field may be shown in Table 1 below:

[0251] [Table 1]

[0252] Additionally, the nn_type field may be used as a model identifier. For example, "2" may be used as a pre-configured identifier value. In this case, the definition of the nn_type field is shown in Table 2:

[0253] [Table 2]

[0254] For example, different identifier values ​​within the unused identifier values ​​corresponding to the nn_type field may represent different types of encoding (or decoding) models; that is, different pre-configured identifier values ​​may represent different types of encoding (or decoding) models. This may be illustrated in Table 3:

[0255] [Table 3]

[0256] Please refer to Table 3. Identifier value "2" may represent user-defined model 1, identifier value "3" may represent user-defined model 3, and identifier value "4" may represent user-defined model 3.

[0257] For example, in another standard, the audio encoder may add an additional field to the header of the audio bitstream, use the additional field as a model identifier indicating whether the first bitstream contains a model information bitstream, and set the field value of the additional field to a pre-configured identifier value.

[0258] For example, only the model identifier in the first ninth ES packet (i.e., the ninth ES packet containing the first audio bitstream) may be set to the pre-configured identifier value, or the model identifiers in some or all of the N ninth ES packets and MN tenth ES packets may be set to the pre-configured identifier value. This is not limited to the present application.

[0259] In this way, after an ES packet is acquired, it can be transmitted directly without being encapsulated according to the target container format or target transport protocol, thereby reducing bitrate overhead and bandwidth requirements. In addition, the second and third bitstreams are encapsulated in the same ES packet, and as a result, the second and third bitstreams can be transmitted on the same channel, thereby reducing transmission complexity.

[0260] Figure 7 shows the audio decoding process. Figure 7 shows the audio decoding process corresponding to the audio encoding process in Figure 6A.

[0261] S701: Receives the first bitstream.

[0262] For example, when the first electronic device performs encoding according to the audio encoding method of the embodiment in Figure 6A, the first bitstream received by the second electronic device is essentially an ES stream. In this case, the first bitstream may include first ES packets. It is assumed that the reconstructed audio signal contains M frames and M first ES packets, and that the M first ES packets may contain N ninth ES packets and MN tenth ES packets.

[0263] S702: Analyze the value of the model identifier from the field corresponding to the model identifier in the first ES packet.

[0264] For example, the model identifier is an ES field.

[0265] For example, after a second electronic device receives the first bitstream, the decapsulation module may parse each ES packet to parse the value of the model identifier from the ES packet. Of the multiple ES packets contained in the first bitstream, at least one ES packet contains a field corresponding to the model identifier. Therefore, the decapsulation module may parse the value of the model identifier from the field corresponding to the model identifier of at least one ES packet (the ninth or tenth ES packet).

[0266] S703: When the value of the model identifier is a pre-configured identifier value, read the model information bitstream from the 9th ES packet.

[0267] For example, when the value of the model identifier is a pre-configured identifier value, the decapsulation module may parse the header of each ES packet to determine the length of the encoded audio signal data, and then determine whether the ES packet contains a model information bitstream (or sub-bitstream) based on the length of the header, the length of the encoded audio signal data, and the total length of the ES packet. If the total length of the ES packet is greater than the sum of the length of the encoded audio signal data and the length of the header, the ES packet is determined to contain a model information bitstream (or sub-bitstream), i.e., the ES packet is the ninth ES packet. If the total length of the ES packet is equal to the sum of the length of the encoded audio signal data and the length of the header, the ES packet is determined not to contain a model information bitstream (or sub-bitstream), i.e., the ES packet is the tenth ES packet.

[0268] For example, for one ninth ES packet, the model information bitstream (or sub-bitstream) may be read without reading the encoded data portion of the audio signal, based on the length of the encoded data of the audio signal. When N is equal to 1, the model information bitstream may be read directly from the ninth ES packet. When N is greater than 1, N sub-bitstreams may be read, or the N sub-bitstreams may form the model information bitstream.

[0269] S704: To obtain the decoding model, perform model reconstruction based on the model information.

[0270] For example, when N is equal to 1, the decapsulation module may analyze the description information of the model information in the model information bitstream to determine the length of the model information (or the encoded data of the model information), and then read the model information (or the encoded data of the model information) based on the length of the model information (or the encoded data of the model information).

[0271] For example, when N is greater than 1, for each sub-bitstream, the decapsulation module analyzes the description information of the information group in the sub-bitstream to determine the length of the current information group (or the encoded data of the current information group), the index corresponding to the current information group, and the total number N of information groups, and then may read out the information group (or the encoded data of the current information group) from the sub-bitstream based on the length of the current information group (or the encoded data of the current information group). In this way, after N information groups are obtained, the N information groups can be joined based on the index corresponding to the information group to obtain model information. Alternatively, after the encoded data of the N information groups is obtained, the encoded data of the N information groups may be decoded to obtain the N information groups. Then, the N information groups may be joined based on the index corresponding to the information group to obtain model information.

[0272] For example, when the encoded data of the model information (or information group) is read, the decapsulation module calls a decoder to decode the encoded data of the model information (or information group), determines the model information (or information group), and then may perform model reconstruction based on the model information (or joined information group) to obtain a decoded model.

[0273] S705: Read an audio bitstream from the ninth ES packet and use the tenth ES packet as the audio bitstream.

[0274] In one possible method, after the header is parsed for a ninth ES packet to determine the length of the encoded audio signal data, the portion of the ninth ES packet corresponding to the encoded audio signal data may be determined based on the length of the encoded audio signal data, and then the header and the portion corresponding to the encoded audio signal data may be extracted from the ninth ES packet to obtain an audio bitstream. The tenth ES packet may be used as the audio bitstream, and then the audio bitstream is sent to an audio decoder.

[0275] In one possible method, the header of a ninth ES packet may be parsed to determine the length of the encoded audio signal data, and then the encoded audio signal data may be read from the ninth ES packet based on the length of the encoded audio signal data. For each tenth ES packet, the header of the tenth ES packet may be parsed, and after the length of the encoded audio signal data is determined, the encoded audio signal data may be read from the tenth ES packet based on the length of the encoded audio signal data. The encoded audio signal data is then sent to the audio decoder.

[0276] S706: Decode the audio bitstream based on the decoding model to obtain the reconstructed audio signal.

[0277] For example, if a decapsulation module sends an audio bitstream to an audio decoder, the audio decoder may read out the reconstructed decoding model, then first read out the encoded data of the audio signal from the audio bitstream, and then decode the encoded data of the audio signal based on the decoding model to obtain the reconstructed audio signal. Alternatively, if a decapsulation module sends encoded data of an audio signal to an audio decoding module, the audio decoding module may read out the reconstructed decoding model and then directly decode the encoded data of the audio signal based on the decoding model to obtain the reconstructed audio signal.

[0278] It should be understood that when N is equal to 1, after the model information bitstream has been parsed from the first ninth ES packet, model reconstruction may be performed to obtain the decoded model. The audio decoder may then decode the first audio bitstream based on the decoded model to obtain the first frame of the reconstructed audio signal. Then, each time the second electronic device receives the tenth ES packet (from the second first ES packet to the Mth first ES packet), the audio decoder may perform real-time decoding on the tenth ES packet to obtain the second to Mth frames of the reconstructed audio signal.

[0279] When N is greater than 1, the second electronic device can only reconstruct the model based on the N subbitstreams after analyzing the N subbitstreams from the 1st to the Nth ES packets in order to obtain the decoding model. The audio decoder may then decode the audio bitstreams from the 1st to the Nth based on the decoding model to obtain the first to the Nth frames of the reconstructed audio signal. Then, each time the second electronic device receives the 10th ES packet (from the (M-N+1)th first ES packet to the Mth first ES packet), the audio decoder may perform real-time decoding on the 10th ES packet to obtain the (M-N+1)th frame to the Mth frame of the reconstructed audio signal. Specifically, in this case, the decoding of the audio bitstreams within the N frames of the 9th ES packet by the audio decoder is delayed, or the decoding delay of the audio decoder is N frames.

[0280] Figure 8 illustrates the audio encoding process. In Figure 8, the target bitstream type is the second type. The audio bitstream and model information bitstream may be encapsulated in the same ES packet, or the ES packet may be encapsulated according to the target container format in order to obtain the first bitstream.

[0281] S801: Acquires the audio signal.

[0282] S802: Determine whether both the encoding model and the decoding model have been updated.

[0283] S803: When both the encoding model and the decoding model are updated, encode the audio signal based on the updated encoding model in order to obtain the audio bitstream.

[0284] S804: A model information bitstream is generated based on the model information of the updated decoding model, and the updated decoding model corresponds to the updated encoding model.

[0285] S805: Encapsulate the model information bitstream and audio bitstream into a second ES packet.

[0286] For example, for S801 to S805, please refer to the explanations above for S601 to S605. Further details will not be explained here again. The format of the second ES packet is the same as that of the first ES packet; that is, M second ES packets contain N ninth ES packets and MN tenth ES packets. Further details will not be explained here again.

[0287] S806: To obtain the first bitstream, encapsulate the second ES packet according to the target container format.

[0288] For example, the second ES packet may be encapsulated according to the target container format, and the type of the second ES packet may be set to audio type in order to obtain the first bitstream.

[0289] In the process of encapsulating a second ES packet according to the target container format, the type identifier of the ES packet may be set to the first identifier value, where the first identifier value indicates that the type of the ES packet is audio. The type identifier may also be a field in the target container format that represents the type of the ES packet.

[0290] For example, in the embodiment shown in Figure 8, in one possible configuration, the model identifier may be an ES field. For further details, please refer to the description of the embodiment shown in Figure 6A. Further details are not described here again.

[0291] In one possible approach, in the process of encapsulating the second ES packet according to the target container format, a field in the additional information (the field is a field in the target container format) may be used as the model identifier, and the value of the model identifier is set to the preset identifier value. For example, the target container format is TS / MP4, and a field in the additional information, such as the nn_type field, may be used as the model identifier, and the field value of the nn_type field is set to the preset identifier value. For details, refer to the description of the foregoing embodiment in FIG. 6A. Details will not be described again here. In this way, the second electronic device can determine whether the decoding model needs to be reconstructed without analyzing the ES packet.

[0292] The ES packet may be divided into a plurality of small packets, and the small packets are encapsulated according to the target container format. This facilitates the bitstream delivery of the bitstream delivery device and the synchronous audio and video control.

[0293] FIG. 9 shows an audio decoding process. FIG. 9 shows an audio decoding process corresponding to the audio encoding process in FIG. 8.

[0294] S901: Receive the first bitstream.

[0295] For example, when the first electronic device performs encoding according to the audio encoding method in the embodiment of FIG. 8, assuming that the target container format is the TS format, the first bitstream received by the second electronic device is essentially a TS stream, or assuming that the target container format is the MP4 format, the first bitstream received by the second electronic device is essentially an MP4 stream. This application is described by using an example where the first bitstream is a TS stream.

[0296] For example, when the model identifier is a field in the target container format, see S902 and S903 for the process of parsing the model identifier from the first bitstream.

[0297] S902: Decapsulate the first bitstream according to the target container format in order to obtain the field corresponding to the model identifier.

[0298] S903: To obtain the value of the model identifier, parse the field corresponding to the model identifier.

[0299] For example, after a second electronic device receives a first bitstream, the decapsulation module may decapsulate the first bitstream to obtain fields corresponding to a model identifier and M second ES packets (the M second ES packets include N ninth ES packets and MN tenth ES packets). The decapsulation module may then parse the fields corresponding to the model identifier to obtain the value of the model identifier.

[0300] For example, when the model identifier is an ES field, the value of the model identifier may be parsed from the field corresponding to the model identifier in the ES packet. See the explanation above in S702 for details. Further details will not be explained here.

[0301] S904: When the value of the model identifier is a pre-configured identifier value, read the model information bitstream from the 9th ES packet.

[0302] S905: To obtain the decoded model, the model is reconstructed based on the model information bitstream.

[0303] S906: Read the audio bitstream from the 9th ES packet and use the 10th ES packet as the audio bitstream.

[0304] S907: Decode the audio bitstream based on the decoding model to obtain the reconstructed audio signal.

[0305] For example, for S904-S907, please refer to the explanations in S503-S506 mentioned above. Further details will not be explained here.

[0306] Figure 10A shows the audio encoding process. In Figure 10A, the target bitstream type is the second type. The audio bitstream and the model information bitstream may each be encapsulated in different ES packets, or the ES packets of the model information bitstream and the audio bitstream may be encapsulated together according to the target container format in order to obtain the first bitstream.

[0307] S1001: Acquires the audio signal.

[0308] S1002: Determine whether both the encoding model and the decoding model have been updated.

[0309] S1003: When both the encoding model and the decoding model are updated, encode the audio signal based on the updated encoding model to obtain the audio bitstream.

[0310] S1004: A model information bitstream is generated based on the model information of the updated decoding model, and the updated decoding model corresponds to the updated encoding model.

[0311] For example, for S1001 to S1004, please refer to the explanations for S601 to S604 mentioned above. Further details will not be explained here.

[0312] S1005: Encapsulate the model information bitstream into a third ES packet.

[0313] For example, model information may be divided into N information groups. Then, one information group is encapsulated in one third ES packet. In this way, N third ES packets can be obtained.

[0314] For example, when N is equal to 1, there is one third ES packet, which contains a model information bitstream and a description of the model information. For further details, please refer to the description of the embodiments above. Details will not be explained again here.

[0315] For example, when N is greater than 1, there are N third ES packets, each containing a sub-bitstream and sub-bitstream description information. For further details, please refer to the description of the embodiments above. Details will not be explained again here.

[0316] S1006: Use the audio bitstream as the fourth ES packet.

[0317] For example, one audio bitstream may be used as one fourth ES packet. In this way, N fourth ES packets can be obtained.

[0318] S1007: To obtain the first bitstream, encapsulate the third and fourth ES packets according to the target container format, set the type of the third ES packet to a pre-configured type, and set the type of the fourth ES packet to an audio type.

[0319] For example, in the process of encapsulating a fourth ES packet and a third ES packet according to the target container format, the type identifier of the fourth ES packet may be set to the first identifier value, where the first identifier value indicates that the type of the ES packet is audio.

[0320] For example, in the process of encapsulating the fourth and third ES packets according to the target container format, the type identifier of the third ES packet may be set to a second identifier value, where the second identifier value indicates that the type of the ES packet is a pre-configured type. The model identifier indicates that the ES packet contains a model information bitstream or sub-bitstream. For further details, please refer to the description of the embodiments above. Further details are not described again here.

[0321] In one possible configuration, the pre-configured type may be a private type. In this case, the private type identifier corresponding to the third ES packet may include private_stream_id_1 and private_stream_id_2. Correspondingly, the private type identifier private_stream_id_1 corresponding to the third ES packet may be set to 0xBD, or the private type identifier private_stream_id_2 corresponding to the third ES packet may be set to 0xBF.

[0322] In one possible approach, the pre-configured type may be a user-defined type. In this case, the user-defined type identifier corresponding to the third ES packet may include stream_id and stream_type. The user-defined type identifiers stream_id and stream_type corresponding to the third ES packet may be set to the second identifier value.

[0323] For example, in the embodiment of Figure 10A, the model identifier may be an ES field or a field in the target container format. This is not limited to the present application. See the above description for further details. Further details are not described here again.

[0324] For example, when the model identifier is an ES field, the model identifier may be placed in at least one third ES packet or at least one fourth ES packet.

[0325] Refer to Figure 10B. For example, in Figure 10B, the target container format is TS, N=1, and the third and fourth ES packets may be encapsulated in a TS stream. In this case, the first bitstream is a TS stream.

[0326] Figure 11 shows the audio decoding process. Figure 11 shows the audio decoding process corresponding to the audio encoding process in Figure 10A.

[0327] S1101: Receive the first bitstream.

[0328] For example, when the model identifier is a field in the target container format, see S1102 and S1103 for the process of parsing the model identifier from the first bitstream.

[0329] S1102: Decapsulate the first bitstream according to the target container format to obtain the fields corresponding to the model identifier, the third ES packet, and the fourth ES packet.

[0330] For example, the first bitstream is decapsulated according to the target container format to obtain fields corresponding to the model identifier, multiple ES packets, and the types of the ES packets. The decapsulation module may parse the type identifier of each ES packet. When the type identifier of an ES packet is determined to be a second identifier value, the decapsulation module may determine that the ES type is a pre-configured type. In this case, the ES packet may be called a third ES packet. For example, when the type identifier of an ES packet is determined to be a first identifier value, the decapsulation model may determine that the ES packet type is an audio type. In this case, the ES packet may be called a fourth ES packet.

[0331] S1103: To obtain the value of the model identifier, parse the field corresponding to the model identifier.

[0332] S1104: When the value of the model identifier is a pre-configured identifier value, read the model information bitstream from the third ES packet.

[0333] S1105: To obtain the decoding model, the model is reconstructed based on the model information.

[0334] S1106: Read the audio bitstream from the fourth ES packet.

[0335] S1107: Decode the audio bitstream based on the decoding model to obtain the reconstructed audio signal.

[0336] For example, for S1104-S1107, please refer to the explanations in S703-S706 mentioned above. Further details will not be explained here.

[0337] In this way, compared to encapsulating the second and third bitstreams in the same ES packet, the second electronic device does not need to decompose the ES packet to analyze the second and third bitstreams, thereby reducing the complexity of the analysis process.

[0338] Figure 12A illustrates the audio encoding process. In Figure 12A, the target bitstream type is the second type. The model information bitstream may be encapsulated as metadata, or the model information bitstream and audio bitstream may be encapsulated in different ES packets. The ES packets for the model information bitstream and the ES packets for the audio bitstream are then encapsulated together according to the target container format to obtain the first bitstream.

[0339] S1201: Acquires the audio signal.

[0340] S1202: Determine whether both the encoding model and the decoding model have been updated.

[0341] S1203: When both the encoding model and the decoding model are updated, encode the audio signal based on the updated encoding model to obtain the audio bitstream.

[0342] S1204: A model information bitstream is generated based on the model information of the updated decoding model, and the updated decoding model corresponds to the updated encoding model.

[0343] For example, regarding S1201 to S1204, please refer to the explanations for S601 to S604 mentioned above. Further details will not be explained here.

[0344] S1205: To obtain the third ES packet, the model information bitstream is encapsulated as metadata.

[0345] For example, model information may be divided into N information groups. Then, one information group is encapsulated in one third ES packet. In this way, N third ES packets can be obtained.

[0346] Refer to Figure 12B. For example, a model information bitstream may be used as a Metadata Access Unit (AU). When N is equal to 1, one metadata Metadata AU may be used as one Metadata AU Cell. Then, one Metadata AU Cell is encapsulated into one third ES packet in order to obtain one third ES packet. When N is greater than 1, one Metadata AU may be divided into N Metadata AU Cells. Then, one Metadata AU Cell is encapsulated into one third ES packet in order to obtain N third ES packets.

[0347] S1206: Use the audio bitstream as the fourth ES packet.

[0348] For example, one audio bitstream may be used as one fourth ES packet. In this way, N fourth ES packets can be obtained.

[0349] S1207: To obtain the first bitstream, encapsulate the third and fourth ES packets according to the target container format, set the type of the third ES packet to metadata type, and set the type of the fourth ES packet to audio type.

[0350] For example, the metadata type is one of the pre-configured types in the embodiment shown in Figure 11.

[0351] In Figure 12B (2), the target container format is TS. Based on Figure 12B (1), N third ES packets and M fourth ES packets are encapsulated in a single TS stream. Thus, the obtained first bitstream is a TS stream.

[0352] For example, in the process of encapsulating a fourth ES packet and a third ES packet according to the target container format, the type identifier of the fourth ES packet may be set to the first identifier value, and the type identifier of the third ES packet may be set to the third identifier value, where the third identifier value indicates that the type of the ES packet is a metadata type.

[0353] In the embodiment of Figure 12A, the syntax of the Metadata AU Cell in the first bitstream may be further modified. Assuming the target container format is TS, i.e., the first bitstream is a TS stream, the syntax of the Metadata AU Cell in the TS stream may be shown in Table 4 below:

[0354] [Table 4]

[0355] In Table 4, `decoder_config_flag` indicates whether the current Metadata AU contains decoder initialization information. In the embodiment shown in Figure 12A, the identifier value of `decoder_config_flag` in the syntax of the Metadata AU Cell separated from the Metadata AU corresponding to the model information bitstream is set to 1 to indicate that the model update information is necessary for the initialization of the audio decoder.

[0356] In the embodiment shown in Figure 12A, the model identifier may be an ES field or a field in the target container format. This is not limited to the present application.

[0357] For example, when the model identifier is an ES field, the model identifier may be placed in at least one third ES packet or at least one fourth ES packet.

[0358] Figure 13 shows the audio decoding process. Figure 13 shows the audio decoding process corresponding to the audio encoding process in Figure 12A.

[0359] S1301: Receive the first bitstream.

[0360] S1302: Decapsulate the first bitstream according to the target container format to obtain the fields corresponding to the model identifier, the third ES packet, and the fourth ES packet.

[0361] For example, the first bitstream is decapsulated according to the target container format to obtain fields corresponding to the model identifier, multiple ES packets, and the types of the ES packets. The decapsulation module may parse the type identifier of each ES packet. When the type identifier of an ES packet is determined to be a third identifier value, the decapsulation module may determine that the ES type is a metadata type. In this case, the ES packet may be called the third ES packet. For example, when the type identifier of an ES packet is determined to be a first identifier value, the decapsulation model may determine that the ES packet type is an audio type. In this case, the ES packet may be called the fourth ES packet.

[0362] S1303: To obtain the value of the model identifier, parse the field corresponding to the model identifier.

[0363] For example, for S1301 to S1303, please refer to the explanations given above for S1101 to S1103. Further details will not be explained here.

[0364] S1304: When the value of the model identifier is a pre-configured identifier value, read the model information bitstream from the third ES packet.

[0365] S1305: To obtain the decoding model, the model is reconstructed based on the model information.

[0366] S1306: Read the audio bitstream from the fourth ES packet.

[0367] S1307: Decode the audio bitstream based on the decoding model to obtain the reconstructed audio signal.

[0368] For example, the decapsulation module may further parse the decoder_config_flag in the syntax of the Metadata AU Cell corresponding to the third ES packet. If decoder_config_flag is "1", the decoder may read (i.e., initialize) the decoding model and then decode the audio bitstream based on the decoding model to obtain the reconstructed audio signal.

[0369] For example, for S1305-S1307, please refer to the explanations in S703-S706 mentioned above. Further details will not be explained here.

[0370] Figure 14A shows the audio encoding process. In Figure 14A, the target bitstream type is the second type. The audio bitstream and the model information bitstream may each be encapsulated in different ES packets, or the ES packets of the model information bitstream and the audio bitstream may be encapsulated separately according to the target container format in order to obtain the first bitstream.

[0371] S1401: Acquire the audio signal.

[0372] S1402: Determine whether both the encoding model and the decoding model have been updated.

[0373] S1403: When both the encoding model and the decoding model are updated, encode the audio signal based on the updated encoding model to obtain the audio bitstream.

[0374] S1404: A model information bitstream is generated based on the model information of the updated decoding model, and the updated decoding model corresponds to the updated encoding model.

[0375] For example, regarding S1401 to S1404, please refer to the explanations for S601 to S604 mentioned above. Further details will not be explained here.

[0376] S1405: Encapsulate the model information bitstream into a fifth ES packet.

[0377] S1406: To obtain the fourth bitstream, encapsulate the fifth ES packet according to the target container format and set the type of the fifth ES packet to a pre-configured type.

[0378] For example, in the process of encapsulating a fifth ES packet according to the target container format, the type identifier of the ES packet may be set to the second identifier value.

[0379] For example, if the target container format is TS, the fifth ES packet is encapsulated in the TS stream, i.e., the fourth bitstream.

[0380] S1407: Use the audio bitstream as the sixth ES packet.

[0381] S1408: To obtain the fifth bitstream, encapsulate the sixth ES packet according to the target container format and set the type of the sixth ES packet to audio type.

[0382] For example, one audio bitstream may be used as one sixth ES packet. In this way, N sixth ES packets can be obtained.

[0383] For example, in the process of encapsulating a sixth ES packet according to the target container format, the type identifier of the sixth ES packet may be set to the first identifier value.

[0384] For example, if the target container format is TS, the sixth ES packet is encapsulated in the TS stream, i.e., the fifth bitstream.

[0385] S1409: The fourth bitstream and the fifth bitstream are combined to obtain the first bitstream.

[0386] Refer to Figure 14B. The target container format is TS, and the first bitstream is a TS stream. The first bitstream may contain two TS streams, namely TS Stream 1 and TS Stream 2, where TS Stream 1 is the fourth bitstream and TS Stream 2 is the fifth bitstream.

[0387] For example, in the embodiment of Figure 14A, the identifier may be the field value of the ES field or the field value of the field corresponding to the target container format. This is not limited to the present application. See the above description for further details. Further details are not described here again.

[0388] For example, when the model identifier is a field in the target container format, the model identifier may be placed in a fourth bitstream or in a fifth bitstream.

[0389] For example, when the model identifier is an ES field, the model identifier may be placed in at least one fifth ES packet or at least one sixth ES packet.

[0390] Figure 15 shows the audio decoding process. Figure 15 shows the audio decoding process corresponding to the audio encoding process in Figure 14A.

[0391] S1501: Receive the first bitstream.

[0392] For example, when the first electronic device performs encoding according to the audio encoding method in the embodiment of Figure 14A, it is assumed that the target container format is TS, and the first bitstream received by the second electronic device is essentially a TS stream, or it is assumed that the target container format is MP4, and the first bitstream received by the second electronic device is essentially an MP4 stream.

[0393] In this case, the first bitstream may include the fourth bitstream and the fifth bitstream.

[0394] For example, when the model identifier is a field in the target container format and the fourth bitstream contains a field corresponding to the model identifier, see S1502 and S1503 for the process of parsing the value of the model identifier from the first bitstream.

[0395] S1502: Decapsulate the fourth bitstream according to the target container format to obtain the field corresponding to the model identifier and the fifth ES packet.

[0396] In the following example, we will use a TS stream as the first bitstream for illustrative purposes.

[0397] For example, after a second electronic device receives a first bitstream, the decapsulation module may decapsulate each TS stream according to the target container format to obtain the ES packets and the type of ES packets, and may further decapsulate a portion of the TS stream to obtain fields corresponding to the model identifier.

[0398] For example, when the type of an ES packet is a pre-configured type, the corresponding TS stream may be determined to be the fourth bitstream and the corresponding ES packet may be determined to be the fifth ES packet. When the type of an ES packet is an audio type, the corresponding TS stream may be determined to be the fifth bitstream and the corresponding ES packet may be determined to be the sixth ES packet.

[0399] S1503: To obtain the value of the model identifier, parse the field corresponding to the model identifier.

[0400] S1504: When the value of the model identifier is a pre-configured identifier value, read the model information bitstream from the fifth ES packet.

[0401] S1505: To obtain the decoding model, the model is reconstructed based on the model information.

[0402] S1506: Decapsulate the fifth bitstream according to the target container format in order to obtain the sixth ES packet.

[0403] S1507: Read the audio bitstream from the 6th ES packet.

[0404] S1508: Decode the audio bitstream based on the decoding model to obtain the reconstructed audio signal.

[0405] For example, for S1503-S1508, please refer to the explanations in S703-S706 mentioned above. Further details will not be explained here.

[0406] In this way, compared to encapsulating the second and third bitstreams in the same TS stream, the second electronic device does not need to decompose the TS stream to analyze the second and third bitstreams, thereby reducing the complexity of the analysis process.

[0407] Figure 16A shows the audio encoding process. In Figure 16A, the target bitstream type is the third type. To obtain the sixth and seventh bitstreams, the audio bitstream and the model information bitstream are encapsulated in different ES packets, and after the ES packets of the model information bitstream and the ES packets of the audio bitstream are encapsulated separately according to the target container format, the sixth and seventh bitstreams are encapsulated according to the target transport protocol to obtain the first bitstream.

[0408] S1601: Acquire the audio signal.

[0409] S1602: Determine whether both the encoding model and the decoding model have been updated.

[0410] S1603: When both the encoding model and the decoding model are updated, encode the audio signal based on the updated encoding model to obtain the audio bitstream.

[0411] S1604: A model information bitstream is generated based on the model information of the updated decoding model, and the updated decoding model corresponds to the updated encoding model.

[0412] For example, regarding S1601-S1604, please refer to the explanations for S601-S604 mentioned above. Further details will not be explained here.

[0413] S1605: Encapsulate the model information bitstream into the seventh ES packet.

[0414] S1606: To obtain the sixth bitstream, encapsulate the seventh ES packet according to the target container format and set the type of the seventh ES packet to a pre-configured type.

[0415] S1607: Use the audio bitstream as the 8th ES packet.

[0416] S1608: To obtain the seventh bitstream, encapsulate the eighth ES packet according to the target container format and set the type of the eighth ES packet to audio type.

[0417] S1609: To obtain the first bitstream, encapsulate the sixth and seventh bitstreams according to the target transport protocol.

[0418] For example, when the target transport protocol is the Hyper Text Transfer Protocol (Http), the first bitstream may be an HLS stream. In this case, the first bitstream may include an index file, a sixth bitstream, and a seventh bitstream.

[0419] Refer to Figure 16B. The target transport protocol is HLS, the target container format is TS, and the first bitstream is an HLS stream. The first bitstream may contain two TS streams encapsulated according to HLS, namely TS stream 1 and TS stream 2, where TS stream 1 is the sixth bitstream and TS stream 2 is the seventh bitstream.

[0420] For example, in the decoding process, the information in the index file is typically analyzed sequentially from the beginning to the end of the index file. Therefore, to ensure that the decoding model is reconstructed before the audio bitstream is acquired, the index information for the sixth bitstream may be set in the index file before all the index information for the seventh bitstream.

[0421] For example, the index file may further contain several tags, such as initialization tags. Initialization tags indicate how to obtain the Media Initialization Section. The Media Initialization Section may contain information that needs to be parsed first before the media bitstream (e.g., an audio bitstream) is decoded. Additionally, the index information of the sixth bitstream may be used as attribute information for the initialization tag.

[0422] For example, the index information may be, for instance, a Uniform Resource Locator (URL). This is not limited to the present application.

[0423] For example, in the embodiment of Figure 16A, the model identifier may be an ES field or a field in the target container format. This is not limited to the present application. See the above description for further details. Further details are not described here again.

[0424] For example, when the model identifier is a field in the target container format, the model identifier may be placed in the sixth bitstream or in the seventh bitstream.

[0425] For example, when the model identifier is an ES field, the model identifier may be placed in at least one seventh ES packet or at least one eighth ES packet.

[0426] In the embodiment shown in Figure 16A, the model identifier may alternatively be a field within the target transport protocol.

[0427] Note that the first bitstream in the embodiment of Figure 16A is different from the first bitstream in the embodiment of Figure 14A. The first bitstream in the embodiment of Figure 14A includes only the sixth and seventh bitstreams. The first bitstream in the embodiment of Figure 16A includes the sixth and seventh bitstreams, as well as fields within the target transport protocol.

[0428] A second bitstream having a different reconstruction accuracy (which may be understood as the accuracy of the reconstructed signal obtained by decoding the second bitstream) may be generated. In this way, the bitstream distribution device distributes the first bitstream containing the second bitstream having a different reconstruction accuracy based on bandwidth, thereby ensuring the smooth reproduction of the reconstructed signal.

[0429] Figure 17 shows the audio decoding process. Figure 17 shows the audio decoding process corresponding to the audio encoding process in Figure 16A.

[0430] S1701: Receive the first bitstream.

[0431] S1702: Decapsulate the first bitstream according to the target transport protocol in order to obtain the sixth and seventh bitstreams.

[0432] For example, if the target transport protocol is HTTP, the first bitstream may be decapsulated according to HTTP in order to obtain the index file and multiple bitstreams.

[0433] In one possible method, the index information for the sixth bitstream may be read from an index file, and then the sixth bitstream may be determined from among multiple bitstreams based on the index information for the sixth bitstream; the index information for the seventh bitstream may be read from an index file, and then the seventh bitstream may be determined from among multiple bitstreams based on the index information for the seventh bitstream.

[0434] In one possible method, the index information for the seventh bitstream may be read from an index file, and then the seventh bitstream may be determined from multiple bitstreams based on the index information for the seventh bitstream; or the index information for the sixth bitstream may be read from the attribute information of the initialization tag in the index file, and then the sixth bitstream may be determined from multiple bitstreams based on the index information for the sixth bitstream.

[0435] S1703: Decapsulate the sixth bitstream according to the target container format to obtain the fields corresponding to the model identifier and the seventh ES packet.

[0436] S1704: To obtain the value of the model identifier, parse the field corresponding to the model identifier.

[0437] S1705: When the value of the model identifier is a pre-configured identifier value, read the model information bitstream from the seventh ES packet.

[0438] S1706: To obtain the decoding model, the model is reconstructed based on the model information.

[0439] S1707: Decapsulate the seventh bitstream according to the target container format in order to obtain the eighth ES packet.

[0440] S1708: Read the audio bitstream from the 8th ES packet.

[0441] S1709: Decode the audio bitstream based on the decoding model to obtain the reconstructed audio signal.

[0442] For example, for S1703 to S1709, please refer to the explanations above for S1502 to S1508. Further details will not be explained here.

[0443] In one possible configuration, the first electronic device may transmit index information corresponding to the model information of the decryption model to the second electronic device. Specifically, the index information corresponding to the model information of the decryption model may be determined, and then the first bitstream is obtained based on the second bitstream and the index information corresponding to the model information of the decryption model. In this case, the pre-configured identifier value indicates that the first bitstream contains index information corresponding to the model information of the decryption model. In the decryption process, the process by which the second electronic device obtains the model information of the decryption model may be as follows: it parses the index information of the model information from the first bitstream and obtains the model information based on the index information of the model information. In other words, the pre-configured identifier value instructs the second electronic device to obtain the model information of the decryption model based on the index information obtained from the first bitstream.

[0444] For example, in one possible method of obtaining a first bitstream based on a second bitstream and index information corresponding to the model information of the decryption model, the second bitstream may be used as a third ES packet, the third ES packet is encapsulated according to the target container format to obtain a fifth bitstream, and the fifth bitstream is encapsulated according to the target transport protocol to obtain the first bitstream. In this case, the first bitstream may include the index file and the fifth bitstream, and the index information of the decryption model may be appended to the index file. In one possible method, the index information of the decryption model may be appended before the index information of the fifth bitstream. In one possible method, the index information corresponding to the model information of the decryption model may be used as attribute information of the initialization tag in the index file.

[0445] Compared to carrying the model information bitstream in the first bitstream, carrying only the index information corresponding to the model information of the decoded model in the first bitstream results in less bitrate overhead and can reduce the bandwidth requirements of the first electronic device.

[0446] For example, in one possible method of obtaining the first bitstream based on a second bitstream and index information corresponding to the model information of the decoding model, the eighth bitstream may be generated based on index information corresponding to the model information of the decoding model, and the eighth bitstream and the second bitstream are encapsulated in order to obtain the first bitstream. For a method of encapsulating the eighth bitstream and the second bitstream in order to obtain the first bitstream, see the aforementioned method of encapsulating the third bitstream and the second bitstream in order to obtain the first bitstream. Further details are not described here again.

[0447] In one possible configuration, the first and second electronic devices may pre-agree on a pre-configured storage location in a third device to be used for storing the updated coding and decoding models. In this way, the first electronic device does not need to transmit a model information bitstream or index information corresponding to the model information of the decoding model. In this case, after the decoding and coding models are updated in the pre-configured storage location in the third device, the value of the model identifier in the first bitstream may be set to the pre-configured identifier value. In this case, the pre-configured identifier value indicates that the model information of the decoding model is stored in the pre-configured storage location in the third device. In this way, the second electronic device can retrieve the model information of the decoding model from the pre-configured storage location in the third device; that is, the pre-configured identifier value instructs the second electronic device to retrieve the model information of the decoding model from the pre-configured storage location in the third device. In this case, the bitrate overhead can be reduced and the bandwidth requirements of the first electronic device can be reduced because the first bitstream does not need to transmit a model information bitstream or index information corresponding to the model information.

[0448] It should be noted that the method by which the second electronic device obtains model information from the first bitstream is called the first method, the method by which the second electronic device obtains model information from the first bitstream based on index information corresponding to the model information is called the second method, and the method by which the second electronic device obtains model information from a pre-configured storage location in the third electronic device is called the third method. In one possible method, the first and second electronic devices may agree in advance on the method by which the second electronic device obtains model information for the decoded model. In the encoding and decoding process, once the first and second electronic devices agree on the first method, the first electronic device may generate a third bitstream based on the model information for the decoded model and encapsulate the second and third bitstreams in order to obtain the first bitstream. In the encoding and decoding process, if the first and second electronic devices agree on a second scheme, the first electronic device may generate a first bitstream based on the second bitstream and index information corresponding to the model information of the decoding model. In the encoding and decoding process, if the first and second electronic devices agree on a third scheme, the first electronic device may generate a first bitstream based on the second bitstream.

[0449] In one possible scheme, during the encoding and decoding process, the first electronic device may select one of the following schemes as the target scheme: the first scheme, the second scheme, and the third scheme. When the target scheme is the first scheme, the first electronic device may generate a third bitstream based on model information of the decoding model and encapsulate the second and third bitstreams to obtain the first bitstream. When the target scheme is the second scheme, the first electronic device may generate the first bitstream based on the second bitstream and index information corresponding to the model information of the decoding model. When the target scheme is the third scheme, the first electronic device may generate the first bitstream based on the second bitstream.

[0450] Figure 18 shows an encoding device. The encoding device may be configured to perform the encoding method in the embodiments described above. Therefore, for the beneficial effects that can be achieved by the encoding device, please refer to the beneficial effects in the corresponding methods provided above. Further details will not be described here again.

[0451] Please refer to Figure 18. For example, the encoding device is A signal acquisition module 1801 configured to acquire a signal to be encoded, An encoding module 1802 configured to generate a first bitstream, the first bitstream comprising a model identifier and a second bitstream, the second bitstream being obtained by encoding a signal to be encoded based on an encoding model, the value of the model identifier being a pre-configured identifier value, the pre-configured identifier value instructing a second electronic device to obtain model information of a decoding model, the decoding model being the encoding module and the encoding model. It may include.

[0452] It should be understood that the encoding device may perform additional steps in the encoding method described above. Further details will not be explained here.

[0453] Figure 19 shows a decoding device. The decoding device may be configured to perform the decoding method in the embodiments described above. Therefore, for the beneficial effects that can be achieved by the decoding device, please refer to the beneficial effects in the corresponding methods provided above. Further details are not described here.

[0454] Please refer to Figure 19. For example, the decoding device is A bitstream receiving module 1901 configured to receive a first bitstream, wherein the first bitstream includes a model identifier and a second bitstream, the second bitstream being generated based on encoding information, and the bitstream receiving module, A decapsulation module 1902 configured to obtain model information of a decrypted model when the value of the model identifier is a pre-configured identifier value, The decapsulation module 1902 is further configured to perform model reconstruction based on model information in order to obtain a decoded model, To obtain the reconstructed signal, a decoder 1903 is configured to decode a second bitstream based on a decoding model. It may include.

[0455] It should be understood that the decryption device may perform additional steps in the aforementioned decryption method. Further details will not be explained here.

[0456] For example, Figure 20 is a block diagram of a device 2000 according to one embodiment of the present application. The device 2000 may include a processor 2001 and a transceiver / transceiver pin 2002, and optionally further include a memory 2003.

[0457] The components of device 2000 are connected to each other via bus 2004. In addition to the data bus, bus 2004 further includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the various buses are referred to as bus 2004 in the diagram.

[0458] Optionally, memory 2003 may be configured to store the instructions in the method embodiment described above. The processor 2001 may be configured to execute the instructions in memory 2003, control the receive pin to receive signals, and control the transmit pin to transmit signals.

[0459] The apparatus 2000 may be an electronic device or a chip of an electronic device in the method embodiment described above. The electronic device may include a first electronic device and a second electronic device.

[0460] All relevant details of the steps in the embodiments of the method described above can be referenced in the functional description of the corresponding functional module. Further details are not provided here.

[0461] One embodiment of this application further provides a chip comprising one or more interface circuits and one or more processors. The one or more processors receive or transmit data through the one or more interface circuits. When the one or more processors execute computer instructions, the electronic device is enabled to perform the aforementioned related method steps in order to carry out the method of the above embodiment. The interface circuit is a transceiver / transceiver pin 2002.

[0462] One embodiment further provides a computer-readable storage medium that stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device is enabled to perform the associated method steps described above in order to carry out the method in the above embodiment.

[0463] One embodiment further provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computer or processor, the computer is enabled to perform the aforementioned related steps in order to carry out the method of the aforementioned embodiment.

[0464] In addition, one embodiment of the present application further provides an apparatus, which may specifically be a chip, a component, or a module. The apparatus may include an attached processor and memory. The memory is configured to store computer executable instructions. When the apparatus is operating, the processor may execute computer executable instructions stored in memory to enable the chip to perform the method in the aforementioned method embodiment.

[0465] The electronic devices, computer-readable storage media, computer program products, or chips provided in the embodiments are configured to perform the corresponding methods provided above. Therefore, for the beneficial effects that can be achieved, please refer to the beneficial effects of the corresponding methods provided above. Further details are not described here.

[0466] Based on the above-described implementation configurations, those skilled in the art will understand that, for the sake of simplicity, the division into functional modules described above is used as an example for illustrative purposes. In actual applications, the above-described functions may be assigned to different functional modules and implemented according to requirements. In other words, the internal structure of the device is divided into different functional modules to implement all or some of the functions described above.

[0467] It should be understood that in some embodiments provided in this application, the disclosed apparatus and methods may be carried out in other ways. For example, the embodiments of the apparatus described are merely examples. For example, the division into modules or units is merely a logical functional division, and other divisions may be used in actual implementation. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not performed. In addition, the interconnections, direct connections or communication connections shown or discussed may be carried out through some interfaces. Indirect connections or communication connections between apparatus or units may be carried out in electronic, mechanical, or other forms.

[0468] Units described as separate parts may or may not be physically separate, and parts presented as units may be one or more physical units, may be located in one place, or may be distributed in different locations. Some or all of the units may be selected based on the actual requirements for achieving the objectives of the solution of the embodiment.

[0469] In addition, the functional units in the embodiments of this application may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit. The integrated unit may be implemented in hardware form or in the form of a software functional unit.

[0470] Any content in the embodiments of this application and any content in the same embodiments may be freely combined. Any combination of the aforementioned content shall be within the scope of this application.

[0471] If the integrated unit is implemented in the form of a software function unit and sold or used as an independent product, the integrated unit may be stored on a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application may be implemented in the form of a software product, either essentially, or in part with respect to the prior art, or all or part of the technical solutions. The software product is stored on a storage medium and includes several instructions for instructing a device or processor (which may be a single-chip microcomputer, chip, etc.) to perform all or part of the steps of the method in the embodiments of this application. The storage medium includes various media capable of storing program code, such as USB flash drives, removable hard disk drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0472] The embodiments of this application have been described above with reference to the attached drawings. However, this application is not limited to the specific implementation configurations described above. The specific implementation configurations described above are merely examples, not limitations. Those skilled in the art may make further modifications to the concepts presented in this application without departing from the subject matter and scope of the claims, and all such modifications shall remain within the scope of the protection of this application.

[0473] The methods or algorithmic steps described in combination with those disclosed in the embodiments of this application may be implemented by hardware or by a processor by executing software instructions. Software instructions may include corresponding software modules. These software modules may be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable hard disks, compact disc read-only memory (CD-ROM), or any other form of storage medium known in the art. For example, the storage medium may be coupled to the processor so that the processor can read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and storage medium may be located within an ASIC.

[0474] Those skilled in the art will recognize that, in one or more of the above-mentioned examples, the functions described in the embodiments of this application may be implemented by hardware, software, firmware, or any combination thereof. When the functions are implemented by software, the functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes in a computer-readable medium. The computer-readable medium includes computer-readable storage media and communication media, the communication media including any medium that enables a computer program to be transmitted from one place to another. The storage medium may be any available medium accessible to a general-purpose or dedicated computer.

[0475] The embodiments of this application have been described above with reference to the attached drawings. However, this application is not limited to the specific implementation configurations described above. The specific implementation configurations described above are merely examples, not limitations. Those skilled in the art may make further modifications to the concepts presented in this application without departing from the subject matter and scope of the claims, and all such modifications shall remain within the scope of the protection of this application. [Explanation of symbols]

[0476] 1801 Signal Acquisition Module 1802 Encoding Module 1901 Bitstream Receiver Module 1902 Decapsulation Module 1903 Decoder 2000 equipment 2001 Processor 2002 Transceiver / Transceiver Pin 2003 Memory 2004 Bus

Claims

1. An encoding method applied to a first electronic device, wherein the method is The steps include acquiring the signal to be encoded, A step of generating a first bitstream, wherein the first bitstream includes a second bitstream and a model identifier, the second bitstream is obtained by encoding the signal to be encoded based on an encoding model, the value of the model identifier is a pre-configured identifier value, the pre-configured identifier value instructs a second electronic device to obtain model information of a decoding model, the decoding model corresponds to the encoding model, and Methods that include...

2. The first bitstream further includes a third bitstream, the third bitstream being generated based on the model information of the decoding model. The method according to claim 1.

3. The aforementioned method, The step of generating the third bitstream based on the model information of the decoding model. It further includes, The step of generating the first bitstream described above is: The step of encapsulating the second bitstream and the third bitstream in order to obtain the first bitstream. The method according to claim 1 or 2, including the method described in claim 1 or 2.

4. The first bitstream further includes index information corresponding to the model information of the decoding model, The method according to claim 1.

5. The step of generating the first bitstream described above is: Steps to obtain the first bitstream based on the second bitstream and the index information corresponding to the model information of the decoding model. The method according to claim 1 or 4, including the method described in claim 1 or 4.

6. The method according to claim 1, wherein the pre-configured identifier value further indicates that the model information of the decoding model is stored in a pre-configured storage location within the third electronic device.

7. The method according to any one of claims 1 to 6, wherein the model identifier is a basic stream ES field.

8. The method according to any one of claims 1 to 6, wherein when the target bitstream type of the first bitstream is a type of bitstream encapsulated according to a target container format or a type of bitstream encapsulated according to a target transport protocol, the model identifier is a field in the target container format.

9. The method according to any one of claims 1 to 6, wherein the target bitstream type of the first bitstream is the type of bitstream encapsulated according to the target transport protocol, and the model identifier is a field in the target transport protocol.

10. The aforementioned method, The step of determining whether the aforementioned decoding model is an updated decoding model, When the decoding model is the updated decoding model, the step is to set the value of the model identifier to the pre-set identifier value. The method according to any one of claims 1 to 9, further comprising:

11. The step of encapsulating the second bitstream and the third bitstream in order to obtain the first bitstream is: The step of encapsulating the second bitstream and the third bitstream according to the target bitstream type in order to obtain the first bitstream. The method according to claim 3, including the method described in claim 3.

12. The target bitstream type is an ES type, and the step of encapsulating the second bitstream and the third bitstream according to the target bitstream type in order to obtain the first bitstream is: The step of encapsulating the third bitstream and the second bitstream in a first ES packet and using the first ES packet as the first bitstream. The method according to claim 11, including the method described in claim 11.

13. The target bitstream type is a type of bitstream encapsulated according to the target container format, and the step of encapsulating the second bitstream and the third bitstream according to the target bitstream type in order to obtain the first bitstream is: The steps include encapsulating the third bitstream and the second bitstream in a second ES packet, To obtain the first bitstream, the second ES packet is encapsulated according to the target container format. The method according to claim 11, including the method described in claim 11.

14. The target bitstream type is a type of bitstream encapsulated according to the target container format, and the step of encapsulating the second bitstream and the third bitstream according to the target bitstream type in order to obtain the first bitstream is: The steps include encapsulating the third bitstream in a third ES packet and using the second bitstream as a fourth ES packet, To obtain the first bitstream, the third ES packet and the fourth ES packet are encapsulated according to the target container format, the type of the third ES packet is set to a pre-configured type, and the type of the fourth ES packet is set to a media type. The method according to claim 11, including the method described in claim 11.

15. The method according to claim 14, wherein the pre-configured type is a metadata type.

16. The target bitstream type is a type of bitstream encapsulated according to the target container format, and the step of encapsulating the second bitstream and the third bitstream according to the target bitstream type in order to obtain the first bitstream is: The steps include encapsulating the third bitstream in a fifth ES packet and using the second bitstream as a sixth ES packet, To obtain a fourth bitstream, the steps include encapsulating the fifth ES packet according to the target container format and setting the type of the fifth ES packet to a pre-configured type, To obtain a fifth bitstream, the steps include encapsulating the sixth ES packet according to the target container format and setting the type of the sixth ES packet to a media type, The steps include: combining the fourth bitstream and the fifth bitstream in order to obtain the first bitstream; The method according to claim 11, including the method described in claim 11.

17. The target bitstream type is a type of bitstream encapsulated according to a target transport protocol, and the step of encapsulating the second bitstream and the third bitstream according to the target bitstream type in order to obtain the first bitstream is: The steps include encapsulating the third bitstream in a seventh ES packet and using the second bitstream as an eighth ES packet, To obtain a sixth bitstream, the steps include encapsulating the seventh ES packet according to the target container format and setting the type of the seventh ES packet to a pre-configured type, To obtain a seventh bitstream, the steps include encapsulating the eighth ES packet according to the target container format and setting the type of the eighth ES packet to a media type, To obtain the first bitstream, the steps include encapsulating the sixth bitstream and the seventh bitstream according to the target transport protocol, and The method according to claim 11, including the method described in claim 11.

18. The first bitstream further includes an index file, the index file includes index information for the sixth bitstream and index information for the seventh bitstream, the index information for the sixth bitstream precedes the index information for the seventh bitstream. The method according to claim 17.

19. The aforementioned first bitstream further includes an index file, The attribute information of the initialization tag in the aforementioned index file includes the index information of the sixth bitstream. The method according to claim 17.

20. The encoded signal comprises M frames, M second bitstreams, M first ES packets, the model information comprises N information groups, the third bitstream comprises N sub-bitstreams, the N information groups correspond one-to-one with the N sub-bitstreams, the M first ES packets comprise N ninth ES packets and M-N tenth ES packets, where M and N are positive integers, and M is greater than or equal to N. The step of encapsulating the third bitstream and the second bitstream in the first ES packet is: The steps include encapsulating the N sub-bitstreams and N second bitstreams into the N ninth ES packets, and using the other M-N second bitstreams as the M-N tenth ES packets. Includes, The 9th ES packet includes one sub-bitstream and one second bitstream, and the 10th ES packet includes one second bitstream. The method according to claim 12.

21. The third bitstream includes the description information of the model information and the model information, or The third bitstream includes the description information of the model information and the encoded data of the model information. The method according to claim 2 or 3 or any one of claims 11 to 20.

22. The aforementioned model information includes model structure information or model parameters. The method according to any one of claims 1 to 21.

23. The aforementioned pre-configured identifier value further represents the type of the decoding model. The method according to any one of claims 1 to 22.

24. A coding device used in a first electronic device, wherein the device, A signal acquisition module configured to acquire a signal to be encoded, An encoding module configured to generate a first bitstream, the first bitstream comprising a second bitstream and a model identifier, the second bitstream being obtained by encoding the signal to be encoded based on an encoding model, the value of the model identifier being a pre-configured identifier value, the pre-configured identifier value instructing a second electronic device to obtain model information for a decoding model, the decoding model being an encoding module corresponding to the encoding model and An encoding device equipped with the following features.

25. It is an electronic device, Memory and processor The memory is connected to the processor, The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device is enabled to perform the encoding method described in any one of claims 1 to 23 by the first electronic device. Electronic devices.

26. A chip comprising one or more interface circuits and one or more processors, wherein the one or more processors receive or transmit data through the one or more interface circuits, and when the one or more processors execute a computer instruction, the steps of the method according to any one of claims 1 to 23 are performed.

27. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is run on a computer or processor, the computer or processor is enabled to perform the encoding method performed by the first electronic device according to any one of claims 1 to 23.

28. A computer program product, wherein the computer program product includes computer instructions, and when the computer instructions are executed on a computer or processor, the steps of the method according to any one of claims 1 to 23 are performed.

29. A bitstream storage device comprising a receiver and at least one storage medium, The aforementioned receiver is configured to receive a bitstream, The at least one storage medium is configured to store the bitstream, The bitstream is a first bitstream generated according to the encoding method described in any one of claims 1 to 23. Bitstream storage device.

30. A bitstream transmission device comprising a transmitter and at least one storage medium, The at least one storage medium is configured to store a bitstream, the bitstream being a first bitstream generated according to the encoding method described in any one of claims 1 to 23. The transmission device is configured to acquire the bitstream from the storage medium and transmit the bitstream to the device-side device via the transmission medium. Bitstream transmission device.

31. A bitstream distribution system, wherein the system is A storage medium comprising at least one storage medium configured to store at least one bitstream, wherein the at least one bitstream is a first bitstream generated according to the encoding method described in any one of claims 1 to 23, A streaming media device configured to acquire a target bitstream from at least one storage medium and transmit the target bitstream to a device-side device, wherein the streaming media device comprises a content server or a content distribution server. A bitstream distribution system equipped with the following features.

Images