Video processing method and device and storage medium

Abstract

The invention provides a video processing method and device and a storage medium. The video processing method applied to a decoding end comprises the following steps: receiving a video image code stream and first sequence indication information corresponding to the video image code stream; wherein the first sequence indication information is used for indicating the processing sequence of each message in the message set, and the message set at least comprises one first type of message; determining a message processing rule of each message associated with the first sequence indication information under the condition that the at least one first type message cannot be processed; processing the first sequence indication information according to a message processing rule; and processing the video image code stream according to the processed first sequence indication information.

Description

Technical Field

[0001] This application relates to the technical field of video processing, and particularly to a video processing method, device, and storage medium. Background Art

[0002] A bitstream is a series of bits used to transmit video image data. The bitstream can be transmitted between an encoder and a decoder. The encoder compresses the video image data through an encoding process to form a bitstream, and the decoder restores and reconstructs the video image data from the bitstream through a decoding process. To ensure that the bitstream obtained after the video image data is compressed and encoded by the encoder can be correctly decoded by different decoders, standard organizations usually formulate corresponding video coding and decoding standards to guide the encoding process of the encoder and the decoding process of the decoder. In the H.26x series of video coding and decoding processes, the bitstream is a series of bits in the form of a Network Abstract Layer (NAL) unit stream or a Byte Stream, forming a sequence of Access Units (AUs) of one or more Encoded Video Sequences (CVSs).

[0003] Meanwhile, the Supplemental Enhancement Information (SEI) message is one of the features of the H.26x series of standards. It provides a way to add information to the video bitstream (or video image bitstream), and its main purpose is to play an auxiliary role in the decoding, display, or other related processes. As the number of SEI message types defined in the standard increases, the order of processing different types of SEI messages is different, which may lead to different qualities of the finally decoded video images or different video viewing experiences for users.

[0004] In the H.26x series of standards, a new type of SEI message has been added: the SEI processing order SEI message, which is used to indicate the preferred processing order between multiple SEI messages of the same type or different types. An importance flag (po_sei_importance_flag) can be added to the SEI processing order SEI message to indicate whether the SEI message corresponding to this flag is important. And if an SEI message is important (po_sei_importance_flag = 1) and the decoding end cannot interpret or process this important SEI message, the entire SEI processing order SEI message should be ignored. Obviously, the way of ignoring the entire SEI processing order because one SEI message cannot be correctly processed here is too simple and not reasonable and flexible enough. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a video processing method, device, and storage medium, achieving the effect of flexibly and effectively processing messages.

[0006] Embodiments of the present application provide a video processing method applied to a decoding end, including:

[0007] Receiving a video image bitstream and first sequence indication information corresponding to the video image bitstream; wherein, the first sequence indication information is used to indicate the processing order of each message in a message set, and at least one first type of message is included in the message set;

[0008] In the case where at least one of the first type of messages cannot be processed, determining the message processing rules associated with each message indicated by the first sequence indication information;

[0009] Processing the first sequence indication information according to the message processing rules;

[0010] Processing the video image bitstream according to the processed first sequence indication information.

[0011] Embodiments of the present application provide a video processing method applied to an encoding end, including:

[0012] Determining the processing order of each message in a message set associated with a source image;

[0013] Determining corresponding first sequence indication information based on the processing order of each message; wherein, the first sequence indication information is used to indicate the processing order of each message in a message set, and at least one first type of message is included in the message set;

[0014] Adding the first sequence indication information to the video image bitstream corresponding to the source image and sending it to the decoding end.

[0015] Embodiments of the present application provide a communication device, including: a memory, and one or more processors;

[0016] The memory is configured to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above embodiments.

[0018] Embodiments of the present application provide a storage medium storing a computer program, and when the computer program is executed by a processor, the method described in any of the above embodiments is implemented. Description of the Drawings

[0019] Figure 1a It is a schematic diagram of the implementation of the processing sequence of an SEI message provided by the prior art;

[0020] Figure 1b It is a schematic diagram of the implementation of the processing sequence of an SEI message provided by an embodiment of the present application;

[0021] Figure 2 It is a flowchart of a video processing method provided by an embodiment of the present application;

[0022] Figure 3 It is a flowchart of another video processing method provided by an embodiment of the present application;

[0023] Figure 4 It is a flowchart of a video processing method applied to a decoding end provided by an embodiment of the present application;

[0024] Figure 5 It is a flowchart of a video processing method applied to an encoding end provided by an embodiment of the present application;

[0025] Figure 6 It is a block diagram of the structure of a video processing device provided by an embodiment of the present application;

[0026] Figure 7 It is a block diagram of the structure of another video processing device provided by an embodiment of the present application;

[0027] Figure 8 It is a schematic diagram of the structure of a communication device provided by an embodiment of the present application. Detailed implementation manners

[0028] In the following, embodiments of the present application will be described in conjunction with the accompanying drawings. The present application will be described below with reference to the accompanying drawings of the embodiments. The examples given are only for explaining the present application and are not intended to limit the scope of the present application.

[0029] Figure 1a It is a schematic diagram of the implementation of the processing sequence of an SEI message provided by the prior art; Figure 1b It is a schematic diagram of the implementation of the processing sequence of an SEI message provided by an embodiment of the present application. As Figure 1a shown, assuming that message A is an important SEI message and cannot be interpreted or processed by the decoding end, the processing sequences of all subsequent SEI messages such as message B, message C, message D, and message E of message A are directly ignored, resulting in a too simple, unreasonable, and inflexible SEI processing sequence manner. As Figure 1bAs shown, assuming that message C is an important SEI message and cannot be interpreted or processed by the decoding end, the processing order of messages D and E after message C can be ignored, while messages A and B are still processed according to the pre-configured processing order.

[0030] In view of this, embodiments of the present application provide a flexible and effective indication information for the processing order and processing rules between multiple SEI messages. In the following solutions, SEI messages will be used as an example for illustration, but other ways that can be used to indicate the processing order and processing rules between multiple SEI messages are not excluded, including but not limited to indicating in the file encapsulation of the bitstream or indicating in the parameter set corresponding to the bitstream, etc.

[0031] It should be noted that the message processing rule can also be simply referred to as the processing rule.

[0032] In one embodiment, Figure 2 is a flowchart of a video processing method provided by an embodiment of the present application. This embodiment is applied to the situation of flexibly indicating the processing order and processing rules of multiple SEI messages. This embodiment can be executed by the decoding end. As Figure 2 shown, this embodiment includes: S110 - S140.

[0033] S110. Receive the video image bitstream and the first order indication information corresponding to the video image bitstream.

[0034] Among them, the first order indication information is used to indicate the processing order of each message in the message set, and at least one first type message is included in the message set. In one example, the first type message refers to an important message in the message set. Exemplarily, each message in the message set can be an SEI message. Multiple SEI messages can be included in the message set, and at least one important SEI message is included in the message set. In one example, the message set refers to the set of all SEI messages whose processing order is indicated in the first order indication information.

[0035] S120. When at least one first type message cannot be processed, determine the message processing rules for each message associated with the first order indication information.

[0036] S130. Process the first order indication information according to the message processing rules.

[0037] S140. Process the video image bitstream according to the processed first order indication information.

[0038] In one embodiment, when the importance identifier of the message is the first identifier, the message is a first type message;

[0039] When the importance identifier of a message is the second identifier, the message is a second type of message. The importance identifier can be used to characterize whether a message is important; the first identifier and the second identifier are two different values. For example, the first identifier can be 1, and correspondingly, the second identifier is 0. When the importance identifier of a message is the first identifier, it indicates that the message is important; when the importance identifier of a message is the second identifier, it indicates that the message is unimportant.

[0040] In one embodiment, determining the message processing rules for each message associated with the first order indication information includes at least one of the following:

[0041] Determining the message processing rules for each associated message according to the processing type in the first order indication information;

[0042] Determining the message processing rules for each associated message according to the dependency relationship in the first order indication information;

[0043] Determining the message processing rules for each associated message according to the dependency strength of the first type of message in the first order indication information;

[0044] Determining the message processing rules for each associated message according to the grouping information in the first order indication information;

[0045] Determining the message processing rules for each associated message according to the importance identifier in the first order indication information.

[0046] In one example, the message processing rules can be determined according to the processing type in the first order indication information. The processing type is used to indicate the message processing rules for other messages in the message set when the first type of message cannot be processed.

[0047] In one example, the message processing rules can also be determined according to the dependency relationship in the first order indication information. The dependency relationship is used to indicate the dependency relationship between different types of messages. The dependency relationship is used to indicate whether the message associated with the dependency relationship depends on the corresponding first type of message, or is used to indicate the first type of message on which the message associated with the dependency relationship depends.

[0048] In one example, the message processing rules can also be determined according to the dependency strength of the first type of message in the first order indication information. The dependency strength of the first type of message is used to indicate the dependency strength of other messages after the processing order of the first type of message associated with the dependency strength.

[0049] In one example, the message processing rules can also be determined according to the grouping information in the first order indication information. The grouping information is used to indicate the groups where different types of messages are located.

[0050] In one example, the message processing rule can also be determined according to the importance identifier in the first-order indication information. The importance identifier is used to indicate the importance level of each first-type message.

[0051] In one embodiment, processing the first-order indication information according to the message processing rule includes:

[0052] In the case where the processing type in the first-order indication information is the first identifier and the first-type message cannot be processed, ignore the processing order of the subsequent messages corresponding to the first-type message in the first-order indication information;

[0053] In the case where the processing type in the first-order indication information is the second identifier and the first-type message cannot be processed, ignore the entire first-order indication information. In one example, the first identifier can be 1 and the corresponding second identifier is 0. In one example, in the case where the value of the processing type of a first-type message in the messages associated with the first-order indication information is 0 and the first-type message cannot be processed, ignore the entire first-order indication information, that is, ignore the processing order among all the messages associated with the first-order indication information. In one example, in the case where the value of the processing type of a first-type message in the messages associated with the first-order indication information is 1 and the first-type message cannot be processed, ignore all the messages after the processing order corresponding to the unprocessable first-type message in the first-order indication information, or ignore the processing order of all the messages after the processing order corresponding to the unprocessable first-type message in the first-order indication information.

[0054] In one embodiment, processing the first-order indication information according to the message processing rule includes:

[0055] In the case where the processing type in the first-order indication information is the first identifier and the encapsulation identifier is the second identifier, if the first-type message cannot be processed, retain the processing order among the messages before the processing order of the first-type message in the first-order indication information;

[0056] In the case where the processing type in the first-order indication information is the first identifier and the encapsulation identifier is the first identifier, if the first-type message cannot be processed, ignore the unprocessable first-type message in the first-order indication information;

[0057] In the case where the processing type in the first-order indication information is the second identifier, if the first-type message cannot be processed, the entire first-order indication information is ignored. In one example, the encapsulation identifier is used to indicate whether a message is encapsulated in an SEI processing order nested SEI message corresponding to the SEI processing order SEI message. In one example, when the encapsulation identifier is the first identifier, it is encapsulated in an SEI processing order nested SEI message corresponding to the SEI processing order SEI message; when the encapsulation identifier is the second identifier, it is not encapsulated in an SEI processing order nested SEI message corresponding to the SEI processing order SEI message. In one example, in the case where the processing type of a first-type message in the messages associated with the first-order indication information is the first identifier, the first-type message is not encapsulated (i.e., the encapsulation identifier of the first-type message is the second identifier), and the first-type message cannot be processed, the processing order between all messages before the processing order corresponding to the first-type message in the first-order indication information can be retained. In one example, in the case where the processing type of a first-type message in the messages associated with the first-order indication information is the first identifier, the first-type message is encapsulated (i.e., the encapsulation identifier of the first-type message is the first identifier), and the first-type message cannot be processed, the unprocessable first-type message in the messages associated with the first-order indication information can be ignored, or the processing order of the unprocessable first-type message in the messages associated with the first-order indication information can be ignored. In one example, in the case where the processing type of a first-type message in the messages associated with the first-order indication information is the second identifier and the first-type message cannot be processed, the processing order between all messages associated with the first-order indication information can be ignored.

[0058] In one embodiment, processing the first-order indication information according to the message processing rule includes: when the first-type message associated with the first-order indication information cannot be processed, ignoring the processing order of all messages having a dependency relationship with the first-type message in the first-order indication information. In one example, in the case where there is no dependency relationship between a message and the first-type message, even if the first-type message cannot be processed, the message is still executed according to the pre-configured processing order. In one example, in the case where there is a dependency relationship between a message and the first-type message and the first-type message cannot be processed, all messages having a dependency relationship with the first-type message are ignored, or the processing order of all messages having a dependency relationship with the first-type message is ignored.

[0059] In one embodiment, processing the first-order indication information according to the message processing rule includes:

[0060] When the first type of message associated with the first-order indication information is dependent with a strength of a second identifier, ignore the processing order of the first quantity of messages after the first type of message in the first-order indication information;

[0061] When the first type of message associated with the first-order indication information is dependent with a strength of a first identifier, ignore the entire first-order indication information. In one example, the value of the first identifier is 1, and the corresponding value of the second identifier is 0. In one example, when the first type of message associated with a first-order indication information has a dependence strength of 1 and the first type of message cannot be processed, all messages after the first type of message in the first-order indication information can be ignored, or, the processing order of all messages after the first type of message in the first-order indication information can be ignored. In one example, when the first type of message associated with a first-order indication information has a dependence strength of 0 and the first type of message cannot be processed, the first quantity of messages after the first type of message in the first-order indication information can be ignored, or, the processing order of the first quantity of messages after the first type of message in the first-order indication information can be ignored.

[0062] In one embodiment, processing the first-order indication information according to the message processing rule includes: when the first type of message associated with the first-order indication information cannot be processed, ignore the processing order of all messages with the same grouping information corresponding to the first type of message in the first-order indication information. In one example, the grouping information is used to indicate the group corresponding to a message. For example, the grouping information may include a group identifier, that is, messages with the same grouping information can also be understood as messages with the same group identifier. In one example, when a first type of message associated with a first-order indication information cannot be processed, all messages with the same group identifier corresponding to the first type of message in the first-order indication information can be ignored, or, the processing order of all messages with the same group identifier corresponding to the first type of message in the first-order indication information can be ignored.

[0063] In one embodiment, processing the first-order indication information according to the message processing rule includes:

[0064] When the importance identifier in the first-order indication information is the first identifier and the first type of message cannot be processed, ignore the entire first-order indication information;

[0065] When the importance identifier in the first-order indication information is the second identifier, the encapsulation identifier is the second identifier, and the first-type message cannot be processed, the processing order among the messages before the processing order corresponding to the first-type message in the first-order indication information is retained, or the processing order among the subsequent messages corresponding to the first-type message in the first-order indication information is ignored;

[0066] When the importance identifier in the first-order indication information is the second identifier, the encapsulation identifier is the first identifier, and the first-type message cannot be processed, the unprocessable first-type message in the first-order indication information is ignored. In one example, the value of the first identifier is 1, and the corresponding value of the second identifier is 0. In one example, when the importance identifier of a first-type message in the messages associated with the first-order indication information is 1 and the first-type message cannot be processed, the processing order of all the messages associated with the first-order indication information can be ignored; when the importance identifier of a first-type message in the messages associated with the first-order indication information is 0, the first-type message is not encapsulated, and the first-type message cannot be processed, the processing order of all the messages after the first-type message in the first-order indication information can be ignored, that is, the processing order among all the messages before the processing order corresponding to the first-type message in the messages associated with the first-order indication information is retained, or it can be understood that the processing order of all the messages before the first-type message is still processed according to the pre-configured processing order; when the importance identifier of a first-type message in the messages associated with the first-order indication information is 0, the first-type message is encapsulated, and the first-type message cannot be processed, the unprocessable first-type message in the messages associated with the first-order indication information can be ignored.

[0067] In one embodiment, the first-order indication information at least includes the following parameters: message identifier; number of messages; message nesting situation; importance identifier; message type; processing order of the messages corresponding to each message type; encapsulation identifier.

[0068] In one embodiment, the first-order indication information further includes one of the following parameters: dependency strength of the first-type message; dependency relationship; grouping information; processing type; importance identifier.

[0069] In one embodiment, Figure 3 is a flowchart of another video processing method provided by an embodiment of the present application. This embodiment is applied to the situation of flexibly indicating the processing order and processing rules of multiple SEI messages. This embodiment can be executed by the encoding end. As Figure 3 shown, this embodiment includes: S210 - S230.

[0070] S210. Determine the processing order of each message in the message set associated with the source image.

[0071] S220. Determine corresponding first order indication information based on the processing order of each message.

[0072] Among them, the first order indication information is used to indicate the processing order of each message in the message set, and at least one first type message is included in the message set.

[0073] S230. Add the first order indication information to the video image bitstream corresponding to the source image and send it to the decoding end.

[0074] In one embodiment, when the importance identifier of a message is the first identifier, the message is a first type message;

[0075] When the importance identifier of a message is the second identifier, the message is a second type message.

[0076] In one embodiment, configure a corresponding encapsulation identifier for the message whose importance identifier in the first order indication information is the second identifier. The message whose importance identifier is the second identifier can be understood as a second type message; the encapsulation identifier is used to characterize whether a message is encapsulated in the SEI processing order nested SEI message corresponding to the SEI processing order SEI message. In one example, if the encapsulation identifier of a message is 0, it means that the message exists independently and is not encapsulated in the SEI processing order nested SEI message corresponding to the SEI processing order SEI message. In one example, if the encapsulation identifier of a message is 1, it means that the message is encapsulated in the SEI processing order nested SEI message corresponding to the SEI processing order SEI message. In one example, a corresponding encapsulation identifier can be configured for each message in the first order indication information, or a corresponding encapsulation identifier can be configured for the message whose importance identifier in the first order indication information is the second identifier (i.e., the second type message). In one embodiment, when the number of first type messages in the first order indication information is greater than or equal to 1, configure the corresponding dependency existence identifier as the first identifier;

[0077] When the number of first type messages in the first order indication information is 0, configure the corresponding dependency existence identifier as the second identifier. In one example, the value of the dependency existence identifier depends on whether there is a first type message among all the messages associated with the first order indication information. If the number of first type messages is greater than or equal to 1, the value of the dependency existence identifier is 1; if the number of first type messages is 0, the value of the dependency existence identifier is 0.

[0078] In one embodiment, when the number of messages of the first type in the first sequence indication information is greater than 1, for a message whose dependency relationship is identified as the first identifier, a dependency message index of the first type of message it depends on is configured. In one example, the first identifier can be 1. The dependency relationship identifier is used to indicate the degree of dependency of a message on the first type of message. In one example, when the value of the dependency relationship identifier is 1, it indicates that there is a dependency relationship between a message and any first type of message. In one example, when the value of the dependency relationship identifier is 0, it indicates that there is no dependency relationship between a message and any first type of message. In one example, the dependency message index is used to indicate the index number corresponding to the first type of message that a message depends on. When a message depends on multiple first type of messages simultaneously, the dependency message index is used to indicate the index numbers corresponding to the multiple first type of messages that the message depends on.

[0079] In one embodiment, when the number of messages with the importance identifier being the first identifier in the first sequence indication information is greater than or equal to 1, the grouping existence identifier is configured as the first identifier;

[0080] When the number of messages with the importance identifier being the first identifier in the first sequence indication information is 0, the grouping existence identifier is configured as the second identifier. In one example, the value of the first identifier is 1, and the corresponding value of the second identifier is 0. The message with the importance identifier being the first identifier can be understood as the first type of message. In one example, the grouping existence identifier can depend on whether there is a first type of message among the multiple messages associated with the first sequence indication information. If there is a first type of message, the grouping existence identifier is configured as 1; if there is no first type of message, the grouping existence identifier is configured as 0.

[0081] In one embodiment, when the grouping existence identifier is the first identifier, a grouping identifier corresponding to each message is configured; wherein, each grouping corresponding to the grouping identifier contains at least one first type of message. The grouping identifier is used to indicate the number of the group where a message is located.

[0082] In one embodiment, for the first type of message with the encapsulation identifier being the second identifier, a corresponding importance level identifier is configured.

[0083] In one embodiment, the first sequence indication information at least includes the following parameters: message identifier; number of messages; message nesting situation; importance identifier; message type; processing order of messages corresponding to each message type; encapsulation identifier.

[0084] In one embodiment, the first sequence indication information further includes one of the following parameters: dependency strength of the first type of message being depended on; dependency relationship; grouping information; processing type; importance level identifier.

[0085] It should be noted that for the explanations of parameters such as the first sequence indication information, the first type of message, the second type of message, the first identifier, the second identifier, the importance identifier, the importance degree identifier, the grouping information, and the dependency strength of the first type of message in the video processing method applied to the encoding end, please refer to the descriptions of the corresponding parameters in the above video processing method applied to the decoding end, and will not be elaborated here.

[0086] In the following Embodiment 1 to Embodiment 7, each message in the message set can be an SEI message. The first type of message refers to an important SEI message (i.e., an SEI message with an importance identifier being true or 1), the second type of message refers to an unimportant SEI message (i.e., an SEI message with an importance identifier being false or 0), the dependency strength of the first type of message is called the dependency strength of the important SEI message, the first sequence indication information is called the SEI processing sequence indication information, or the SEI processing sequence SEI message (i.e., SEI processing order SEI message), and the processing sequence can also be called the preferred processing sequence. Taking this as an example, the video processing process will be described.

[0087] In the first embodiment, Figure 4 is a flowchart of a video processing method applied to a decoding end provided by an embodiment of the present application. This embodiment is applied to a device for decoding and processing a video bitstream. The input of the device is a video image bitstream or a media file, and the output is a decoded and reconstructed video image. As Figure 4 shown, this embodiment includes S310 - S340.

[0088] S310: Receive a video image bitstream and its corresponding SEI processing sequence indication information for indicating the processing sequence of multiple SEI messages.

[0089] Among them, at least one of the multiple SEI messages is an important SEI message. Determine whether an SEI message is important according to the importance identifier of the SEI message included in the SEI processing sequence indication information. For example, when the importance identifier of the SEI message is true (i.e., sei_importance_flag = 1), it indicates that the SEI message is important. On the contrary, when the importance identifier of the SEI message is false (i.e., sei_importance_flag = 0), it indicates that the SEI message is unimportant.

[0090] S320: When at least one important SEI message cannot be processed, determine the processing rule of the SEI message associated with the SEI processing sequence indication information.

[0091] The processing rule can be determined according to the processing type (ProcessingType) in the SEI processing order indication information. The processing type is used to indicate the processing rule for other SEI messages in the case where important SEI messages (i.e., sei_importance_flag = 1) cannot be processed.

[0092] The processing rule can also be determined according to the dependency relationship in the SEI processing order indication information. The dependency relationship is used to indicate the dependency relationship between different types of SEI messages. The dependency relationship is used to indicate whether the SEI message associated with this dependency relationship depends on the corresponding important SEI message; or to indicate the important SEI message on which the SEI message associated with this dependency relationship depends.

[0093] The processing rule can also be determined according to the degree of being depended on of the important SEI message in the SEI processing order indication information. The degree of being depended on is used to indicate the degree of being depended on by other SEI messages whose processing order is after the important SEI message associated with the degree of being depended on.

[0094] The processing rule can also be determined according to the grouping information in the SEI processing order indication information. The grouping information is used to indicate the groups where different types of SEI messages are located.

[0095] The processing rule can also be determined according to the importance identifier in the SEI processing order indication information. The importance identifier is used to indicate the importance degree of each first type of message.

[0096] In addition to being any of the above-mentioned processing rules, the processing rule can also be any combination of the different processing rules listed above.

[0097] It should be noted that different types of SEI messages can include but are not limited to one of the following: SEI messages with different message types (Payload Type); SEI messages with the same message type but incomplete identical message contents; SEI messages with different importance identifiers, etc.

[0098] S330. Process the SEI processing order indication information according to the processing rule.

[0099] Determine the processing method for multiple SEI messages associated with the SEI processing order indication information according to the processing rule determined in S320, including but not limited to:

[0100] The processing rule is determined according to the processing type ProcessingType. Moreover, when ProcessingType is 0, if the SEI message with the importance flag being true cannot be processed, the entire SEI processing sequence indication information is ignored; when ProcessingType is 1, if the SEI message with the importance flag being true cannot be processed, only the processing sequences of all subsequent SEI messages in the SEI processing sequence indication information for which the importance flag is true are ignored.

[0101] The processing rule is determined according to the dependency relationship between SEI messages. If the SEI message with the importance flag being true cannot be processed, only the processing sequences of the SEI messages in the SEI processing sequence indication information that are dependent on the SEI message with the importance flag being true are ignored.

[0102] The processing rule can also be determined according to the strength of the dependency of important SEI messages. If the important SEI message associated with the dependency strength cannot be processed and the dependency strength is a strong dependency, the entire SEI processing sequence indication information is ignored; if the important SEI message associated with the dependency strength cannot be processed and the dependency strength is not a strong dependency, only the processing sequences of the first quantity of SEI messages in the SEI processing sequence indication information are ignored. The first quantity can be carried in the SEI processing sequence indication information, can also be a default configuration, or can be the number of SEI messages between an important SEI message and the next important SEI message.

[0103] The processing rule can also be determined according to the grouping information in the SEI processing sequence indication information. If the SEI message with the importance flag being true cannot be processed, only the processing sequences of the SEI messages in the SEI processing sequence indication information that have the same grouping information as the important SEI message are ignored.

[0104] S340. Process the video image bitstream or the decoded video image according to the processed SEI processing sequence indication information.

[0105] Process the SEI processing sequence indication information according to any of the processing rules in S330 to obtain the processed SEI processing sequence indication information. If the processed SEI processing sequence indication information still indicates the processing sequences of at least two SEI messages, process the video image bitstream or the decoded video image according to the processing sequences of the at least two SEI messages indicated by the processed SEI processing sequence indication information, that is, perform the operations corresponding to the two SEI messages on the video image bitstream or the decoded video image respectively according to the processing sequences of the at least two SEI messages.

[0106] In the second embodiment, this embodiment provides an example of SEI processing order indication information when determining a processing rule according to a processing type. The SEI processing order indication information is the SEI message of the SEI processing order.

[0107]

[0108] Semantic information:

[0109] po_id: Used to indicate the identification number of the SEI message of the SEI processing order;

[0110] po_num_sei_messages_minus2: Plus 2 indicates the number of SEI message types indicating the preferred processing order in the SEI message of the SEI processing order;

[0111] po_processing_type: Indicates the processing rule for the SEI message of the SEI processing order or other SEI messages in the SEI message of the SEI processing order when there is a case where the SEI message type with the importance flag being true cannot be processed; for example: when the value is 0, the entire SEI message of the SEI processing order is ignored, that is, the preferred processing order among all the SEI message types indicated in the SEI message of the SEI processing order is ignored; when the value is 1, ignore all the SEI messages after the processing order of the SEI message with the importance flag being true that cannot be processed in the SEI processing order indication information, or ignore the processing order of all the SEI messages after the processing order of the SEI message with the importance flag being true that cannot be processed in the SEI processing order indication information.

[0112] It should be noted that the processing type can also be used in combination with other processing rules. That is, when the value of po_processing_type is 1, the processing rules in Embodiments 3 to 5 can be further combined to determine the final processing rule for all the SEI messages or the processing order of the SEI messages after the processing order of the SEI message with the importance flag being true that cannot be processed in the SEI processing order indication information.

[0113] po_sei_wrapping_flag[i]: When the value is equal to 1, it is used to indicate that the i-th SEI message (type) is encapsulated in the SEI processing order nested SEI message corresponding to the SEI message of the SEI processing order; when the value is equal to 0, it is used to indicate that the i-th SEI message exists independently and is not encapsulated in the SEI processing order nested SEI message corresponding to the SEI message of the SEI processing order.

[0114] po_sei_importance_flag[i]: Indicates the importance determined by the encoder for the SEI message type with index i. When the value is 1, it is used to indicate that the SEI message type with index i is important; when the value is 0, it is used to indicate that the SEI message type with index i is not important.

[0115] po_sei_payload_type[i]: Specifies the payloadType value of the SEI message of the i-th type. The type of the SEI message can be determined according to the payloadType value.

[0116] po_sei_processing_order[i]: Indicates the preferred processing order of the i-th type of sei message that provides preferred processing order information in the SEI processing order SEI message.

[0117]

[0118] po_sei_wrapping_flag[i]: When the importance of the i-th SEI message type is not 1 (i.e., the i-th SEI message type is not important), if the value is equal to 1, it is used to indicate that the i-th SEI message is encapsulated in the SEI processing order nested SEI message corresponding to the SEI processing order SEI message; if the value is equal to 0, it is used to indicate that the i-th SEI message exists independently and is not encapsulated in the SEI processing order nested SEI message corresponding to the SEI processing order SEI message.

[0119] In the following embodiments, no special description will be given to this kind of extension. The encapsulation flag (po_sei_wrapping_flag) can be set for each SEI message in the SEI processing order SEI message, or the encapsulation flag (po_sei_wrapping_flag) can be set only for the SEI messages whose importance flag (po_sei_importance_flag) is false in the SEI processing order SEI message (i.e., the second type of message in the above embodiment).

[0120] In the third embodiment, this embodiment provides an example of SEI processing order indication information when determining the processing rule according to the dependency relationship. The SEI processing order indication information is the SEI processing order SEI message.

[0121]

[0122] Semantic information (syntax elements repeated in all subsequent embodiments will not be explained for semantic information, and directly refer to the description of the second embodiment above):

[0123] po_sei_dependence_flag[i]: It is used to indicate the dependence degree of the i-th SEI message type on important SEI messages. When the value is 0, it is used to indicate that there is no dependence relationship between the i-th SEI message type and important SEI messages; when the value is 1, it is used to indicate that there is a dependence relationship between the i-th SEI message type and important SEI messages.

[0124] At this time, the processing rule is as follows: when there is no dependence relationship between an SEI message type and important SEI messages, even if the important SEI messages cannot be processed or executed, the SEI message still executes normally according to the processing order; when there is a dependence relationship between an SEI message type and important SEI messages, and the important SEI messages cannot be processed or executed, then the SEI message or the processing order of the SEI message is ignored.

[0125]

[0126] Semantic information:

[0127] po_sei_dependence_present_flag: When the value is equal to 0, it is used to indicate that there is no dependence relationship flag (po_sei_dependence_flag) in the SEI processing order SEI messages; when the value is equal to 1, it is used to indicate that there is no dependence relationship flag (po_sei_dependence_flag) in the SEI processing order SEI messages.

[0128] The dependence relationship existence flag (po_sei_dependence_present_flag) can depend on whether there is an SEI message with a true importance flag among multiple SEI messages associated with the SEI processing order indication information. If there is an SEI message with a true importance flag, the dependence relationship existence flag takes the value of 1; if there is no SEI message with a true importance flag, the dependence relationship existence flag takes the value of 0.

[0129] po_sei_dependence_flag[i]: The same as the description in the above Embodiment 3.

[0130]

[0131] Semantic information:

[0132] po_num_important_messages_minus1: After adding 1, it indicates the number of SEI messages with a true importance flag associated with the SEI processing order indication information.

[0133] po_sei_dependence_flag[i]: Indicates the dependence degree of the i-th SEI message type on important SEI messages. If the value is 0, it is used to indicate that there is no dependence relationship between the i-th SEI message type and any important SEI messages; if the value is 1, it is used to indicate that there is a dependence relationship between the i-th SEI message type and any important SEI messages.

[0134] po_sei_dependence_idx[i]: Used to indicate the index number of the important SEI message on which the i-th SEI message type depends.

[0135] It should be noted that although in this embodiment, an SEI message type can only depend on one important SEI message. However, the solution of this application does not limit the number of important SEI messages that an SEI message type can depend on. In the case where an SEI message type depends on multiple important SEI messages simultaneously, the dependence relationship index (po_sei_dependence_idx[i]) can be used to indicate the index numbers of the multiple important SEI messages on which the i-th SEI message type depends.

[0136] In the fourth embodiment, this embodiment provides an example of SEI processing order indication information when determining the processing rule according to the dependence strength of important SEI messages, and the SEI processing order indication information is the SEI message of the SEI processing order.

[0137]

[0138] Semantic information:

[0139] po_sei_strong_influence_flag[i]: When the value is 1 and the associated important SEI message cannot be processed, all SEI messages or the processing order of SEI messages after this important SEI message can be ignored; when the value is 0 and the associated important SEI message cannot be processed, the processing order of the first number of SEI messages or SEI messages after this important SEI message can be ignored.

[0140] It should be added that in the case of multiple important SEI messages, if the value of po_sei_strong_influence_flag is 1, it can also be used to indicate that when the associated important SEI message cannot be processed, all SEI messages or the processing order of SEI messages after this important SEI message and before the next SEI message with the value of po_sei_strong_influence_flag being 1 are ignored.

[0141] po_num_influence_sei_messages_minus1: An increment by 1 indicates the number of SEI messages affected, that is, the number of SEI messages whose processing order can be ignored after the important SEI message in question when its associated important SEI message cannot be processed, or the number of SEI messages in the SEI message sequence that can be ignored.

[0142] In the fifth embodiment, this embodiment provides an example of SEI processing order indication information when determining a processing rule based on packet information in SEI processing order indication information. The SEI processing order indication information is the SEI processing order SEI message.

[0143]

[0144] Semantic information:

[0145] po_sei_group_id[i]: Indicates the packet identification of the i-th SEI message. In each packet, at least one SEI message has an importance identification of true (i.e., at least one first type message).

[0146] The processing rule at this time is: In the case where an important SEI message cannot be processed or executed, all SEI messages with the same packet identification as this important SEI message or the processing order of the SEI messages can be ignored.

[0147]

[0148] Semantic information:

[0149] po_sei_group_present_flag: If the value is equal to 0, it is used to indicate that there is no packet identification (po_sei_group_id) in the SEI processing order SEI message; if the value is equal to 1, it is used to indicate that there is a packet identification (po_sei_group_id) in the SEI processing order SEI message.

[0150] The packet presence flag (po_sei_group_present_flag) can depend on whether there is an SEI message with an importance identification of true among multiple SEI messages associated with the SEI processing order indication information. If there is an SEI message with an importance identification of true, the packet presence flag takes a value of 1; if there is no SEI message with an importance identification of true, the packet presence flag takes a value of 0.

[0151]

[0152] Semantic information:

[0153] po_processing_type: When the value is 0 and there is a SEI message type with a true importance flag in the SEI processing order SEI message that cannot be processed, the entire SEI processing order SEI message can be ignored, that is, the preferred processing order among all SEI message types associated in the SEI processing order SEI message is ignored; when the value is 1 and there is a SEI message type with a true importance flag in the SEI processing order SEI message that cannot be processed, part of the SEI messages or the processing order of part of the SEI messages associated in the SEI processing order SEI message is ignored.

[0154] po_sei_group_id[i]: If the processing type (po_processing_type) has a value of 1, it is used to indicate the group identifier where the i-th SEI message is located. At least one SEI message in each group has a true importance flag (i.e., 1).

[0155] At this time, the final processing rule is: if the processing type (po_processing_type) has a value of 0 and an important SEI message cannot be processed or executed, then the entire SEI processing order SEI message is ignored, that is, the preferred processing order among all SEI message types associated in the SEI processing order SEI message is ignored; conversely, if the processing type (po_processing_type) has a value of 1, then all SEI messages or the processing order of SEI messages with the same group identifier as this important SEI message are ignored.

[0156] In the sixth embodiment, this embodiment provides an example of SEI processing order indication information when determining the processing rule according to the SEI processing order indication information, and the SEI processing order indication information is the SEI processing order SEI message.

[0157]

[0158] Semantic information:

[0159] When the importance flag (po_sei_importance_flag[i]) of a SEI message has a value of 1, that is, when the SEI message is an important SEI message, an importance degree flag (po_sei_importance_degree_flag[i]) can be further set for the SEI message, and the importance degree flag is the same as the processing degree flag po_sei_processing_degree_flag.

[0160] po_sei_importance_degree_flag[i]: If the value is equal to 1 and it indicates that important SEI messages cannot be correctly processed, the decoder shall ignore the entire SEI processing order SEI message, that is, the preferred processing order among all SEI message types associated in the SEI processing order SEI message is ignored; when po_sei_importance_degree_flag[i] is equal to 0, it indicates that the decoder shall still process all SEI messages whose po_sei_processing_order[i] is less than or equal to the po_sei_processing_order[i] of the important SEI message that cannot be processed.

[0161]

[0162] The main difference from the sixth embodiment is that: in this embodiment, when the importance flag (po_sei_importance_flag[i]) of an SEI message has a value of 1, that is, the SEI message is an important SEI message, and the encapsulation flag of the important SEI message has a value equal to 0, that is, the important SEI message is not encapsulated in a processing order nesting SEI message, the importance degree flag (po_sei_importance_degree_flag[i]) is further set for the SEI message, and the importance degree flag is the same as the processing degree flag po_sei_processing_degree_flag.

[0163] In the seventh embodiment, Figure 5 It is a flowchart of a video processing method applied to the encoding end provided by an embodiment of the present application. This embodiment is applied to a device for encoding an input video image. The input of the device is the source video image, and the output is a video image bitstream or a media file.

[0164] S410. Determine the processing order of multiple SEI messages associated with the video image.

[0165] Multiple SEI messages associated with a video image refer to information carried in multiple SEI messages that will act on the decoding, display, or other related processes after video image encoding. The multiple SEI messages are multiple different types of SEI messages. Different types of SEI messages include but are not limited to: SEI messages with different payload types, SEI messages with the same payload type but incomplete identical message contents, or SEI messages with different importance flags, etc.

[0166] The encoding end can determine the preferred processing order of multiple SEI messages associated with a video image according to the SEI message type, the information carried in the SEI message, the decoded and reconstructed image quality, the profile, etc.

[0167] S420. Determine the SEI processing order indication message and the processing rule according to the processing order.

[0168] The SEI processing order indication message is used to indicate the preferred processing order of multiple SEI messages. The processing rule can be any one of the processing rules mentioned in Embodiments 1 to 5, or any combination of multiple processing rules. The processing rule can be included in the SEI processing order indication message or determined according to the SEI processing order indication message.

[0169] In addition, the SEI processing order indication message can also include one or more pieces of information such as the number of associated SEI messages, the type of associated SEI messages, the importance identifier of the SEI message, the encapsulation identifier of the SEI message, etc.

[0170] The processing rule is used to indicate the processing method for the preferred processing order in the SEI processing order indication message or for the other SEI messages except the unexecutable SEI messages when there are unexecutable SEI messages among multiple SEI messages. Further, it can be restricted that only when the importance identifier of the unexecutable SEI message is true, the processing method for the preferred processing order in the SEI processing order indication message or for the other SEI messages except the unexecutable SEI messages among multiple SEI messages is carried out according to the processing rule.

[0171] S430. Send the SEI processing order indication message and the processing rule to the decoding end.

[0172] The SEI processing order indication message and the processing rule can be added to the coded stream after video image encoding and sent to the decoding end together, or the SEI processing order indication message and the processing rule can be sent to the decoding end separately as a separate coded stream, or the SEI processing order indication message and the processing rule can be sent to the decoding end by any other means.

[0173] In one embodiment, Figure 6 is a structural block diagram of a video processing device provided by an embodiment of the present application. This embodiment is applied to the decoding end. As Figure 6 shown, the video processing device in this embodiment includes: a receiver 510, a first determination module 520, a first processing module 530, and a second processing module 540.

[0174] A receiver 510, configured to receive a video image bitstream and first sequence indication information corresponding to the video image bitstream; wherein the first sequence indication information is used to indicate the processing order of each message in a message set, and at least one first type of message is included in the message set.

[0175] A first determination module 520, configured to determine a message processing rule for each message associated with the first sequence indication information when at least one first type of message cannot be processed.

[0176] A first processing module 530, configured to process the first sequence indication information according to the message processing rule.

[0177] A second processing module 540, configured to process the video image bitstream according to the processed first sequence indication information.

[0178] In one embodiment, when the importance identifier of a message is a first identifier, the message is a first type of message;

[0179] When the importance identifier of a message is a second identifier, the message is a second type of message.

[0180] In one embodiment, determining the message processing rule for each message associated with the first sequence indication information includes at least one of the following:

[0181] Determining the message processing rule for each associated message according to the processing type in the first sequence indication information;

[0182] Determining the message processing rule for each associated message according to the dependency relationship in the first sequence indication information;

[0183] Determining the message processing rule for each associated message according to the dependency strength of the first type of message in the first sequence indication information;

[0184] Determining the message processing rule for each associated message according to the grouping information in the first sequence indication information;

[0185] Determining the message processing rule for each associated message according to the importance identifier in the first sequence indication information.

[0186] In one embodiment, processing the first sequence indication information according to the message processing rule includes:

[0187] When the processing type in the first sequence indication information is a first identifier and the first type of message cannot be processed, ignoring the processing order of subsequent messages corresponding to the first type of message in the first sequence indication information;

[0188] When the processing type in the first-order indication information is the second identifier and the first-type message cannot be processed, the entire first-order indication information is ignored.

[0189] In one embodiment, processing the first-order indication information according to the message processing rule includes:

[0190] When the processing type in the first-order indication information is the first identifier and the encapsulation identifier is the second identifier, if the first-type message cannot be processed, retain the processing order among the messages before the first-type message in the first-order indication information.

[0191] When the processing type in the first-order indication information is the first identifier and the encapsulation identifier is the first identifier, if the first-type message cannot be processed, ignore the first-type message in the first-order indication information that cannot be processed.

[0192] When the processing type in the first-order indication information is the second identifier, if the first-type message cannot be processed, ignore the entire first-order indication information.

[0193] In one embodiment, processing the first-order indication information according to the message processing rule includes:

[0194] When the first-type message associated with the first-order indication information cannot be processed, ignore the processing order of all messages in the first-order indication information that have a dependency relationship with the first-type message.

[0195] In one embodiment, processing the first-order indication information according to the message processing rule includes:

[0196] When the first-type message associated with the first-order indication information has a dependency strength of the second identifier, ignore the processing order of the first number of messages after the first-type message in the first-order indication information.

[0197] When the first-type message associated with the first-order indication information has a dependency strength of the first identifier, ignore the entire first-order indication information.

[0198] In one embodiment, processing the first-order indication information according to the message processing rule includes:

[0199] When the first-type message associated with the first-order indication information cannot be processed, ignore the processing order of all messages in the first-order indication information that have the same grouping information as the first-type message.

[0200] In one embodiment, processing the first-order indication information according to the message processing rule includes:

[0201] When the importance identifier in the first-order indication information is the first identifier and the first-type message cannot be processed, the entire first-order indication information is ignored;

[0202] When the importance identifier in the first-order indication information is the second identifier, the encapsulation identifier is the second identifier, and the first-type message cannot be processed, the processing order between the messages before the processing order corresponding to the first-type message in the first-order indication information is retained, or the processing order between the subsequent messages corresponding to the first-type message in the first-order indication information is ignored;

[0203] When the importance identifier in the first-order indication information is the second identifier, the encapsulation identifier is the first identifier, and the first-type message cannot be processed, the first-type message that cannot be processed in the first-order indication information is ignored.

[0204] In one embodiment, the first-order indication information at least includes the following parameters: message identifier; number of messages; message nesting situation; importance identifier; message type; processing order of messages corresponding to each message type; encapsulation identifier.

[0205] In one embodiment, the first-order indication information further includes one of the following parameters: the dependence strength of the first-type message; dependence relationship; grouping information; processing type; importance identifier.

[0206] The video processing device provided in this embodiment is configured to implement Figure 2 the video processing method applied to the decoding end in the illustrated embodiment. The implementation principle and technical effect of the video processing device provided in this embodiment are similar and will not be elaborated here.

[0207] In one embodiment, Figure 7 is a structural block diagram of another video processing device provided in an embodiment of the present application. This embodiment is applied to the encoding end. As Figure 7 shown, the video processing device in this embodiment includes: a first determination module 610, a second determination module 620, and a transmitter 630.

[0208] The first determination module 610 is configured to determine the processing order of each message in the message set associated with the source image.

[0209] The second determination module 620 is configured to determine the corresponding first-order indication information based on the processing order of each message; wherein, the first-order indication information is used to indicate the processing order of each message in the message set, and at least one first-type message is included in the message set.

[0210] The transmitter 630 is configured to add the first-order indication information to the video image bitstream corresponding to the source image and send it to the decoding end.

[0211] In one embodiment, when the importance identifier of a message is the first identifier, the message is a first type of message;

[0212] When the importance identifier of a message is the second identifier, the message is a second type of message.

[0213] In one embodiment, a corresponding encapsulation identifier is configured for the message with the importance identifier being the second identifier in the first sequence indication information.

[0214] In one embodiment, when the number of first type messages in the first sequence indication information is greater than or equal to 1, the corresponding dependency existence identifier is configured as the first identifier;

[0215] When the number of first type messages in the first sequence indication information is 0, the corresponding dependency existence identifier is configured as the second identifier.

[0216] In one embodiment, when the number of first type messages in the first sequence indication information is greater than 1, for the message with the dependency identifier being the first identifier, the dependency message index of the first type message it depends on is configured.

[0217] In one embodiment, when the number of messages with the importance identifier being the first identifier in the first sequence indication information is greater than or equal to 1, the group existence identifier is configured as the first identifier;

[0218] When the number of messages with the importance identifier being the first identifier in the first sequence indication information is 0, the group existence identifier is configured as the second identifier.

[0219] In one embodiment, when the group existence identifier is the first identifier, a group identifier corresponding to each message is configured; wherein, each group corresponding to a group identifier contains at least one first type message.

[0220] In one embodiment, a corresponding importance level identifier is configured for the first type message with the encapsulation identifier being the second identifier.

[0221] In one embodiment, the first sequence indication information at least includes the following parameters: message identifier; number of messages; message nesting situation; importance identifier; message type; processing order of messages corresponding to each message type; encapsulation identifier.

[0222] In one embodiment, the first sequence indication information further includes one of the following parameters: dependency strength of the first type message; dependency relationship; grouping information; processing type; importance level identifier.

[0223] The video processing device provided in this embodiment is set to implement Figure 3The video processing method applied to the encoding end in the illustrated embodiment. The implementation principle and technical effect of the video processing device provided in this embodiment are similar, and will not be elaborated here.

[0224] In one embodiment, Figure 8 is a schematic structural diagram of a communication device provided in an embodiment of the present application. As Figure 8 shown, the device provided in the present application includes: a processor 710, a memory 720, and a communication module 730. The number of processors 710 in this device can be one or more, Figure 8 and one processor 710 is taken as an example here. The number of memories 720 in this device can be one or more, Figure 8 and one memory 720 is taken as an example here. The processor 710, memory 720, and communication module 730 of this device can be connected through a bus or other means, Figure 8 and connected through a bus is taken as an example here. In this embodiment, this device can be a decoding end or an encoding end.

[0225] The memory 720, as a computer-readable storage medium, can be set to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the device in any embodiment of the present application (for example, the receiver 510, the first determination module 520, the first processing module 530, and the second processing module 540 in the video processing device applied to the decoding end). The memory 720 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the device, etc. In addition, the memory 720 can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 720 can further include a memory remotely set relative to the processor 710, and these remote memories can be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and their combinations.

[0226] When the communication device is a decoding end, the device provided above can be set to execute the video processing method applied to the decoding end provided in any of the above embodiments, and has corresponding functions and effects.

[0227] When the communication device is an encoding end, the device provided above can be set to execute the video processing method applied to the encoding end provided in any of the above embodiments, and has corresponding functions and effects.

[0228] An embodiment of the present application further provides a storage medium including computer-executable instructions, which are used to execute a video processing method applied to a decoding end when executed by a computer processor. The method includes: receiving a video image bitstream and first sequence indication information corresponding to the video image bitstream; wherein, the first sequence indication information is used to indicate the processing sequence of each message in a message set, and at least one first type of message is included in the message set; in the case where at least one of the first type of messages cannot be processed, determining the message processing rules associated with each message indicated by the first sequence indication information; processing the first sequence indication information according to the message processing rules; and processing the video image bitstream according to the processed first sequence indication information.

[0229] An embodiment of the present application further provides a storage medium including computer-executable instructions, which are used to execute a video processing method applied to an encoding end when executed by a computer processor. The method includes: determining the processing sequence of each message in a message set associated with a source image; determining corresponding first sequence indication information based on the processing sequence of each message; wherein, the first sequence indication information is used to indicate the processing sequence of each message in the message set, and at least one first type of message is included in the message set; adding the first sequence indication information to the video image bitstream corresponding to the source image and sending it to the decoding end.

[0230] Those skilled in the art should understand that the term user equipment covers any suitable type of wireless user equipment, such as a mobile phone, a portable data processing device, a portable network browser, or a vehicle-mounted mobile station.

[0231] Generally, various embodiments of the present application can be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects can be implemented in hardware, while other aspects can be implemented in firmware or software that can be executed by a controller, a microprocessor, or other computing devices, although the present application is not limited thereto.

[0232] Embodiments of the present application can be implemented by a data processor of a mobile device executing computer program instructions, such as in a processor entity, or by hardware, or by a combination of software and hardware. The computer program instructions can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages.

[0233] Any block diagram of a logical process in the drawings of this application may represent a program step, or may represent interconnected logic circuits, modules, and functions, or may represent a combination of program steps and logic circuits, modules, and functions. A computer program may be stored in a memory. The memory may have any type suitable for the local technical environment and may be implemented using any suitable data storage technology, such as but not limited to Read-Only Memory (ROM), Random Access Memory (RAM), optical memory devices and systems (Digital Video Disc (DVD) or Compact Disk (CD)), etc. The computer-readable medium may include a non-transitory storage medium. The data processor may be any type suitable for the local technical environment, such as but not limited to general-purpose computers, special-purpose computers, microprocessors, Digital Signal Processing (DSP), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FGPA), and processors based on multi-core processor architectures.

[0234] The above are only the preferred embodiments of this application and are not used to limit this application. For those skilled in the art, this application may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the protection scope of this application.

Claims

1. A video processing method, characterized in that, Applied to the decoding end, including: Receiving a video image bitstream and first sequence indication information corresponding to the video image bitstream; wherein, the first sequence indication information is used to indicate the processing order of each message in a message set, and at least one first type of message is included in the message set; When at least one of the first type of messages cannot be processed, determining the message processing rules for each message associated with the first sequence indication information; Processing the first sequence indication information according to the message processing rules; Processing the video image bitstream according to the processed first sequence indication information.

2. The method according to claim 1, wherein When the importance identifier of the message is the first identifier, the message is a first type of message; When the importance identifier of the message is the second identifier, the message is a second type of message.

3. The method according to claim 1, wherein The determining the message processing rules for each message associated with the first sequence indication information includes at least one of the following: Determining the message processing rules for each associated message according to the processing type in the first sequence indication information; Determining the message processing rules for each associated message according to the dependency relationship in the first sequence indication information; Determining the message processing rules for each associated message according to the dependency strength of the first type of message in the first sequence indication information; Determining the message processing rules for each associated message according to the grouping information in the first sequence indication information; Determining the message processing rules for each associated message according to the importance identifier in the first sequence indication information.

4. The method according to claim 1, wherein The processing the first sequence indication information according to the message processing rules includes: When the processing type in the first sequence indication information is the first identifier and the first type of message cannot be processed, ignoring the processing order of the subsequent messages corresponding to the first type of message in the first sequence indication information; When the processing type in the first sequence indication information is the second identifier and the first type of message cannot be processed, ignoring the entire first sequence indication information.

5. The method according to claim 1, wherein The processing the first sequence indication information according to the message processing rules includes: When the processing type in the first sequence indication information is the first identifier and the encapsulation identifier is the second identifier, if the first type of message cannot be processed, retaining the processing order between the messages before the processing order of the first type of message in the first sequence indication information; When the processing type in the first sequence indication information is the first identifier and the encapsulation identifier is the first identifier, if the first type of message cannot be processed, ignoring the first type of message that cannot be processed in the first sequence indication information; When the processing type in the first sequence indication information is the second identifier, if the first type of message cannot be processed, ignoring the entire first sequence indication information.

6. The method according to claim 1, characterized in that, The processing the first sequence indication information according to the message processing rules includes: In the case where the first type of message associated with the first-order indication information cannot be processed, the processing order of all messages that have a dependency relationship with the first type of message in the first-order indication information is ignored.

7. The method according to claim 1, wherein The processing of the first-order indication information according to the message processing rule includes: In the case where the first type of message associated with the first-order indication information is dependent with a second identifier, the processing order of the first quantity of messages after the first type of message in the first-order indication information is ignored; In the case where the first type of message associated with the first-order indication information is dependent with a first identifier, the entire first-order indication information is ignored.

8. The method according to claim 1, wherein The processing of the first-order indication information according to the message processing rule includes: In the case where the first type of message associated with the first-order indication information cannot be processed, the processing order of all messages with the same packet information corresponding to the first type of message in the first-order indication information is ignored.

9. The method according to claim 1, characterized in that, The processing of the first-order indication information according to the message processing rule includes: In the case where the importance identifier in the first-order indication information is the first identifier and the first type of message cannot be processed, the entire first-order indication information is ignored; In the case where the importance identifier in the first-order indication information is the second identifier, the encapsulation identifier is the second identifier, and the first type of message cannot be processed, the processing order between the messages before the processing order corresponding to the first type of message in the first-order indication information is retained, or the processing order between the subsequent messages corresponding to the first type of message in the first-order indication information is ignored; In the case where the importance identifier in the first-order indication information is the second identifier, the encapsulation identifier is the first identifier, and the first type of message cannot be processed, the first type of message that cannot be processed in the first-order indication information is ignored.

10. The method according to claim 1, characterized in that, The first-order indication information at least includes the following parameters: message identifier; message quantity; message nesting situation; importance identifier; message type; the processing order of the messages corresponding to each message type; encapsulation identifier.

11. The method according to claim 10, characterized in that, The first-order indication information further includes one of the following parameters: the dependency strength of the first type of message; dependency relationship; packet information; processing type; importance identifier.

12. A video processing method, characterized in that, Applied to the encoding end, it includes: Determining the processing order of each message in the message set associated with the source image; Determining the corresponding first-order indication information based on the processing order of each message; wherein, the first-order indication information is used to indicate the processing order of each message in the message set, and at least one first type of message is included in the message set; Adding the first-order indication information to the video image bitstream corresponding to the source image and sending it to the decoding end.

13. The method according to claim 12, wherein In the case where the importance identifier of the message is the first identifier, the message is a first type of message; In the case where the importance identifier of the message is the second identifier, the message is a second type of message.

14. The method according to claim 12, wherein Configure a corresponding encapsulation identifier for the message with the importance identifier being the second identifier in the first-order indication information.

15. The method according to claim 12, characterized in that, When the number of first-type messages in the first-order indication information is greater than or equal to 1, configure the corresponding dependency existence identifier as the first identifier; When the number of first-type messages in the first-order indication information is 0, configure the corresponding dependency existence identifier as the second identifier.

16. The method according to claim 12, wherein When the number of first-type messages in the first-order indication information is greater than 1, for the message with the dependency identifier being the first identifier, configure the dependency message index of the first-type message it depends on.

17. The method according to claim 12, characterized in that, When the number of messages with the importance identifier being the first identifier in the first-order indication information is greater than or equal to 1, configure the group existence identifier as the first identifier; When the number of messages with the importance identifier being the first identifier in the first-order indication information is 0, configure the group existence identifier as the second identifier.

18. The method according to claim 17, wherein When the group existence identifier is the first identifier, configure a corresponding group identifier for each message; wherein, each group corresponding to the group identifier contains at least one first-type message.

19. The method according to claim 14, wherein For the first-type message with the encapsulation identifier being the second identifier, configure the corresponding importance level identifier.

20. The method according to claim 12, characterized in that, The first-order indication information at least includes the following parameters: message identifier; number of messages; message nesting situation; importance identifier; message type; processing order of messages corresponding to each message type; encapsulation identifier.

21. The method according to claim 20, wherein The first-order indication information further includes one of the following parameters: dependency strength of the first-type message; dependency relationship; grouping information; processing type; importance level identifier.

22. A communication device, characterized in that, Comprising: A memory, and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-11 or 12-21 above.

23. A storage medium, characterized in that, The storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1-11 or 12-21 above.

Images