Video processing method, device, and storage medium

Abstract

The present application provides a video processing method, a device, and a storage medium. The video processing method applied to a decoding end comprises: receiving a video image bitstream and first order indication information corresponding to the video image bitstream, wherein the first order indication information is used for indicating a processing order of each message in a message set, and the message set comprises at least one first-type message; when at least one first-type message cannot be processed, determining a message processing rule for each message associated with the first order indication information; processing the first order indication information on the basis of the message processing rule; and processing the video image bitstream on the basis of the processed first order indication information.

Description

Video processing method, device and storage medium Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video processing method, device and storage medium. Background Art

[0002] A bitstream is a series of bits used to transmit video image data. The bitstream can be transmitted between an encoder and a decoder. The encoder compresses the video image data to form a bitstream through the encoding process, and the decoder recovers and reconstructs the video image data from the bitstream through the decoding process. To ensure that the bitstream of video image data compressed and encoded by the encoder can be correctly decoded by different decoders, standards organizations usually formulate corresponding video codec standards to guide the encoding process of encoders and the decoding process of decoders. In the H.26x series video codec process, the bitstream is a series of bits in the form of a Network Abstract Layer (NAL) unit stream (NAL unit stream) or a byte stream (Byte stream), forming a sequence of access units (AU) of one or more coded video sequences (CVS).

[0003] The Supplemental Enhancement Information (SEI) message, a feature of the H.26x series of standards, provides a method for adding information to a video stream (or video image stream) to aid in decoding, display, and other related processes. With the increasing number of SEI message types defined in the standards, the order in which different SEI messages are processed may result in varying decoded video quality or a different viewing experience for users.

[0004] In the H.26x series of standards, a new SEI message type has been added: the SEI processing order SEI message, which is used to indicate the preferred processing order between multiple SEI messages of the same or different types. An importance flag (po_sei_importance_flag) can be added to the SEI processing order SEI message to indicate whether the SEI message corresponding to the flag is important. And if an SEI message is important (po_sei_importance_flag = 1) and the decoding end cannot interpret or process this important SEI message, the entire SEI processing order SEI message should be ignored. Obviously, the method of ignoring the entire SEI processing order because one SEI message cannot be processed correctly is too simple and not reasonable or flexible enough.

[0005] Summary of the Invention

[0006] In view of this, embodiments of the present application provide a video processing method, device, and storage medium, which achieve the effect of flexible and effective message processing.

[0007] The present invention provides a video processing method, which is applied to a decoding end and includes:

[0008] Receive a video image code stream and first sequence indication information corresponding to the video image code stream; wherein the first sequence indication information is used to indicate the processing order of each message in a message set, and the message set includes at least one first type message;

[0009] In a case where at least one message of the first type cannot be processed, determining a message processing rule for each message associated with the first sequence indication information;

[0010] Processing the first sequence indication information according to the message processing rule;

[0011] The video image code stream is processed according to the processed first sequence indication information.

[0012] The present invention provides a video processing method, which is applied to an encoding end and includes:

[0013] Determine the processing order of each message in the message set associated with the source image;

[0014] Determining corresponding first sequence indication information based on the processing order of each message; wherein the first sequence indication information is used to indicate the processing order of each message in the message set, and the message set includes at least one first type message;

[0015] The first sequence indication information is added to the video image code stream corresponding to the source image and sent to the decoding end.

[0016] An embodiment of the present application provides a communication device, comprising: a memory, and one or more processors;

[0017] The memory is configured to store one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of the above embodiments.

[0019] An embodiment of the present application provides a storage medium storing a computer program. When the computer program is executed by a processor, the method described in any one of the above embodiments is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] FIG1a is a schematic diagram of an implementation of a processing sequence of SEI messages provided by the related art;

[0021] FIG1b is a schematic diagram illustrating an implementation of an SEI message processing sequence provided in an embodiment of the present application;

[0022] FIG2 is a flow chart of a video processing method provided in an embodiment of the present application;

[0023] FIG3 is a flowchart of another video processing method provided in an embodiment of the present application;

[0024] FIG4 is a flow chart of a video processing method applied to a decoding end provided by an embodiment of the present application;

[0025] FIG5 is a flowchart of a video processing method applied to an encoding end provided by an embodiment of the present application;

[0026] FIG6 is a structural block diagram of a video processing device provided in an embodiment of the present application;

[0027] FIG7 is a structural block diagram of another video processing device provided in an embodiment of the present application;

[0028] FIG8 is a schematic structural diagram of a communication device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The following describes the present application in conjunction with the accompanying drawings. The examples are only used to explain the present application and are not used to limit the scope of the present application.

[0030] Figure 1a is a schematic diagram of the implementation of the processing order of an SEI message provided by the related art; Figure 1b is a schematic diagram of the implementation of the processing order of an SEI message provided by an embodiment of the present application. As shown in Figure 1a, assuming that message A is an important SEI message and cannot be interpreted or processed by the decoding end, the processing order of all SEI messages subsequent to message A, such as message B, message C, message D and message E, is directly ignored, resulting in the SEI processing order being too simple, unreasonable and inflexible. As shown in Figure 1b, assuming that message C is an important SEI message and cannot be interpreted or processed by the decoding end, the processing order of message D and message E after message C can be ignored, while message A and message B are still processed according to the pre-configured processing order.

[0031] In view of this, the embodiment of the present application provides a flexible and effective indication information of the processing order and processing rules between multiple SEI messages. The subsequent scheme will be explained using SEI messages as an example, but other other methods that can be used to indicate the processing order and processing rules between multiple SEI messages are not excluded, including but not limited to indicating in the file encapsulation of the codestream or indicating in the parameter set corresponding to the codestream.

[0032] It should be noted that message processing rules can also be referred to as processing rules for short.

[0033] In one embodiment, FIG2 is a flowchart of a video processing method provided by an embodiment of the present application. This embodiment is applicable to situations where the processing order and processing rules of multiple SEI messages are flexibly indicated. This embodiment can be executed by a decoding end. As shown in FIG2, this embodiment includes: S110-S140.

[0034] S110: Receive a video image code stream and first sequence indication information corresponding to the video image code stream.

[0035] The first order indication information is used to indicate the processing order of each message in a message set, and the message set includes at least one first-type message. In one example, the first-type message refers to an important message in the message set. For example, each message in the message set may be an SEI message. The message set may include multiple SEI messages, and the message set includes at least one important SEI message. In one example, the message set refers to the set of all SEI messages whose processing order is indicated in the first order indication information.

[0036] S120: When at least one first-type message cannot be processed, determine a message processing rule for each message associated with the first sequence indication information.

[0037] S130: Process the first sequence indication information according to a message processing rule.

[0038] S140: Process the video image code stream according to the processed first sequence indication information.

[0039] In one embodiment, when the importance identifier of the message is the first identifier, the message is a first type message;

[0040] If the message importance flag is the second flag, the message is a second-type message. Importance flags can be used to indicate whether a message is important. The first flag and the second flag can have different values. For example, the first flag can be 1, and the second flag can be 0. If the importance flag of a message is the first flag, the message is important; if the importance flag of a message is the second flag, the message is unimportant.

[0041] In one embodiment, determining a message processing rule for each message associated with the first sequence indication information includes at least one of the following:

[0042] Determining a message processing rule for each associated message according to the processing type in the first order indication information;

[0043] Determining a message processing rule for each associated message according to the dependency relationship in the first order indication information;

[0044] Determining a message processing rule for each associated message according to the first type message dependency strength in the first order indication information;

[0045] Determining a message processing rule for each associated message according to the grouping information in the first order indication information;

[0046] A message processing rule associated with each message is determined according to the importance identifier in the first order indication information.

[0047] In one example, the message processing rule may be determined according to the processing type in the first order indication information. The processing type is used to indicate the message processing rule for other messages in the message set when the first type of message cannot be processed.

[0048] In one example, the message processing rule may also be determined based on a dependency relationship in the first order indication information. The dependency relationship indicates a dependency relationship between messages of different types. The dependency relationship indicates whether the message associated with the dependency relationship is dependent on the corresponding first type message, or indicates the first type message on which the message associated with the dependency relationship is dependent.

[0049] In one example, the message processing rule may also be determined according to the first type message dependency strength in the first order indication information. The first type message dependency strength is used to indicate the dependency strength of other messages after the first type message associated by the dependency strength is processed.

[0050] In one example, the message processing rule may also be determined based on the grouping information in the first order indication information. The grouping information is used to indicate the groups to which different types of messages belong.

[0051] In one example, the message processing rule may also be determined according to the importance identifier in the first order indication information. The importance identifier is used to indicate the importance of each first type message.

[0052] In one embodiment, processing the first order indication information according to the message processing rule includes:

[0053] If the processing type in the first sequence indication information is the first identifier and the first type message cannot be processed, the processing order of subsequent messages corresponding to the first type message in the first sequence indication information is ignored;

[0054] In the case where the processing type in the first sequence indication information is the second identifier and the first type message cannot be processed, the entire first sequence indication information is ignored. In one example, the first identifier can be 1 and the corresponding second identifier is 0. In one example, in the case where the processing type value of a first type message in the message associated with the first sequence indication information is 0 and the first type message cannot be processed, the entire first sequence indication information is ignored, that is, the processing order between all messages associated with the first sequence indication information is ignored. In one example, in the case where the processing type value of a first type message in the message associated with the first sequence indication information is 1 and the first type message cannot be processed, all messages after the processing order corresponding to the first type message that cannot be processed in the first sequence indication information are ignored, or the processing order of all messages after the processing order corresponding to the first type message that cannot be processed in the first sequence indication information is ignored.

[0055] In one embodiment, processing the first order indication information according to the message processing rule includes:

[0056] When the processing type in the first sequence indication information is the first identifier and the encapsulation identifier is the second identifier, if the first type message cannot be processed, retaining the processing order of the messages before the processing order of the first type message in the first sequence indication information;

[0057] When the processing type in the first sequence indication information is the first identifier and the encapsulation identifier is the first identifier, if the first type message cannot be processed, ignoring the first type message that cannot be processed in the first sequence indication information;

[0058] In the case where the processing type in the first sequence indication information is the second identifier, if the first type message cannot be processed, the entire first sequence indication information is ignored. In one example, the encapsulation identifier is used to indicate whether a message is encapsulated in a SEI processing order nested SEI message corresponding to the SEI processing order SEI message. In one example, in the case where the encapsulation identifier is the first identifier, it is encapsulated in a SEI processing order nested SEI message corresponding to the SEI processing order SEI message; in the case where the encapsulation identifier is the second identifier, it is not encapsulated in a SEI processing order nested SEI message corresponding to the SEI processing order SEI message. In one example, in the message associated with the first sequence indication information, the processing type of a first type message is the first identifier, the first type message is not encapsulated (that is, the encapsulation identifier of the first type message is the second identifier), and when the first type message cannot be processed, the processing order between all messages before the processing order corresponding to the first type message in the first sequence indication information can be retained. In one example, if the processing type of a first type message in the message associated with the first sequence indication information is the first identifier, the first type message is encapsulated (i.e., the encapsulation identifier of the first type message is the first identifier), and the first type message cannot be processed, the first type message that cannot be processed in the message associated with the first sequence indication information can be ignored, or the processing order of the first type message that cannot be processed in the message associated with the first sequence indication information can be ignored. In one example, if the processing type of a first type message in the message associated with the first sequence indication information is the second identifier, and the first type message cannot be processed, the processing order among all messages associated with the first sequence indication information can be ignored.

[0059] In one embodiment, the first sequence indication information is processed according to a message processing rule, including: when a first type of message associated with the first sequence indication information cannot be processed, ignoring the processing order of all messages in the first sequence indication information that have a dependency relationship with the first type of message. In one example, when there is no dependency relationship between a message and a first type of message, even if the first type of message cannot be processed, the message is still executed according to a preconfigured processing order. In one example, when there is a dependency relationship between a message and a first type of message, and the first type of message cannot be processed, all messages that have a dependency relationship with the first type of message are ignored, or the processing order of all messages that have a dependency relationship with the first type of message is ignored.

[0060] In one embodiment, processing the first order indication information according to the message processing rule includes:

[0061] When the first type of message associated with the first sequence indication information is identified by a second dependency strength, ignoring the processing order of a first number of messages following the first type of message in the first sequence indication information;

[0062] In the case where the first type of message associated with the first sequence indication information has a dependency strength of the first identifier, the entire first sequence indication information is ignored. In one example, the value of the first identifier is 1, and the corresponding value of the second identifier is 0. In one example, in the case where the first type of message of a first type of message associated with the first sequence indication information has a dependency strength of 1, and the first type of message cannot be processed, all messages after the first type of message in the first sequence indication information can be ignored, or the processing order of all messages after the first type of message in the first sequence indication information can be ignored. In one example, in the case where the first type of message of a first type of message associated with the first sequence indication information has a dependency strength of 0, and the first type of message cannot be processed, the first number of messages after the first type of message in the first sequence indication information can be ignored, or the processing order of the first number of messages after the first type of message in the first sequence indication information can be ignored.

[0063] In one embodiment, the first sequence indication information is processed according to a message processing rule, including: when a first type message associated with the first sequence indication information cannot be processed, ignoring the processing order of all messages in the first sequence indication information that have the same grouping information as the first type message. In one example, the grouping information is used to indicate the group corresponding to a message. For example, the grouping information may include a group identifier, that is, messages with the same grouping information can also be understood as messages with the same group identifier. In one example, when a first type message associated with a first sequence indication information cannot be processed, all messages in the first sequence indication information that have the same group identifier as the first type message can be ignored, or the processing order of all messages in the first sequence indication information that have the same group identifier as the first type message can be ignored.

[0064] In one embodiment, processing the first order indication information according to the message processing rule includes:

[0065] If the importance identifier in the first sequence indication information is the first identifier and the first type of message cannot be processed, ignore the entire first sequence indication information;

[0066] If the importance identifier in the first sequence indication information is the second identifier, the encapsulation identifier is the second identifier, and the first type message cannot be processed, retain the processing order of messages before the processing order corresponding to the first type message in the first sequence indication information, or ignore the processing order of subsequent messages corresponding to the first type message in the first sequence indication information;

[0067] If the importance identifier in the first sequence indication information is the second identifier, the encapsulation identifier is the first identifier, and the first type of message cannot be processed, the first type of message that cannot be processed in the first sequence indication information is ignored. In one example, the value of the first identifier is 1, and the corresponding value of the second identifier is 0. In one example, when the importance of a first type message in the message associated with the first sequence indication information is identified as 1 and the first type message cannot be processed, the processing order of all messages associated with the first sequence indication information can be ignored; when the importance of a first type message in the message associated with the first sequence indication information is identified as 0, the first type message is not encapsulated, and the first type message cannot be processed, the processing order of all messages after the first type message in the first sequence indication information can be ignored, that is, the processing order between all messages before the corresponding processing order of the first type message in the message associated with the first sequence indication information is retained, which can also be understood as still processing all messages before the first type message according to the pre-configured processing order; when the importance of a first type message in the message associated with the first sequence indication information is identified as 0, the first type message is encapsulated, and the first type message cannot be processed, the first type message that cannot be processed in the message associated with the first sequence indication information can be ignored.

[0068] In one embodiment, the first sequence indication information includes at least the following parameters: message identifier; number of messages; message nesting; importance identifier; message type; processing order of messages corresponding to each message type; and encapsulation identifier.

[0069] In one embodiment, the first sequence indication information further includes one of the following parameters: dependency strength of the first type of message; dependency relationship; grouping information; processing type; and importance identifier.

[0070] In one embodiment, FIG3 is a flowchart of another video processing method provided by an embodiment of the present application. This embodiment is applied to situations where the processing order and processing rules of multiple SEI messages are flexibly indicated. This embodiment can be executed by the encoder. As shown in FIG3, this embodiment includes: S210-S230.

[0071] S210: Determine a processing order for each message in a message set associated with the source image.

[0072] S220. Determine corresponding first sequence indication information based on the processing sequence of each message.

[0073] The first sequence indication information is used to indicate the processing order of each message in the message set, and the message set includes at least one first type message.

[0074] S230: Add the first sequence indication information to the video image code stream corresponding to the source image, and send it to the decoding end.

[0075] In one embodiment, when the importance identifier of the message is the first identifier, the message is a first type message;

[0076] When the importance identifier of the message is the second identifier, the message is a second type message.

[0077] In one embodiment, a corresponding encapsulation identifier is configured for a message whose importance identifier is the second identifier in the first sequence indication information. A message whose importance identifier is the second identifier can be understood as a second type message; the encapsulation identifier is used to indicate whether a message is encapsulated in a SEI processing order nested SEI message corresponding to the SEI processing order SEI message. In one example, if the encapsulation identifier of a message is 0, it means that the message exists independently and is not encapsulated in a SEI processing order nested SEI message corresponding to the SEI processing order SEI message. In one example, if the encapsulation identifier of a message is 1, it means that the message is encapsulated in a SEI processing order nested SEI message corresponding to the SEI processing order SEI message. In one example, a corresponding encapsulation identifier can be configured for each message in the first sequence indication information, and a corresponding encapsulation identifier can also be configured for a message (i.e., a second type message) whose importance identifier is the second identifier in the first sequence indication information. In one embodiment, when the number of first type messages in the first sequence indication information is greater than or equal to 1, the corresponding dependency existence identifier is configured as the first identifier;

[0078] When the number of first-type messages in the first sequence indication information is 0, the corresponding dependency existence flag is configured as the second flag. In one example, the value of the dependency existence flag depends on whether the first-type message exists among all messages associated with the first sequence indication information. If the number of first-type messages is greater than or equal to 1, the value of the dependency existence flag is 1; if the number of first-type messages is 0, the value of the dependency existence flag is 0.

[0079] In one embodiment, when the number of first-type messages in the first sequence indication information is greater than 1, for the message whose dependency identifier is the first identifier, the dependent message index of the first-type message on which it depends is configured. In one example, the first identifier can be 1. The dependency identifier is used to indicate the degree of dependency of a message on the first-type message. In one example, when the value of the dependency identifier is 1, it indicates that a message has a dependency on any first-type message. In one example, when the value of the dependency identifier is 0, it indicates that a message has no dependency on any first-type message. In one example, the dependent message index is used to indicate the index number corresponding to the first-type message on which a message depends. In the case where a message depends on multiple first-type messages at the same time, the dependent message index is used to indicate the index numbers corresponding to the multiple first-type messages on which the message depends.

[0080] In one embodiment, when the number of messages whose importance identifier is the first identifier in the first sequence indication information is greater than or equal to 1, the group existence identifier is configured as the first identifier;

[0081] When the number of messages with the first importance identifier in the first sequence indication information is 0, the group existence identifier is configured as the second identifier. In one example, the value of the first identifier is 1, and the corresponding value of the second identifier is 0. Messages with the first importance identifier can be understood as first-type messages. In one example, the group existence identifier can depend on whether there are first-type messages among the multiple messages associated with the first sequence indication information. If there are first-type messages, the group existence identifier is configured as 1; if there are no first-type messages, the group existence identifier is configured as 0.

[0082] In one embodiment, when the group existence identifier is the first identifier, a group identifier corresponding to each message is configured, wherein the group corresponding to each group identifier contains at least one first type message. The group identifier is used to indicate the number of the group to which a message belongs.

[0083] In one embodiment, a corresponding importance identifier is configured for the first type message whose encapsulation identifier is the second identifier.

[0084] In one embodiment, the first sequence indication information includes at least the following parameters: message identifier; number of messages; message nesting; importance identifier; message type; processing order of messages corresponding to each message type; and encapsulation identifier.

[0085] In one embodiment, the first sequence indication information further includes one of the following parameters: dependency strength of the first type of message; dependency relationship; grouping information; processing type; and importance identifier.

[0086] It should be noted that the explanations of parameters such as the first order indication information, first type message, second type message, first identifier, second identifier, importance identifier, importance identifier, grouping information, and first type message dependence intensity in the video processing method applied to the encoding end can be found in the description of the corresponding parameters in the above-mentioned video processing method applied to the decoding end, and will not be repeated here.

[0087] In the following embodiments 1 to 7, each message in the message set can be an SEI message, the first type of message refers to an important SEI message (i.e., an SEI message with an importance identifier of true or 1), the second type of message refers to an unimportant SEI message (i.e., an SEI message with an importance identifier of false or 0), the first type of message dependence intensity is called the important SEI message dependence intensity, the first order indication information is called SEI processing order indication information, or SEI processing order SEI message (i.e., SEI processing order SEI message), and the processing order can also be called the preferred processing order. For example, the video processing process is described.

[0088] In a first embodiment, FIG4 is a flowchart of a video processing method applied to a decoder, provided in an embodiment of the present application. This embodiment is applied to an apparatus for decoding a video stream. The apparatus inputs a video image stream or a media file, and outputs a decoded and reconstructed video image. As shown in FIG4 , this embodiment includes steps S310-S340.

[0089] S310: Receive a video image code stream and its corresponding SEI processing order indication information for indicating the processing order of multiple SEI messages.

[0090] Among the multiple SEI messages, at least one is an important SEI message. Whether the SEI message is important is determined based on the importance flag of the SEI message included in the SEI processing order indication information. For example, if the importance flag of the SEI message is true (i.e., sei_importance_flag = 1), it indicates that the SEI message is important. Conversely, if the importance flag of the SEI message is false (i.e., sei_importance_flag = 0), it indicates that the SEI message is not important.

[0091] S320: When at least one important SEI message cannot be processed, determine a processing rule for the SEI message associated with the SEI processing order indication information.

[0092] The processing rule may be determined according to the processing type (ProcessingType) in the SEI processing order indication information. The processing type is used to indicate the processing rule for other SEI messages when an important SEI message (ie, sei_importance_flag=1) cannot be processed.

[0093] The processing rules can also be determined based on the dependency relationship in the SEI processing order indication information. The dependency relationship is used to indicate the dependency relationship between different types of SEI messages. The dependency relationship is used to indicate whether the SEI message associated with the dependency relationship is dependent on the corresponding important SEI message; or it is used to indicate the important SEI message on which the SEI message associated with the dependency relationship is dependent.

[0094] The processing rule may also be determined based on the dependency strength of the important SEI message in the SEI processing order indication information, where the dependency strength is used to indicate the degree of dependency of the important SEI message associated with the dependency strength on other SEI messages following it in the processing order.

[0095] The processing rule may also be determined according to the grouping information in the SEI processing order indication information, where the grouping information is used to indicate the groups to which different types of SEI messages are located.

[0096] The processing rule may also be determined according to the importance flag in the SEI processing order indication information, where the importance flag is used to indicate the importance of each first type message.

[0097] In addition to being any of the processing rules mentioned above, the processing rule may also be any combination of the different processing rules listed above.

[0098] It should be noted that different types of SEI messages may include but are not limited to the following: SEI messages with different message types (Payload Type); SEI messages with the same message type but different message contents; SEI messages with different importance identifiers, etc.

[0099] S330: Process the SEI processing order indication information according to the processing rule.

[0100] Determine, according to the processing rule determined in S320, a processing method for the multiple SEI messages associated with the SEI processing order indication information, including but not limited to:

[0101] The processing rules are determined according to the processing type ProcessingType (or processing degree flag process_degree_flag), and when ProcessingType (or process_degree_flag) is 1, if the SEI message with the importance flag set to true cannot be processed, the entire SEI processing order indication information is ignored; when ProcessingType (or process_degree_flag) is 0, if the SEI message with the importance flag set to true cannot be processed, only the processing order of all subsequent SEI messages of the SEI message with the importance flag set to true in the SEI processing order indication information is ignored.

[0102] In a specific implementation, the above processing rules can be implemented by constructing a message processing list (PoSeiList), for example:

[0103] Set the initial PoSeiList to empty.

[0104] If the decoding system is able to interpret and support the function indicated by the i-th message seiA in the SEI processing order indication information, seiA is added to the end of PoSeiList.

[0105] On the other hand, if the decoding system cannot interpret or support the function indicated by the i-th message seiA in the SEI processing order indication information, if the importance flag of seiA is equal to 1 and the processing type value (ProcessingType or process_degree_flag) is equal to 0, the construction of PoSeiList is terminated. If the importance flag of seiA is equal to 1 and the processing type value (ProcessingType or process_degree_flag) is equal to 1, PoSeiList is set to empty and the derivation of PoSeiList is terminated.

[0106] It should be noted that the processing type values ​​here are only examples. When the processing type value is 0, the entire SEI processing order indication information is ignored. When the processing type value is 1, only the processing order of all SEI messages subsequent to the SEI message with the importance flag set to true in the SEI processing order indication information is ignored. This description will not be repeated in subsequent embodiments.

[0107] The processing rules are determined according to the dependency relationship between SEI messages. If the SEI message with the importance flag set to true cannot be processed, only the processing order of the SEI messages that have a dependency relationship with the SEI message with the importance flag set to true in the SEI processing order indication information is ignored.

[0108] The processing rules are determined based on the dependency strength of the important SEI message. If the important SEI message associated with the dependency strength cannot be processed and the dependency strength is strong, the entire SEI processing order indication information is ignored. If the important SEI message associated with the dependency strength cannot be processed and the dependency strength is not strong, only the processing order of the first number of SEI messages in the SEI processing order indication information is ignored. The first number can be carried in the SEI processing order indication information, or it can be a default configuration, or it can be the number of SEI messages between the important SEI message and the next important SEI message.

[0109] The processing rules can also be determined based on the grouping information in the SEI processing order indication information. If the SEI message with the importance flag set to true cannot be processed, only the processing order of the SEI messages with the same grouping information as the important SEI message in the SEI processing order indication information is ignored.

[0110] S340: Process the video image code stream or the decoded video image according to the processed SEI processing order indication information.

[0111] The SEI processing order indication information is processed according to any processing rule in S330 to obtain processed SEI processing order indication information. If the processed SEI processing order indication information still indicates the processing order of at least two SEI messages, the video image stream or the decoded video image is processed according to the processing order of the at least two SEI messages indicated by the processed SEI processing order indication information, that is, the operations corresponding to the two SEI messages are respectively performed on the video image stream or the decoded video image according to the processing order of the at least two SEI messages.

[0112] In the second embodiment, this embodiment provides an example of SEI processing order indication information when determining a processing rule according to a processing type. The SEI processing order indication information is an SEI processing order SEI message.

[0113] Semantic information:

[0114] po_id: an identification number of a SEI message used to indicate the SEI processing order;

[0115] po_num_sei_messages_minus2: plus 2 to indicate the number of SEI message types that indicate the preferred processing order in the SEI processing order SEI message;

[0116] po_processing_type: indicates the processing rules for the SEI processing order SEI message or other SEI messages in the SEI processing order SEI message when there is an SEI message type in the SEI processing order SEI message that cannot be processed; for example: when the value is 1, the entire SEI processing order SEI message is ignored, that is, the preferred processing order between all SEI message types indicated in the SEI processing order SEI message is ignored; when the value is 0, all SEI messages after the processing order of the SEI message with the importance flag that cannot be processed set to true in the SEI processing order indication information are ignored, or the processing order of all SEI messages after the processing order of the SEI message that cannot be processed in the SEI processing order indication information is ignored.

[0117] po_processing_type[i]: indicates the processing rules for the SEI processing order SEI message or other SEI messages in the SEI processing order SEI message when the SEI message associated with the i-th SEI message type in the SEI processing order SEI message cannot be processed; for example: when the value is 1, the entire SEI processing order SEI message is ignored, that is, the preferred processing order between all SEI message types indicated in the SEI processing order SEI message is ignored; when the value is 0, the i-th message and all subsequent SEI messages are ignored, or the processing order of all SEI messages after the processing order of the i-th SEI message in the SEI processing order indication information is ignored.

[0118] It should be noted that the processing type can also be used in combination with other processing rules. That is, when the value of po_processing_type is 1, the processing rules in Examples 3 to 5 can be further combined to determine the final processing rules for all SEI messages or SEI message processing orders after the processing order of the SEI message whose importance identifier cannot be processed is true in the SEI processing order indication information.

[0119] po_sei_wrapping_flag[i]: When the value is 1, it is used to indicate that the i-th SEI message (type) is encapsulated in the SEI processing order nested SEI message corresponding to the SEI processing order SEI message; when the value is 0, it is used to indicate that the i-th SEI message exists independently and is not encapsulated in the SEI processing order nested SEI message corresponding to the SEI processing order SEI message.

[0120] po_sei_importance_flag[i]: indicates the importance determined by the encoder for the SEI message type with index i. If the value is 1, it indicates that the SEI message type with index i is important; if the value is 1, it indicates that the SEI message type with index i is not important.

[0121] po_sei_payload_type[i]: Specifies the payloadType value of the i-th type of SEI message. The type of SEI message can be determined based on the payloadType value.

[0122] po_sei_processing_order[i]: indicates the preferred processing order of the i-th type of sei message for which preferred processing order information is provided in the SEI processing order SEI message.

[0123] po_sei_wrapping_flag[i]: When the importance of the i-th SEI message type is not 1 (i.e., the i-th SEI message type is not important), if the value is equal to 1, it is used to indicate that the i-th SEI message is encapsulated in the SEI processing order nested SEI message corresponding to the SEI processing order SEI message; if the value is equal to 0, it is used to indicate that the i-th SEI message exists independently and is not encapsulated in the SEI processing order nested SEI message corresponding to the SEI processing order SEI message.

[0124] In subsequent embodiments, this extension will no longer be specifically described. The encapsulation flag (po_sei_wrapping_flag) may be set for each SEI message in the SEI processing order SEI message, or the encapsulation flag (po_sei_wrapping_flag) may be set only for the SEI message whose importance flag (po_sei_importance_flag) in the SEI processing order SEI message is false (i.e., the second type of message in the above embodiment).

[0125] In the third embodiment, this embodiment provides an example of SEI processing order indication information when determining a processing rule according to a dependency relationship. The SEI processing order indication information is an SEI processing order SEI message.

[0126] Semantic information (the semantic information of the repeated grammatical elements in all subsequent embodiments will not be explained, and the description of the above embodiment 2 will be directly referred to):

[0127] po_sei_dependence_flag[i]: indicates the dependency of the i-th SEI message type on the important SEI message. If the value is 0, it indicates that the i-th SEI message type has no dependency on the important SEI message; if the value is 1, it indicates that the i-th SEI message type has a dependency on the important SEI message.

[0128] At this time, the processing rules are: when there is no dependency between an SEI message type and an important SEI message, even if the important SEI message cannot be processed or executed, the SEI message is still executed normally according to the processing order; when there is a dependency between an SEI message type and an important SEI message, and the important SEI message cannot be processed or executed, the SEI message is ignored or the processing order of the SEI message is ignored.

[0129] Semantic information:

[0130] po_sei_dependence_present_flag: When the value is equal to 0, it is used to indicate that there is no dependency identifier (po_sei_dependence_flag) in the SEI processing order SEI message; when the value is equal to 1, it is used to indicate that there is no dependency identifier (po_sei_dependence_flag) in the SEI processing order SEI message.

[0131] The dependency presence flag (po_sei_dependence_present_flag) may depend on whether there is an SEI message with an importance flag set to true among the multiple SEI messages associated with the SEI processing order indication information. If there is an SEI message with an importance flag set to true, the dependency presence flag takes a value of 1; if there is no SEI message with an importance flag set to true, the dependency presence flag takes a value of 0.

[0132] po_sei_dependence_flag[i]: Same as the description in the third embodiment above.

[0133] Semantic information:

[0134] po_num_important_messages_minus1: plus 1 indicates the number of SEI messages with the importance flag associated with the SEI processing order indication information being true.

[0135] po_sei_dependence_flag[i]: indicates the dependency of the i-th SEI message type on important SEI messages. If the value is 0, it indicates that the i-th SEI message type has no dependency on any important SEI message; if the value is 1, it indicates that the i-th SEI message type has a dependency on any important SEI message.

[0136] po_sei_dependence_idx[i]: used to indicate the index number of the important SEI message on which the i-th SEI message type depends.

[0137] It should be noted that, although in this embodiment, an SEI message type can only depend on one important SEI message, the present application solution does not limit the number of important SEI messages that an SEI message type can depend on. When an SEI message type depends on multiple important SEI messages at the same time, the dependency index (po_sei_dependence_idx[i]) can be used to indicate the index numbers of the multiple important SEI messages that the i-th SEI message type depends on.

[0138] In the fourth embodiment, this embodiment provides an example of SEI processing order indication information when a processing rule is determined according to the dependency strength of an important SEI message. The SEI processing order indication information is an SEI processing order SEI message.

[0139] Semantic information:

[0140] po_sei_strong_influence_flag[i]: When the value is 1 and the associated important SEI message cannot be processed, all SEI messages or the processing order of SEI messages after the important SEI message can be ignored; when the value is 0 and the associated important SEI message cannot be processed, the first number of SEI messages or the processing order of SEI messages after the important SEI message can be ignored.

[0141] It should be noted that, when there are multiple important SEI messages, if the po_sei_strong_influence_flag value is 1, it can also be used to indicate that when the associated important SEI message cannot be processed, all SEI messages or the processing order of SEI messages that are after the important SEI message and before the next SEI message with the po_sei_strong_influence_flag value of 1 are ignored.

[0142] po_num_influence_sei_messages_minus1: plus 1 indicates the number of affected SEI messages, that is, the number of SEI messages or SEI message smooth sequences that can be ignored in the processing order after the important SEI message if its associated important SEI message cannot be processed.

[0143] In the fifth embodiment, this embodiment provides an example of SEI processing order indication information when determining a processing rule according to grouping information in SEI processing order indication information. The SEI processing order indication information is an SEI processing order SEI message.

[0144] Semantic information:

[0145] po_sei_group_id[i]: indicates the group ID that the i-th SEI message is in. In each group, at least one SEI message has its importance flag set to true (ie, at least one first-type message).

[0146] The processing rule at this time is: when an important SEI message cannot be processed or executed, all SEI messages or the processing order of SEI messages with the same group identifier as the important SEI message can be ignored.

[0147] Semantic information:

[0148] po_sei_group_present_flag: If the value is equal to 0, it is used to indicate that the group identifier (po_sei_group_id) does not exist in the SEI processing order SEI message; if the value is equal to 1, it is used to indicate that the group identifier (po_sei_group_id) exists in the SEI processing order SEI message.

[0149] The group presence flag (po_sei_group_present_flag) may depend on whether there is an SEI message with an importance flag set to true among the multiple SEI messages associated with the SEI processing order indication information. If there is an SEI message with an importance flag set to true, the group presence flag takes a value of 1; if there is no SEI message with an importance flag set to true, the group presence flag takes a value of 0.

[0150] Semantic information:

[0151] po_processing_type: When the value is 0 and the SEI message type with the importance flag set to true in the SEI processing order SEI message cannot be processed, the entire SEI processing order SEI message can be ignored, that is, the preferred processing order between all SEI message types associated in the SEI processing order SEI message is ignored; when the value is 1 and the SEI message type with the importance flag set to true in the SEI processing order SEI message cannot be processed, some SEI messages associated with the SEI processing order SEI message or the processing order of some SEI messages are ignored.

[0152] po_sei_group_id[i]: If the processing type (po_processing_type) value is 1, it is used to indicate the group ID of the i-th SEI message. In each group, at least one SEI message has its importance flag set to true (i.e., 1).

[0153] The final processing rule at this time is: if the processing type (po_processing_type) value is 0 and an important SEI message cannot be processed or executed, the entire SEI processing order SEI message is ignored, that is, the preferred processing order between all SEI message types associated in the SEI processing order SEI message is ignored; conversely, if the processing type (po_processing_type) value is 1, all SEI messages or SEI message processing orders with the same group identifier as the important SEI message are ignored.

[0154] In the sixth embodiment, this embodiment provides an example of SEI processing order indication information when determining a processing rule according to SEI processing order indication information, and the SEI processing order indication information is an SEI processing order SEI message.

[0155] Semantic information:

[0156] When the importance flag (po_sei_importance_flag[i]) of an SEI message takes the value of 1, that is, when the SEI message is an important SEI message, the importance flag (po_sei_importance_degree_flag[i]) can be further set for the SEI message. The importance flag is equivalent to the processing degree flag po_sei_processing_degree_flag.

[0157] po_sei_importance_degree_flag[i]: If the value is equal to 1 and indicates that an important SEI message cannot be processed correctly, the decoder should ignore the entire SEI processing order SEI message, that is, the preferred processing order between all SEI message types associated in the SEI processing order SEI message is ignored; when po_sei_importance_degree_flag[i] is equal to 0, it indicates that the decoder should still process all SEI messages whose po_sei_processing_order[i] is less than or equal to the po_sei_processing_order[i] of the aforementioned important SEI message that cannot be processed in the processing order.

[0158] The main difference from Example 6 is that in this embodiment, when the importance flag (po_sei_importance_flag[i]) of an SEI message is 1, that is, the SEI message is an important SEI message, and the encapsulation flag value of the important SEI message is equal to 0, that is, the important SEI message is not encapsulated in a processing order nesting SEI message (Processing order nesting SEI message), the importance flag (po_sei_importance_degree_flag[i]) is further set for the SEI message, and the importance flag is equivalent to the processing degree flag po_sei_processing_degree_flag.

[0159] In a seventh embodiment, FIG5 is a flow chart of a video processing method applied to an encoding end provided by an embodiment of the present application. This embodiment is applied to an apparatus for encoding an input video image. The input of the apparatus is a source video image, and the output is a video image code stream or a media file.

[0160] S410: Determine a processing order of multiple SEI messages associated with a video image.

[0161] Multiple SEI messages associated with a video image refer to information carried in multiple SEI messages that is used in decoding, displaying, or other related processes after the video image is encoded. Multiple SEI messages are different types of SEI messages, including but not limited to SEI messages with different message types (Payload Type), SEI messages of the same message type but different message contents, or SEI messages with different importance identifiers.

[0162] The encoder can determine the preferred processing order of multiple SEI messages associated with the video image based on the SEI message type, information carried in the SEI message, decoded reconstructed image quality, configuration file, etc.

[0163] S420: Determine the SEI processing order indication message and processing rules according to the processing order.

[0164] The SEI processing order indication message is used to indicate the preferred processing order of multiple SEI messages. The processing rule can be any of the processing rules mentioned in Examples 1 to 5, or any combination of multiple processing rules. The processing rule can be included in the SEI processing order indication message or determined based on the SEI processing order indication message.

[0165] In addition, the SEI processing order indication message may also include one or more information including the number of associated SEI messages, the type of associated SEI messages, the importance identifier of the SEI messages, the encapsulation identifier of the SEI messages, and the like.

[0166] The processing rule is used to indicate the preferred processing order in the SEI processing order indication message or the processing method for SEI messages other than the non-executable SEI message in the multiple SEI messages when there is an unexecutable SEI message in the multiple SEI messages. Furthermore, it is also possible to limit the preferred processing order in the SEI processing order indication message or the processing method for SEI messages other than the non-executable SEI message in the multiple SEI messages to only when the importance flag of the unexecutable SEI message is true.

[0167] S430: Send the SEI processing order indication message and processing rules to the decoding end.

[0168] The SEI processing order indication message and processing rules can be added to the bitstream after video image encoding and sent to the decoding end together, or the SEI processing order indication message and processing rules can be sent to the decoding end as a separate bitstream, or the SEI processing order indication message and processing rules can be sent to the decoding end through any other method.

[0169] In one embodiment, FIG6 is a block diagram of a video processing device provided by an embodiment of the present application. This embodiment is applied to a decoding end. As shown in FIG6 , the video processing device in this embodiment includes: a receiver 510, a first determination module 520, a first processing module 530, and a second processing module 540.

[0170] The receiver 510 is configured to receive a video image code stream and first sequence indication information corresponding to the video image code stream; wherein the first sequence indication information is used to indicate the processing order of each message in a message set, and the message set includes at least one first type message.

[0171] The first determining module 520 is configured to determine a message processing rule for each message associated with the first sequence indication information when at least one message of the first type cannot be processed.

[0172] The first processing module 530 is configured to process the first sequence indication information according to a message processing rule.

[0173] The second processing module 540 is configured to process the video image code stream according to the processed first sequence indication information.

[0174] In one embodiment, when the importance identifier of the message is the first identifier, the message is a first type message;

[0175] When the importance identifier of the message is the second identifier, the message is a second type message.

[0176] In one embodiment, determining a message processing rule for each message associated with the first sequence indication information includes at least one of the following:

[0177] Determining a message processing rule for each associated message according to the processing type in the first order indication information;

[0178] Determining a message processing rule for each associated message according to the dependency relationship in the first order indication information;

[0179] Determining a message processing rule for each associated message according to the first type message dependency strength in the first order indication information;

[0180] determining a message processing rule for each associated message according to the grouping information in the first order indication information;

[0181] A message processing rule associated with each message is determined according to the importance identifier in the first order indication information.

[0182] In one embodiment, processing the first order indication information according to the message processing rule includes:

[0183] If the processing type in the first sequence indication information is the first identifier and the first type message cannot be processed, the processing order of subsequent messages corresponding to the first type message in the first sequence indication information is ignored;

[0184] When the processing type in the first sequence indication information is the second identifier and the first type message cannot be processed, the entire first sequence indication information is ignored.

[0185] In one embodiment, processing the first order indication information according to the message processing rule includes:

[0186] When the processing type in the first sequence indication information is the first identifier and the encapsulation identifier is the second identifier, if the first type message cannot be processed, retaining the processing order of the messages before the processing order of the first type message in the first sequence indication information;

[0187] When the processing type in the first sequence indication information is the first identifier and the encapsulation identifier is the first identifier, if the first type message cannot be processed, ignoring the first type message that cannot be processed in the first sequence indication information;

[0188] In the case where the processing type in the first sequence indication information is the second identifier, if the first type of message cannot be processed, the entire first sequence indication information is ignored.

[0189] In one embodiment, processing the first order indication information according to the message processing rule includes:

[0190] In a case where the first type message associated with the first sequence indication information cannot be processed, the processing order of all messages in the first sequence indication information that have a dependency relationship with the first type message is ignored.

[0191] In one embodiment, processing the first order indication information according to the message processing rule includes:

[0192] When the first type of message associated with the first sequence indication information is identified by a second dependency strength, ignoring the processing order of a first number of messages following the first type of message in the first sequence indication information;

[0193] In a case where the first type of message associated with the first sequence indication information is identified by a first dependency strength, the entire first sequence indication information is ignored.

[0194] In one embodiment, processing the first order indication information according to the message processing rule includes:

[0195] In the case that the first type message associated with the first sequence indication information cannot be processed, the processing order of all messages in the first sequence indication information that have the same grouping information as that corresponding to the first type message is ignored.

[0196] In one embodiment, processing the first order indication information according to the message processing rule includes:

[0197] If the importance identifier in the first sequence indication information is the first identifier and the first type of message cannot be processed, ignore the entire first sequence indication information;

[0198] If the importance identifier in the first sequence indication information is the second identifier, the encapsulation identifier is the second identifier, and the first type message cannot be processed, retain the processing order of messages before the processing order corresponding to the first type message in the first sequence indication information, or ignore the processing order of subsequent messages corresponding to the first type message in the first sequence indication information;

[0199] When the importance identifier in the first sequence indication information is the second identifier, the encapsulation identifier is the first identifier, and the first type message cannot be processed, the first type message that cannot be processed in the first sequence indication information is ignored.

[0200] In one embodiment, the first sequence indication information includes at least the following parameters: message identifier; number of messages; message nesting; importance identifier; message type; processing order of messages corresponding to each message type; and encapsulation identifier.

[0201] In one embodiment, the first sequence indication information further includes one of the following parameters: dependency strength of the first type of message; dependency relationship; grouping information; processing type; and importance identifier.

[0202] The video processing device provided in this embodiment is configured to implement the video processing method applied to the decoding end of the embodiment shown in FIG2 . The implementation principle and technical effects of the video processing device provided in this embodiment are similar and will not be described in detail here.

[0203] In one embodiment, FIG7 is a block diagram of another video processing device provided by an embodiment of the present application. This embodiment is applied to an encoding end. As shown in FIG7 , the video processing device in this embodiment includes: a first determination module 610, a second determination module 620, and a transmitter 630.

[0204] The first determining module 610 is configured to determine a processing order of each message in a message set associated with a source image.

[0205] The second determination module 620 is configured to determine corresponding first sequence indication information based on the processing order of each message; wherein the first sequence indication information is used to indicate the processing order of each message in a message set, and the message set includes at least one first type message.

[0206] The transmitter 630 is configured to add the first sequence indication information to the video image code stream corresponding to the source image, and send it to the decoding end.

[0207] In one embodiment, when the importance identifier of the message is the first identifier, the message is a first type message;

[0208] When the importance identifier of the message is the second identifier, the message is a second type message.

[0209] In one embodiment, a corresponding encapsulation identifier is configured for the message whose importance identifier is the second identifier in the first sequence indication information.

[0210] In one embodiment, when the number of first type messages in the first sequence indication information is greater than or equal to 1, the corresponding dependency relationship existence flag is configured as the first flag;

[0211] When the number of first type messages in the first sequence indication information is 0, the corresponding dependency existence identifier is configured as the second identifier.

[0212] In one embodiment, when the number of first type messages in the first sequence indication information is greater than 1, for a message whose dependency identifier is the first identifier, a dependent message index of the first type message on which it depends is configured.

[0213] In one embodiment, when the number of messages whose importance identifier is the first identifier in the first sequence indication information is greater than or equal to 1, the group existence identifier is configured as the first identifier;

[0214] When the number of messages with the first importance identifier in the first sequence indication information is 0, the group existence identifier is configured as the second identifier.

[0215] In one embodiment, when the group existence identifier is the first identifier, a group identifier corresponding to each message is configured; wherein the group corresponding to each group identifier contains at least one first type message.

[0216] In one embodiment, a corresponding importance identifier is configured for the first type message whose encapsulation identifier is the second identifier.

[0217] In one embodiment, the first sequence indication information includes at least the following parameters: message identifier; number of messages; message nesting; importance identifier; message type; processing order of messages corresponding to each message type; and encapsulation identifier.

[0218] In one embodiment, the first sequence indication information further includes one of the following parameters: dependency strength of the first type of message; dependency relationship; grouping information; processing type; and importance identifier.

[0219] The video processing device provided in this embodiment is configured to implement the video processing method applied to the encoding end in the embodiment shown in FIG3 . The implementation principle and technical effects of the video processing device provided in this embodiment are similar and will not be described in detail here.

[0220] In one embodiment, FIG8 is a schematic diagram of the structure of a communication device provided by an embodiment of the present application. As shown in FIG8 , the device provided by the present application includes: a processor 710, a memory 720, and a communication module 730. The number of processors 710 in the device can be one or more, and FIG8 uses one processor 710 as an example. The number of memories 720 in the device can be one or more, and FIG8 uses one memory 720 as an example. The processor 710, memory 720, and communication module 730 of the device can be connected via a bus or other means, and FIG8 uses a bus connection as an example. In this embodiment, the device can be a decoding end or an encoding end.

[0221] The memory 720, as a computer-readable storage medium, can be configured to store software programs, computer executable programs, and modules, such as program instructions / modules corresponding to the device of any embodiment of the present application (for example, the receiver 510, the first determination module 520, the first processing module 530, and the second processing module 540 in the video processing device applied to the decoding end). The memory 720 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 720 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 720 may further include a memory remotely located relative to the processor 710, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0222] In the case where the communication device serves as a decoding end, the above-provided device can be configured to execute the video processing method applied to the decoding end provided in any of the above-mentioned embodiments, and have corresponding functions and effects.

[0223] In the case where the communication device serves as the encoding end, the above-mentioned device can be configured to execute the video processing method applied to the encoding end provided in any of the above-mentioned embodiments, and have corresponding functions and effects.

[0224] An embodiment of the present application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to execute a video processing method applied to a decoding end, the method comprising: receiving a video image stream and first sequence indication information corresponding to the video image stream; wherein the first sequence indication information is used to indicate the processing order of each message in a message set, and the message set includes at least one first type message; when at least one first type message cannot be processed, determining a message processing rule for each message associated with the first sequence indication information; processing the first sequence indication information according to the message processing rule; and processing the video image stream according to the processed first sequence indication information.

[0225] An embodiment of the present application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to execute a video processing method applied to an encoding end, the method comprising: determining the processing order of each message in a message set associated with a source image; determining corresponding first sequence indication information based on the processing order of each message; wherein the first sequence indication information is used to indicate the processing order of each message in the message set, and the message set includes at least one first type message; adding the first sequence indication information to the video image code stream corresponding to the source image, and sending it to the decoding end.

[0226] It will be appreciated by those skilled in the art that the term user equipment encompasses any suitable type of wireless user equipment, such as a mobile phone, a portable data processing device, a portable web browser or a car-mounted mobile station.

[0227] In general, various embodiments of the present application may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device, although the present application is not limited thereto.

[0228] Embodiments of the present application may be implemented by executing computer program instructions by a data processor of a mobile device, for example, in a processor entity, or by hardware, or by a combination of software and hardware. The computer program instructions may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages.

[0229] The block diagram of any logic flow in the drawings of the present application may represent program steps, or may represent interconnected logic circuits, modules and functions, or may represent a combination of program steps and logic circuits, modules and functions. A computer program may be stored on a memory. The memory may be of any type suitable for the local technical environment and may be implemented using any suitable data storage technology, such as, but not limited to, read-only memory (ROM), random access memory (RAM), optical storage devices and systems (digital versatile discs (DVD) or compact disks (CD)), etc. Computer-readable media may include non-transient storage media. A data processor may be of any type suitable for the local technical environment, such as, but not limited to, a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), and a processor based on a multi-core processor architecture.

[0230] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A video processing method, applied to a decoding end, includes: Receiving a video image bitstream and first-order indication information corresponding to the video image bitstream; wherein, the first-order indication information is used to indicate the processing order of each message in a message set, and at least one first-type message is included in the message set; Determining a message processing rule for each message associated with the first-order indication information when at least one of the first-type messages cannot be processed; Processing the first-order indication information according to the message processing rule; Processing the video image bitstream according to the processed first-order indication information.

2. The method according to claim 1, wherein, When the importance identifier of the message is a first identifier, the message is a first-type message; When the importance identifier of the message is a second identifier, the message is a second-type message.

3. The method according to claim 1, wherein, The determining the message processing rule for each message associated with the first-order indication information includes at least one of the following: Determining the message processing rule for each associated message according to the processing type in the first-order indication information; Determining the message processing rule for each associated message according to the dependency relationship in the first-order indication information; Determining the message processing rule for each associated message according to the dependency strength of the first-type message in the first-order indication information; Determining the message processing rule for each associated message according to the grouping information in the first-order indication information; Determining the message processing rule for each associated message according to the importance identifier in the first-order indication information.

4. The method according to claim 1, wherein The processing the first-order indication information according to the message processing rule includes: When the processing type in the first-order indication information is a first identifier and the first-type message cannot be processed, ignoring the processing order of subsequent messages corresponding to the first-type message in the first-order indication information; When the processing type in the first-order indication information is a second identifier and the first-type message cannot be processed, ignoring the entire first-order indication information.

5. The method according to claim 1, wherein The processing the first-order indication information according to the message processing rule includes: When the processing type in the first-order indication information is a first identifier and the encapsulation identifier is a second identifier, in response to the first-type message not being processable, retaining the processing order between messages before the processing order of the first-type message in the first-order indication information; When the processing type in the first-order indication information is a first identifier and the encapsulation identifier is a first identifier, in response to the first-type message not being processable, ignoring the unprocessable first-type message in the first-order indication information; When the processing type in the first-order indication information is a second identifier, in response to the first-type message not being processable, ignoring the entire first-order indication information.

6. The method according to claim 1, wherein The processing the first-order indication information according to the message processing rule includes: In the case where the first type of message associated with the first-order indication information cannot be processed, ignore the processing order of all messages in the first-order indication information that have a dependency relationship with the first type of message.

7. The method according to claim 1, wherein The processing of the first-order indication information according to the message processing rule includes: In the case where the first type of message associated with the first-order indication information is dependent with a strength of a second identifier, ignore the processing order of the first quantity of messages after the first type of message in the first-order indication information; In the case where the first type of message associated with the first-order indication information is dependent with a strength of a first identifier, ignore the entire first-order indication information.

8. The method according to claim 1, wherein, The processing of the first-order indication information according to the message processing rule includes: In the case where the first type of message associated with the first-order indication information cannot be processed, ignore the processing order of all messages in the first-order indication information that have the same grouping information as the first type of message.

9. The method according to claim 1, wherein The processing of the first-order indication information according to the message processing rule includes: In the case where the importance identifier in the first-order indication information is a first identifier and the first type of message cannot be processed, ignore the entire first-order indication information; In the case where the importance identifier in the first-order indication information is a second identifier and the encapsulation identifier is a second identifier and the first type of message cannot be processed, retain the processing order among the messages before the processing order corresponding to the first type of message in the first-order indication information, or ignore the processing order among the subsequent messages corresponding to the first type of message in the first-order indication information; In the case where the importance identifier in the first-order indication information is a second identifier and the encapsulation identifier is a first identifier and the first type of message cannot be processed, ignore the first type of message that cannot be processed in the first-order indication information.

10. The method according to claim 1, wherein, The first-order indication information at least includes the following parameters: message identifier; message quantity; message nesting situation; importance identifier; message type; processing order of each message corresponding to the message type; encapsulation identifier.

11. The method according to claim 10, wherein The first-order indication information further includes one of the following parameters: dependency strength of the first type of message; dependency relationship; grouping information; processing type; importance identifier.

12. A video processing method, applied to an encoding end, includes: Determine the processing order of each message in the message set associated with the source image; Determine corresponding first-order indication information based on the processing order of each message; wherein, the first-order indication information is used to indicate the processing order of each message in the message set, and at least one first type of message is included in the message set; Add the first-order indication information to the video image bitstream corresponding to the source image and send it to the decoding end.

13. The method according to claim 12, wherein, In the case where the importance identifier of the message is a first identifier, the message is a first type of message; In the case where the importance identifier of the message is a second identifier, the message is a second type of message.

14. The method according to claim 12, wherein, Configure a corresponding encapsulation identifier for a message with an importance identifier of the second identifier in the first-order indication information.

15. The method according to claim 12, wherein, When the number of messages of the first type in the first-order indication information is greater than or equal to 1, configure the corresponding dependency existence identifier as the first identifier; When the number of messages of the first type in the first-order indication information is 0, configure the corresponding dependency existence identifier as the second identifier.

16. The method according to claim 12, wherein, When the number of messages of the first type in the first-order indication information is greater than 1, for a message with a dependency identifier of the first identifier, configure the dependency message index of the first type of message it depends on.

17. The method according to claim 12, wherein When the number of messages with an importance identifier of the first identifier in the first-order indication information is greater than or equal to 1, configure the group existence identifier as the first identifier; When the number of messages with an importance identifier of the first identifier in the first-order indication information is 0, configure the group existence identifier as the second identifier.

18. The method according to claim 17, wherein, When the group existence identifier is the first identifier, configure a corresponding group identifier for each message; wherein, each group corresponding to the group identifier contains at least one message of the first type.

19. The method according to claim 14, wherein, For a message of the first type with an encapsulation identifier of the second identifier, configure a corresponding importance level identifier.

20. The method according to claim 12, wherein The first-order indication information at least includes the following parameters: message identifier; number of messages; message nesting situation; importance identifier; message type; processing order of messages corresponding to each message type; encapsulation identifier.

21. The method according to claim 20, wherein, The first-order indication information further includes one of the following parameters: dependency strength of the first type of message; dependency relationship; grouping information; processing type; importance level identifier.

22. A communication device, comprising: A memory, and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-11 or 12-21 above.

23. A storage medium storing a computer program, where the computer program, when executed by a processor, implements the method according to any one of claims 1-11 or 12-21 above.

Images