Segmentation method and device for effective compression of biomedical waveform signal, and method and device for transmitting segmentation information

Abstract

The present invention relates to a method and device for encoding / decoding a biomedical waveform signal, the method comprising the steps of: deriving a sub-block or a lower sub-block of the current block by a recursive binary segmentation method, deriving a prediction unit and an inverse transform unit on the basis of at least one among the sub-block and the lower sub-block, and performing prediction on the basis of the prediction unit; and performing inverse transform on the basis of the inverse transform unit, wherein the lower sub-block is divided into a prediction sub-block and a transform sub-block.

Description

Method and apparatus for segmentation and segmentation information transmission for effective compression of biomedical waveform signals

[0001] The present invention relates to biosignal encoding technology, and more specifically, to an encoding / decoding device and method for precisely compressing and restoring biomedical waveform data, such as electrocardiogram (ECG) and brainwave (EEG), in real time.

[0002] Recently, the volume of high-resolution biosignal data has been rapidly increasing due to the proliferation of telemedicine and wearable healthcare devices. In particular, electrocardiogram (ECG), electroencephalogram (EEG), or electromyogram (EMG) signals are characterized by being transmitted and received through multiple channels and requiring continuous recording over long periods. Accordingly, high-performance compression technology is required to minimize distortion of biosignals and efficiently transmit signals within a limited bandwidth.

[0003] The present disclosure aims to provide a method and apparatus for encoding / decoding a biomedical waveform signal.

[0004] The present disclosure aims to provide a method for dividing a current block based on a recursive binary partitioning method, a method for deriving a subblock and a lower subblock according to the partitioning, and an apparatus.

[0005] The present disclosure aims to provide a method and apparatus for determining a prediction unit for each prediction sub-block or determining a conversion unit for each conversion sub-block.

[0006] The present disclosure aims to provide a method and apparatus for signaling partitioning information according to a recursive binary partitioning method and performing prediction and transformation based on said partitioning information.

[0007] The present disclosure aims to provide a method and apparatus for obtaining a restored signal according to the prediction method and the conversion method.

[0008] The biomedical waveform signal decoding method and apparatus according to the present disclosure can derive a subblock or a lower subblock of a current block by a predetermined division method.

[0009] In a biomedical waveform signal decoding method and apparatus according to the present disclosure, the decoding method may include the steps of deriving a prediction unit and an inverse transformation unit based on at least one of the subblock or the sub-subblock, performing a prediction based on the prediction unit, and performing an inverse transformation based on the inverse transformation unit.

[0010] In the biomedical waveform signal decoding method and apparatus according to the present disclosure, the lower subblock may be divided into a prediction subblock and a conversion subblock.

[0011] In the biomedical waveform signal decoding method and apparatus according to the present disclosure, the prediction subblock is divided from the subblock, and the conversion subblock may be divided from at least one of the subblock or the prediction subblock.

[0012] In the biomedical waveform signal decoding method and apparatus according to the present disclosure, the splitting method may include a method performed recursively on a subblock of the current block or on a subblock of the current block.

[0013] In the biomedical waveform signal decoding method and apparatus according to the present disclosure, the segmentation method may be determined based on block segmentation information, and the block segmentation information may include at least one of whether segmentation of a block is allowed, a segmentation depth, or a segmentation ratio.

[0014] In the biomedical waveform signal decoding method and apparatus according to the present disclosure, the prediction unit or the inverse transformation unit may be derived based on at least one of the prediction subblock or the inverse transformation subblock.

[0015] In the biomedical waveform signal decoding method and apparatus according to the present disclosure, the prediction may be performed based on at least one of a reference sample or a pre-recovered sample, and the method of the prediction may include at least one of a weighted average prediction, an extrapolated prediction, or a prediction unit matching prediction.

[0016] In the biomedical waveform signal decoding method and apparatus according to the present disclosure, the reference sample may be derived based on at least one of a specific sample or a sample index among the previously reconstructed samples, and the sample index may be an index representing the size of the timestamp of the reference sample.

[0017] In the biomedical waveform signal decoding method and apparatus according to the present disclosure, the prediction method may be a cross-channel prediction, wherein the cross-channel prediction may be performed based on at least one of a single or multiple reference samples, pre-restored samples, or regression parameters.

[0018] In the biomedical waveform signal decoding method and apparatus according to the present disclosure, when the number of previously reconstructed samples is smaller than the sample index, the previously reconstructed samples can be generated through padding.

[0019] According to the present disclosure, a biomedical waveform signal can be restored by dividing the current block into a plurality of sub-blocks and lower sub-blocks.

[0020] According to the present disclosure, a subblock can be recursively binary partitioned into a prediction subblock and / or a transformation subblock.

[0021] According to the present disclosure, the method of recursive binary partitioning can be determined based on partitioning information.

[0022] According to the present disclosure, a prediction unit may be derived based on a prediction subblock, or a transformation unit may be derived based on a transformation subblock.

[0023] According to the present disclosure, a prediction may be performed per prediction unit, or a conversion (or / and a prediction for the conversion coefficient) may be performed per conversion unit.

[0024] According to the present disclosure, a restored signal can be derived by combining a predicted signal restored according to a prediction operation and a residual signal transformed according to a transformation operation.

[0025] FIG. 1 is a block diagram illustrating a method for decoding a biomedical waveform signal according to the present invention.

[0026] FIG. 2 illustrates a channel group divided into frames according to the present disclosure.

[0027] Figure 3 illustrates a frame recursively divided into blocks or sub-blocks.

[0028] Figure 4 illustrates the process of dividing a sub-block to induce a lower sub-block.

[0029] Figure 5 illustrates a prediction method according to the prediction unit when the prediction method for the current sub-block in the decoder is an in-channel prediction.

[0030] Figure 6 illustrates the process of performing a prediction according to the prediction unit when the prediction method for the current sub-block in the decoder is a prediction unit matching prediction.

[0031] Figure 7 illustrates an example of performing a prediction according to the prediction unit when the prediction method for the current sub-block in the decoder is cross-channel prediction.

[0032] FIG. 8 is a block diagram illustrating a signal encoding method according to the present invention.

[0033] Embodiments of the present invention are described in detail with reference to the drawings attached to this specification so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0034] Throughout this specification, when a part is described as being 'connected' to another part, this includes not only cases where they are directly connected, but also cases where they are electrically connected with other elements in between.

[0035] Furthermore, throughout this specification, when a part is described as 'comprising' a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0036] Additionally, terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0037] Additionally, in the embodiments relating to the device and method described herein, some components of the device or some steps of the method may be omitted. Also, the order of some components of the device or some steps of the method may be changed. Additionally, other components or other steps may be inserted into some components of the device or some steps of the method.

[0038] In addition, some components or steps of the first embodiment of the present invention may be added to or replace some components or steps of the second embodiment of the present invention.

[0039] Furthermore, the components shown in the embodiments of the present invention are depicted independently to represent different characteristic functions and do not imply that each component consists of separate hardware or a single software unit. That is, for convenience of explanation, each component is described by listing it as a separate component, and at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform a function. Such integrated and separated embodiments of each component are also included within the scope of the present invention as long as they do not deviate from the essence of the invention.

[0040] First, the terms used in this application are briefly explained as follows.

[0041] The signal decoding apparatus described below may be a device included in a private medical device, a private medical system, a research medical system, a research medical device, a personal medical terminal, a notebook computer, a smartphone, a TV application server, a service server, etc., and may refer to various devices equipped with a user terminal such as various devices, a communication device such as a communication modem for performing communication with a wired or wireless communication network, a memory for storing various programs and data for decoding signals or predicting between or within channels for decoding, and a microprocessor for executing programs to perform calculations and control.

[0042] Additionally, a signal encoded into a bitstream by an encoder can be transmitted to a signal decoding device in real-time or non-real-time via wired or wireless communication networks such as the Internet, local area wireless networks, wireless LAN networks, WiBro networks, and mobile communication networks, or through various communication interfaces such as cables and Universal Serial Bus (USB), to be decoded, restored, and reproduced in the form of a signal. Alternatively, the bitstream generated by the encoder can be stored in memory. The memory may include both volatile and non-volatile memory. In this specification, memory may be described as a recording medium that stores the bitstream.

[0043] Typically, a biomedical waveform signal has a series of channel groups. A channel group may consist of only one channel or may consist of multiple channels within the same biomedical waveform signal. A channel group may consist of a different number of channels or may be recursively divided in a hierarchical order of frames, blocks, and sub-subblocks.

[0044] As used in this specification, the term "current block" refers to a spatial unit constituting all or part of the reconstructed samples generated during the signal reconstruction process, and means a block or sub-subblock that is the subject of recursive binary division. In this case, the current block may correspond to a Coding Unit (CU), a Prediction Unit (PU), or a Transform Unit (TU). Furthermore, those skilled in the art to which this embodiment belongs will understand that the term Coding Unit may be replaced with other terms having equivalent meanings, such as unit block, unit sub-block, etc.

[0045] Hereinafter, embodiments of the present invention will be described in more detail with reference to the attached drawings. In describing the present invention, redundant descriptions of identical components are omitted.

[0046] FIG. 1 is a block diagram illustrating a method for decoding a biomedical waveform signal (hereinafter referred to as 'signal') according to the present invention.

[0047] Referring to FIG. 1, the signal decoding method according to the present invention may include a sub-block and lower sub-block derivation step (S100), a prediction unit and inverse transformation unit derivation step (S110), and an inverse transformation and prediction execution step (S120).

[0048] The sub-block and sub-sub-block derivation step (S100) is a step of dividing and selecting blocks and sub-blocks from a frame. A sub-block may be recursively divided into sub-sub-blocks. Alternatively, it may be divided from a sub-block in the order of prediction sub-blocks and transformation sub-blocks. Alternatively, a prediction sub-block or a transformation sub-block may be divided from a sub-block. Alternatively, the sub-block may not be further divided.

[0049] In the prediction unit and inverse transformation unit derivation step (S110), the prediction unit and the inverse transformation unit can be divided and selected, respectively. The prediction unit determination unit is a step of dividing and selecting the prediction unit from the sub-block. The inverse transformation unit determination unit is a step of dividing and selecting the inverse transformation unit from the sub-block. At this time, information regarding the division or selection can be signaled from the bitstream or derived during the decoding process.

[0050] The inverse transformation and prediction execution step (S120) is divided into an inverse transformation execution step and a prediction execution step.

[0051] In the inverse transformation execution step, the decoder signals the most appropriate inverse transformation kernel from the bitstream for each inverse transformation unit, and can derive residual samples from the transformation coefficients based on the inverse transformation kernels. In this disclosure, the transformation coefficients may be understood as being replaced with residual coefficients, residual transformation samples, or transformation samples. In this disclosure, the predicted samples may be understood as being replaced with predicted transformation coefficients, predicted residual samples, or predicted transformation samples.

[0052] Inverse transformation can be applied to transformation coefficients. Transformation coefficients can be determined based on at least one of signaling / parsing of residual information or prediction of transformation coefficients. Here, the prediction of transformation coefficients can be performed independently for each inverse transformation unit. As an example, within the current inverse transformation unit, the current transformation coefficient can be derived as the sum of the predicted transformation coefficient and the signaled transformation coefficient. The predicted transformation coefficient can be derived based on the weighted sum of Q transformation coefficients adjacent to the current transformation coefficient. Here, Q can be an integer greater than or equal to 1. The signaled transformation coefficient may refer to a transformation coefficient derived based on the signaled residual information.

[0053] The inverse transform kernel may include at least one of a DCT-based kernel, a DST-based kernel, or an identity kernel. Constraints of the inverse transform kernel and the kernel used may be signaled or derived during the decoding process.

[0054] If the transformation coefficients have specific values, an inverse transformation kernel can be derived. Alternatively, if the size of the inverse transformation unit is a specific value or a specific prediction mode, an inverse transformation kernel can be derived.

[0055] Constraints on the inverse transform unit can be signaled or induced. Examples of such constraints on the inverse transform unit may include bit depth.

[0056] The inverse transform may be omitted if there is no transformed sample value derived from the value of the residual transform sample or if there is no signaled transformed sample value. Here, "not existing" may include the case where the derived transformed sample value is 0.

[0057] In the prediction execution phase, the decoder can signal a prediction mode for each prediction unit. Based on the signaled prediction mode, the decoder can perform predictions for each prediction unit.

[0058] Predictions can be performed as intra-channel predictions or cross-channel predictions. In intra-channel predictions, the current channel and the reference channel may be the same. In cross-channel predictions, the current channel and the reference channel may be different. In cross-channel predictions, the reference channel may be a single channel or multiple channels.

[0059] The method of performing prediction within the channel may include at least one of weighted average prediction, extrapolated prediction, or prediction unit matching prediction.

[0060] The method of performing cross-channel prediction may include at least one of weighted average prediction, extrapolation prediction, prediction unit matching prediction, or cross-channel prediction.

[0061] Prediction can be performed in at least one of the time domain or the transformation domain.

[0062] FIG. 2 illustrates a channel group divided into frames according to the present disclosure.

[0063] As shown in the example of FIG. 2, a channel group can be divided into multiple frames according to specific time intervals. Frames can be encoded / decoded independently or by referencing previous or subsequent frames, and can be signaled via a bitstream. Encoding / decoding information, such as the availability of a prediction mode, the prediction method for each prediction mode, and the range of block sizes within a frame, is signaled on a frame-by-frame basis.

[0064] Figure 3 illustrates a frame recursively divided into blocks or sub-blocks.

[0065] As shown in the example of FIG. 3, a frame can be divided into multiple blocks according to a specific length. A block may be a set of samples corresponding to the same timestamp within a frame. Channels belonging to the same block may all have the same number of samples.

[0066] Block length conditions can be derived during the decoding process or signaled from the bitstream. For example, if the decoder can derive the minimum and maximum lengths of the block at the frame level containing the block, the signaling of the block length conditions is omitted. If the decoder cannot derive the minimum and maximum lengths of the block at the frame level containing the block, information regarding the minimum and maximum lengths of the block is signaled from the bitstream.

[0067] 1. Subblock and sub-subblock induction step

[0068] FIG. 4 is a schematic illustration of an embodiment according to the present disclosure, which divides a subblock to induce a lower subblock.

[0069] A subblock may be a set of samples grouped by channel within a block. A single block composed of multiple channels may have a specific number of subblocks. Prediction mode information, parameter information based on the prediction mode, residual transformation samples, or transformation samples, etc., may be signaled at the subblock or sub-subblock level.

[0070] A sub-block may be a set of samples grouped by processing stage within a sub-block. Sub-blocks are classified into prediction sub-blocks or transformation sub-blocks. A prediction sub-block includes a single or multiple prediction units and may be a part where the prediction process is performed. A transformation sub-block may include a single or multiple transformation units.

[0071] Information used in a lower subblock may be inherited from a parent subblock, derived during the decoding process, or signaled. Prediction mode information, prediction mode parameter information for the prediction mode, residual transformation samples, or transformation samples, etc., may be signaled at the subblock, prediction subblock, or transformation subblock level.

[0072] For example, a prediction mode or parameters for the prediction mode can be signaled for a prediction subblock. A residual transformation sample or a transformation sample can be signaled for a transformation subblock.

[0073] A sample can be a specific value that a specific channel has at a specific timestamp.

[0074] A prediction unit is the basic unit for performing intra-channel and cross-channel predictions. When creating a prediction unit, it may not be the minimum decoding unit. In this case, predictions can be performed without further dividing the prediction unit into multiple sub-prediction units.

[0075] To split and select frames, you can select the frames to be encoded. The frames to be encoded can be independent frames or dependent frames.

[0076] A frame can be divided into blocks of a specific length. This is for the purpose of dividing and selecting blocks. In this case, the length of each channel belonging to the same block may be the same. A block may contain a single or multiple samples within the frame. The minimum and maximum lengths of a block may be derived during the decoding process or signaled from the bitstream. The length of each block belonging to the same frame may be the same.

[0077] In the frame-by-frame restoration process, a frame can be classified as either an independent frame or a dependent frame based on whether there are other frames within the specific frame it references. A dependent frame can only be decoded after all independent frames within the same channel group or preceding dependent frames in the restoration sequence have been restored. Conversely, an independent frame can serve as an arbitrary starting point during frame decoding. A dependent frame can inherit encoding parameter information from independent frames or preceding dependent frames in the decoding sequence. If a dependent frame inherits encoding parameter information from a preceding independent or dependent frame, it may inherit all or part of the encoding parameters. If a dependent frame inherits only part of the encoding parameters, the remaining parameters can be signaled from the bitstream.

[0078] Similarly, during the block-unit restoration process, blocks can be classified as independent or dependent depending on whether they reference other blocks within the same frame. Dependent blocks can only be decoded after all independent blocks within the same frame or dependent blocks preceding them in the decoding order have been restored. Conversely, independent blocks can serve as arbitrary starting points during block decoding.

[0079] For example, if the timestamp of the first sample of a specific block in any decoder is the same as the timestamp of the first sample of a frame, that block can be derived as an independent block. In this case, the encoder can omit signaling for the information of that block.

[0080] As another example, if the timestamp of the first sample of a specific block within the same frame is later than the timestamp of the last sample of an independent block, that block can be guided to become a dependent block. In this case, the encoder can omit signaling for the information of that block.

[0081] A block can consist of multiple channels. A block can be divided into sub-blocks for each channel. Whether a prediction is within a channel or across channels can be signaled at the sub-block level.

[0082] Information used in the prediction subblock can be inherited from the subblock, derived at the level of the prediction subblock, or signaled at the level of the prediction subblock. Information used in the transformation subblock can be inherited from the subblock or the prediction subblock, derived at the level of the prediction subblock or the transformation subblock, or newly signaled at the level of the prediction subblock or the transformation subblock.

[0083] Referring to Fig. 4(a), the subblock can be recursively divided in the order of prediction subblock and transformation subblock.

[0084] Alternatively, as shown in Fig. 4(b), the predicted subblock can be recursively divided from the subblock.

[0085] Alternatively, as shown in Fig. 4(c), the transformed subblock can be recursively divided from the subblock.

[0086] Or, as shown in Fig. 4(d), the sub-block may not be further divided.

[0087] The recursive division of a sub-block into a lower sub-block in this manner is called recursive binary partitioning.

[0088] A method for dividing a sub-block into a sub-block can be determined based on block division information. The block division information may include at least one of (1) whether the division of a sub-block or a sub-block is allowed, (2) the depth of division, and (3) the division ratio.

[0089] The partition depth can have a natural number value and can indicate the maximum number of times the subblock is partitioned.

[0090] The division ratio can indicate the ratio in which sub-blocks will be divided, and can represent the ratio of each divided block determined by the division depth.

[0091] Whether to split and the split ratio can be determined based on each flag. Each flag can be signaled. For example, when splitting three prediction subblocks from a subblock, the flags can be signaled in the order of "1:1 split" - "Do not split" - "1:1 split". When splitting three transformation subblocks from the first prediction subblock, the flags can be signaled in the order of "1:1 split" - "Do not split" - "3:1 split". This can be repeated for subsequent prediction subblocks and transformation subblocks.

[0092] FIG. 4(a) shows an example of splitting prediction and transformation subblocks from a subblock. The example allows for the splitting of a prediction subblock from a subblock. The block splitting information for the prediction subblock has a splitting depth of up to 2 and allows for no splitting or only 1:1 splitting. Additionally, it allows for the splitting of a transformation subblock from a prediction subblock. The block splitting information for the transformation subblock has a splitting depth of up to 2 and allows for no splitting or only 1:3, 1:1, or 3:1 splitting.

[0093] At this time, the decoder splits three prediction subblocks from the subblock. From the split prediction subblocks, one prediction subblock is split into three transformation subblocks. Accordingly, a total of 9 transformation subblocks are generated. If the splitting and splitting ratio flags are signaled as "1:1 split"-"no split"-"1:1 split", then 1) the entire subblock is split 1:1, 2) the first prediction subblock after splitting is not split again, and 3) the second prediction subblock is split 1:1.

[0094] When splitting three transformation subblocks from the first prediction subblock after splitting, if the flags are signaled in the order of "1:1 split" - "do not split" - "3:1 split", then 1) split the entire prediction subblock 1:1, 2) do not split the first transformation subblock after splitting again, and 3) split the second transformation subblock 3:1.

[0095] In this way, the prediction subblock and the transformation subblock can be recursively partitioned from the subblock.

[0096] As another example, FIG. 4(b) shows an example of splitting a predicted subblock from a subblock. The example allows for splitting a predicted subblock from a subblock. The block splitting information of the predicted subblock allows for a splitting depth of up to 2, no splitting, or only 1:1 splitting, 1:3 splitting, or 3:1 splitting.

[0097] At this time, the decoder divides three predicted subblocks from the subblock. When block division information and division ratio flags from the subblock are signaled in the order of "1:1 division" - "no division" - "1:1 division", 1) the entire subblock is divided 1:1, 2) the first predicted subblock after division is not divided, and the second predicted subblock after division is divided 1:1.

[0098] As another example, FIG. 4(c) shows an example of splitting a transformation subblock from a subblock. The example allows splitting a transformation subblock from a subblock. The block splitting information of the transformation subblock allows for a splitting depth of up to 2, no splitting, or only 1:1 splitting, 1:3 splitting, or 3:1 splitting.

[0099] At this time, the decoder divides three transform subblocks from the subblock. When block division information and division ratio flags from the subblock are signaled in the order of "1:1 division" - "no division" - "3:1 division", 1) the entire subblock is divided 1:1, 2) the first transform subblock after division is not divided, and the second transform subblock after division is divided 3:1.

[0100] As another example, as shown in Fig. 4(d), if the encoder signals a non-segmentation flag, the decoder does not segment the subblock.

[0101] Prediction and reconstruction of samples within a subblock can be performed using a zigzag scan. Additionally, a wavefront parallel process can be applied to prediction and reconstruction for each subblock.

[0102] A decoder according to the present invention receives sample and block partitioning information from a bitstream, divides prediction and inverse transformation units, and performs prediction and inverse transformation for each divided unit to perform restoration. At this time, the prediction may be performed in at least one of a time domain or a transformation domain.

[0103] Specifically, the decoder can obtain a predicted residual sample through the prediction of the residual sample. A residual sample (hereinafter referred to as the signaled residual sample) can be obtained through inverse quantization and inverse transform of the transform coefficients derived based on the residual information. A final residual sample can be derived based on the predicted residual sample and the signaled residual sample.

[0104] Alternatively, inversely quantized transformation coefficients can be obtained through inverse quantization of transformation coefficients derived based on residual information. The decoder can obtain predicted transformation coefficients through the prediction of transformation coefficients. Transformation coefficients can be obtained by adding the predicted transformation coefficients to the inversely quantized transformation coefficients, and residual samples can be obtained by performing an inverse transformation thereon.

[0105] Alternatively, a conversion coefficient (hereinafter referred to as the signaled conversion coefficient) can be obtained based on residual information signaled through a bitstream. The decoder can obtain a predicted conversion coefficient by predicting the aforementioned conversion coefficient. A final conversion coefficient can be obtained by adding the predicted conversion coefficient to the signaled conversion coefficient. A residual sample can be obtained by performing inverse quantization and inverse transformation on the final conversion coefficient.

[0106] Block partitioning information of a lower sub-block can be signaled from or derived from a bitstream.

[0107] The method of dividing the subblocks, prediction units, or inverse transformation units constituting the signal is determined by the encoder and can be signaled through the bitstream.

[0108] The encoder can determine hierarchical block partitioning information for a prediction subblock or a transform inverse transform subblock from a subblock. In this case, the encoder can determine the block partitioning information in the order of the prediction subblock and the inverse transform subblock.

[0109] The encoder may decide not to split the subblock.

[0110] The decoder can signal block splitting information to be used for restoration and each block splitting flag from the bitstream.

[0111] A prediction unit may be a portion divided within a single decoding unit to have an arbitrary shape. The shape of the prediction unit may be square. The size of each prediction unit may be the same or different.

[0112] 2. Derivation of Prediction and Inverse Transformation Units

[0113] The decoder can receive a subblock bitstream as input and derive a prediction unit and an inverse transform unit. The subblock bitstream may be a bitstream obtained by dividing a biomedical waveform bitstream into subblock units using subblock partitioning information. The subblock partitioning information may be embedded in the bitstream.

[0114] The derivation process of the prediction unit and the inverse transformation unit may be as follows.

[0115] The decoder can derive a prediction unit and an inverse unit from a subblock. The prediction unit may be a subblock, a prediction subblock, or an inverse subblock. The prediction unit may be signaled. The inverse unit may be a subblock, a prediction subblock, or an inverse subblock. The inverse unit may be signaled.

[0116] The decoder can derive a prediction unit from a subblock. The prediction unit can be a subblock, a prediction subblock, or an inverse transformation subblock. The prediction unit can be signaled. The prediction unit can be derived from a subblock.

[0117] The decoder can derive an inverse unit from a subblock. The inverse unit may include at least one of a subblock, a prediction subblock, and an inverse subblock. The inverse unit may be signaled. The inverse unit may be derived from a subblock.

[0118] The decoder can derive the prediction unit into a subblock. The inverse transform unit can be derived from the subblock.

[0119] 3. Perform Inverse Transformation and Prediction

[0120] The decoder can perform inverse transformation and prediction based on derived prediction units and inverse transformation units. The inverse transformation and prediction can be performed sequentially or in parallel in the inverse transformation unit or the prediction unit.

[0121] The inverse transform execution unit can signal or derive the most appropriate inverse transform kernel for each inverse transform unit. The inverse transform execution unit can signal or derive constraints for the inverse transform unit. Examples of constraints for the inverse transform unit may include bit depth.

[0122] A residual transform sample or a residual sample restored from a transform sample can be calculated. The inverse transform may be omitted if there is no transform sample value derived from the value of the residual transform sample or no signaled transform sample value. Here, "not existing" may include the case where the derived transform sample value is 0.

[0123] The inverse transform kernel may include one of a DCT-based kernel, a DST-based kernel, or an identity kernel. When the inverse transform kernel is used, it may signal or derive constraints of the kernels used. When the inverse transform kernel is used,

[0124] 1) When the size of the inverse transform unit is a specific value or a specific prediction mode

[0125] 2) When the residual transformation samples or transformation samples are specific values

[0126] It could be.

[0127] In the prediction execution unit, the decoder can perform predictions. Predictions can be performed on a prediction unit basis. Predictions can be performed based on an derived prediction mode, or the predicted values ​​can be signaled. The prediction mode may include one of weighted average prediction, extrapolation prediction, block matching prediction, or cross-channel prediction.

[0128] In the case of intra-channel forecasting, the current channel and the reference channel may be the same. In the case of cross-channel forecasting, the current channel and the reference channel may be different. The reference channel may be a single or multiple channels.

[0129] The decoder can signal information regarding the reconstructed transform samples and weights per transform sample for reconstructing transform samples from the prediction unit bitstream. Based on the signaled information, the decoder can calculate the reconstructed transform samples to be referenced and the weights per reconstructed transform samples to be referenced.

[0130] The decoder can perform predictions based on calculated samples or sample-specific weights. As a result of performing the prediction, a predicted transformation sample can be derived. An inverse transformation according to an inverse transformation unit is performed on the predicted transformation sample to finally obtain a restored residual sample.

[0131] The prediction unit bitstream may be a bitstream obtained by dividing a bitstream containing information of a subblock according to a prediction unit. The information required to divide according to a prediction unit may include at least one of prediction unit information or division information of the prediction unit. The information of the subblock may include at least one of prediction unit information or division information of the prediction unit.

[0132] FIG. 5 illustrates a method of performing prediction according to a prediction unit when the prediction method for the current subblock in the decoder is an in-channel prediction. The current subblock may be at least one of a subblock or a prediction subblock.

[0133] Figure 5(a) is an example where the current subblock is composed of a single prediction unit, and Figure 5(b) is an example where the current subblock is divided into two prediction units.

[0134] The P prediction samples {pred[0], pred[1] to pred[P-1]} constituting the prediction unit of Fig. 5(a) can be generated based on a portion of the R pre-recovered samples {rec[-R], rec[-R+1] to rec[-1]} of the same channel.

[0135] The reference samples can be derived by selecting the top N samples with the largest timestamps among the recovered samples. This may involve selecting N samples from the right among the recovered samples. Alternatively, the value of N may be signaled from the bitstream. For example, if the 4 samples with the largest timestamps are selected, N can always be derived as 4, or N = 4 can be signaled.

[0136] If the number of pre-restored samples is less than N, pre-restored samples can be randomly generated through appropriate padding.

[0137] In FIG. 5(b), the prediction unit 1 may include T prediction samples. The T prediction samples pred[0], pred[1] to pred[T-1] constituting the prediction unit 1 may be generated based on a portion of the R previously restored samples rec[-R], rec[-R+1] to rec[-1] of the same channel. The reference samples may be derived by selecting the top N samples with the largest timestamps among the previously restored samples. Alternatively, the value of N may be signaled from the bitstream. For example, if the 4 samples with the largest timestamps are selected, N may always be derived as 4, or N = 4 may be signaled.

[0138] If the number of pre-restored samples is less than N, pre-restored samples can be randomly generated through appropriate padding.

[0139] In FIG. 5(b), the prediction unit 2 may include PT samples. The PT prediction samples pred[T], pred[T+1] to pred[P-1] constituting the prediction unit 2 may be generated based on a portion of the T transformation samples rec[0], rec[1] to rec[T-1] that have already been restored within the same prediction unit. The restored transformation samples rec[0], rec[1] to rec[T-1] may be generated by combining the predicted transformation samples pred[0], pred[1] to pred[T-1] with the signaled residual transformation samples, and then the residual samples may be generated by inversely transforming the restored transformation samples based on the inverse transformation unit.

[0140] Predicted samples can be derived from reference samples. The prediction method can be weighted average prediction, extrapolation prediction, or prediction unit matching prediction.

[0141] Perform weighted average prediction

[0142] When performing a weighted average forecast for each forecast unit, the forecast can be performed based on the following mathematical formulas 1 and 2.

[0143]

[0144]

[0145] For each mathematical expression, at least one of an appropriate number of multiplications, divisions, or roundings may be performed to maintain bit precision and bit scale.

[0146] In mathematical formula 1, The values ​​of can be signaled from the bitstream.

[0147] For example, when N is 4 By defining them to derive 1 / 4, 1 / 4, 1 / 4, and 1 / 4 respectively, the prediction sample can be generated as the arithmetic mean value of the reference samples.

[0148] As another example, While making 1 By signaling on a frame-by-frame basis, it is possible to generate a prediction sample as the weighted average value of the reference samples.

[0149] As another example, While preventing ga from becoming 1 By signaling on a frame-by-frame basis, it is possible to generate a prediction sample as a scaled weighted average value of the reference samples.

[0150] Perform extrapolation prediction

[0151] When performing extrapolation predictions for each prediction unit, the prediction can be performed based on the following mathematical formulas 3, 4, and 5.

[0152]

[0153]

[0154]

[0155] For each mathematical expression, at least one of an appropriate number of multiplications, divisions, or roundings may be performed to maintain bit precision and bit scale.

[0156] In mathematical formula 3, The values ​​of can be signaled from the bitstream.

[0157] For example, when N is 4 By defining to derive -15, -5, 5, and 15 respectively, It can be made to become "the result of high-pass filtering rec[i]".

[0158] As another example, α is 0, and By signaling on a frame-by-frame basis, It can be made to have the weighted average value with the offsets of the reference samples removed.

[0159] As another example, g is not 0, By signaling on a frame-by-frame basis, It can be made to have a scaled weighted average value with the offsets of the reference samples removed.

[0160] In mathematical formula 4, The value of can be signaled from a bitstream. For example, when N is 4, By defining the values ​​to derive 1 / 4, 1 / 4, 1 / 4, and 1 / 4 respectively, the offset can be made to be the arithmetic mean of the reference samples.

[0161] or, ga is 1, and By signaling on a frame-by-frame basis, the offset can be made to be the scaled weighted average value of the reference samples.

[0162] In mathematical equation 5, m and The values ​​of can be signaled.

[0163] For example, when N=4 and m=2, By deriving the values ​​of -10, 55, and 200, a quadratic function can be derived from four reference samples. Based on the derived quadratic function, prediction samples can be extrapolated.

[0164] Or, m and By signaling the values ​​of frame by frame, an m-th order function can be derived from N reference samples to extrapolate the prediction samples.

[0165] The decoder can perform prediction unit matching predictions for each prediction unit. The decoder receives a prediction unit bitstream, determines a pre-recovered sample to reference, performs prediction unit matching predictions, and performs inverse transformations for each prediction sample to derive a recovered prediction unit.

[0166] The prediction unit bitstream may be a bitstream obtained by dividing a bitstream containing information of a subblock according to a prediction unit. The information required to divide according to a prediction unit may include at least one of prediction unit information or division information of the prediction unit. The information of the subblock may include at least one of prediction unit information or division information of the prediction unit.

[0167] FIG. 6 illustrates the process of performing a prediction according to a prediction unit when the prediction method for the current sub-block in the decoder is a prediction unit matching prediction. The current sub-block may be at least one of a sub-block or a prediction sub-block.

[0168] After a reference sample is selected, a prediction unit matching prediction can be performed.

[0169] Figure 6(a) is an example where the current subblock is composed of a single prediction unit, and Figure 6(b) is an example where the current subblock is divided into two prediction units.

[0170] The P prediction samples constituting the prediction unit of Fig. 6(a) may be generated based on all or part of the P samples already restored in the current channel. Alternatively, they may be generated based on another P samples (rec) already restored in the current channel. Alternatively, they may be generated based on all or part of 2*P samples (rec) already restored in the current channel.

[0171] The bitstream can signal how many reference samples a prediction sample will be generated based on. The reference sample information signaled from the bitstream may include at least one of the number of referenced samples or the sample indices of the referenced samples.

[0172] The sample index may be an index representing the size of the timestamp of the sample being referenced.

[0173] For example, after imposing a constraint that the number of reference samples is always P or 2*P, the number of reference samples and the sample index of each reference sample can be signaled.

[0174] When the number of referenced samples is P, the sample index of the sample with the smallest timestamp among the referenced samples can be derived. The sample index of that sample can also be signaled. Furthermore, it can be derived that the sample indices of the remaining P-1 samples have an ascending order. Through this, it can be derived that the remaining P-1 samples are arranged adjacently starting from the sample with the smallest timestamp, and that the order of arrangement is the ascending order of the sample indices.

[0175] When the number of referenced samples is 2*P, the sample index of the sample with the smallest timestamp among the referenced samples can be derived. The sample index of that sample can also be signaled. Furthermore, it can be derived that the sample indices of the remaining P-1 samples have an ascending order. This process can be repeated for the remaining P samples. In this process, there may be samples that are referenced multiple times.

[0176] At this time, even if the number of reference samples is fixed to a specific number, a specific number of reference samples can be derived through the above method.

[0177] If the referenced sample index exceeds the sample index of the previously restored samples, the previously restored samples can be randomly generated through appropriate padding.

[0178] The prediction unit 1 of FIG. 6(b) may include T prediction samples. The prediction samples may be generated based on all or part of the T samples already restored in the current channel. Alternatively, they may be generated based on another P samples already restored in the current channel. Alternatively, they may be generated based on all or part of 2*T samples already restored in the current channel. In this case, the number of reference samples on which the prediction samples will be generated can be signaled.

[0179] For example, after imposing a constraint that the number of reference samples is always T or 2*T, the number of reference samples and the sample indices of the reference samples can be signaled.

[0180] If the number of referenced samples is T, the sample index of the sample with the smallest timestamp among the referenced samples can be derived. The sample index of that sample can also be signaled. It can be derived that the sample indices of the remaining T-1 samples appear adjacently in ascending order.

[0181] When the number of reference samples is 2*T, the sample index of the sample with the smallest timestamp among the reference samples can be derived. The sample index of that sample can also be signaled. For any T reference samples among 2*T reference samples, the sample indices of T-1 samples can be derived to appear adjacently in ascending order with respect to the sample with the smallest timestamp. Then, only the sample index of the sample with the smallest timestamp among the remaining T samples can be signaled, or after deriving it, the sample indices of the remaining T-1 samples can be derived to appear adjacently in ascending order.

[0182] At this time, even if the number of reference samples is fixed to a specific number, a specific number of reference samples can be derived through the above method.

[0183] If the referenced sample index exceeds the sample index of the previously restored samples, the previously restored samples can be randomly generated through appropriate padding.

[0184] Prediction unit 2 may include PT prediction samples. Prediction samples may be generated by referencing PT or 2*(PT) samples among the previously restored transformation samples. In this case, the previously restored transformation sample portion may include prediction unit 1. T residual samples may be generated by summing the T predicted transformation samples and the signaled residual transformation coefficients, and then performing an inverse transformation with the inverse transformation unit.

[0185] The sample index of the sample with the smallest timestamp among the reference samples can be derived. The sample index of that sample can also be signaled. When deriving the remaining reference samples, the same procedure as described above, which was performed for prediction unit 1, can be followed.

[0186] When referencing 2*(PT) samples, there may be samples that are referenced redundantly with prediction unit 1 when predicting prediction unit 2.

[0187] If the referenced sample index exceeds the sample index of the recovered samples, the recovered samples can be randomly generated through appropriate padding.

[0188] Perform prediction unit matching prediction

[0189] Prediction unit matching prediction can be performed based on the following mathematical formulas 6 and 7.

[0190]

[0191]

[0192] For each mathematical expression, at least one of an appropriate number of multiplications, divisions, or roundings may be performed to maintain bit precision and scale.

[0193] In mathematical equation 6, the value of N can be signaled from the bitstream. For example, N can be signaled after imposing a constraint that N is always 1 or 2.

[0194] Through mathematical formula 6, the current prediction unit can be predicted from the pre-recovered samples of the current channel. At this time, the i-th sample of the current channel can be predicted based on the j-th sample among the pre-recovered samples of the current channel.

[0195] The index j assigned to the pre-restored sample can be derived as the sum of base and offset through Equation 7. The values ​​of base and offset can be derived from the prediction unit or signaled. Alternatively, the method for determining the values ​​of base and offset can be signaled. For example, the value of base can be derived through a decoding process for each prediction unit, and the offset can be signaled for each prediction unit.

[0196] When deriving the base or offset of an index j of a previously restored sample, the base or offset of j can be derived based on the base or offset values ​​of other previously restored samples. One example of deriving a sample index j based on the base or offset values ​​of other previously restored samples may be linear regression. The number of other previously restored samples to be referenced when deriving a sample index based on the base or offset values ​​of the previously restored samples can be arbitrarily determined. For example, the base or offset of 16 samples prior to the current sample can be derived through regression using a prediction model such as a linear model.

[0197] The decoder can perform cross-channel prediction. The decoder receives a prediction unit bitstream and can derive at least one of the reference channel itself or pre-recovered samples within the reference channel. The decoder can perform cross-channel prediction based on the reference channel or pre-recovered samples. The decoder can derive a recovered prediction unit by performing an inverse transform on the prediction samples unit by unit.

[0198] The prediction unit bitstream may be a bitstream obtained by dividing a bitstream containing information of a subblock according to a prediction unit. The information required to divide according to a prediction unit may include at least one of prediction unit information or division information of the prediction unit. The information of the subblock may include at least one of prediction unit information or division information of the prediction unit.

[0199] Figure 7 illustrates an example of performing a prediction according to the prediction unit when the prediction method for the current sub-block in the decoder is cross-channel prediction.

[0200] Before performing a prediction, appropriate regression parameters can be derived through linear regression between the pre-restored samples and the prediction samples. The pre-restored samples that can be used to derive the regression parameters for the prediction sample rec may be at least one of rec1 or rec2.

[0201] FIG. 7(a) is an example divided into one prediction unit, and FIG. 7(b) is an example divided into two prediction units. The P prediction samples constituting the prediction unit of FIG. 7(a) may be generated based on all or part of R+P samples that have already been restored in a single channel other than the current channel. The prediction samples may also be generated based on all or part of R+P samples that have already been restored in a single channel. The prediction samples may also be generated based on all or part of 2*(R+P) samples that have already been restored in multiple channels.

[0202] From a bitstream, 1) the number of referenced channels and 2) the index of a specific channel among the referenced channels can be signaled. For example, the number of referenced channels and the index of the referenced channel can be signaled respectively after imposing a constraint that the number of referenced channels is always 1 or 2. Alternatively, it can be signaled that only one channel is referenced and only the index of the referenced channel can be signaled.

[0203] Reference samples can be derived by selecting the top N samples with the largest timestamps among the recovered samples. In this case, the value of N can be signaled. For example, N can always be derived as 16, or N=16 can be signaled. If there are fewer than N recovered samples, the recovered samples can be randomly generated through appropriate padding.

[0204] The prediction unit 1 of FIG. 7(b) may include T prediction samples. The prediction samples may be generated by referencing all or part of R+T samples that have already been restored in a single channel other than the current channel. Alternatively, they may be generated based on all or part of R+P samples that have already been restored in another single channel. Alternatively, they may be generated based on all or part of 2*(R+T) samples that have already been restored in multiple channels.

[0205] Before performing prediction, appropriate regression parameters can be derived by modeling the relationships between rec and rec1 and rec and rec and rec2 through linear regression of rec on the pre-restored samples rec1 and rec2, and these regression parameters can be used for prediction.

[0206] The decoder can signal the number of reference channels and the index of the referenced channel from the bitstream.

[0207] For example, you can signal the number of referenced channels and the index of the referenced channels, respectively, after imposing a constraint that the number of referenced channels is always 1 or 2. Or, you can signal only the index of the referenced channels after imposing a constraint that the number of referenced channels is always 1.

[0208] Reference samples can be derived by selecting the top N samples with the largest timestamps among the pre-recovered samples. In this case, the value of N can be signaled.

[0209] For example, N can always be derived to be 16, or signal that N=16.

[0210] If the number of pre-recovered samples is less than N, pre-recovered samples can be randomly generated through appropriate padding.

[0211] PT prediction samples (pred) constituting prediction unit 2 may be generated based on all or part of T pre-recovered samples. The pre-recovered samples may be samples that were in the same prediction unit as the prediction samples.

[0212] Residual samples can be generated by combining T predicted transformation samples and parsed residual transformation samples, and then performing an inverse transformation on the inverse transformation unit.

[0213] Before performing prediction, appropriate regression parameters can be derived by modeling the relationships between rec and rec1 and rec and rec and rec2 through linear regression of rec on the pre-restored samples rec1 and rec2, and these regression parameters can be used for prediction.

[0214] Performing predictions across channels

[0215] Cross-channel prediction can be performed based on the following mathematical formulas 8 and 9.

[0216]

[0217]

[0218] For each mathematical expression, at least one of an appropriate number of multiplications, divisions, or roundings may be performed to maintain bit precision and scale.

[0219] or The value of can be derived based on Equation 8. Or, or The value of can be signaled. Here, M can be the number of reference channels. The size of N can be signaled. For example, N can be 16.

[0220] If M is set as the number of reference channels, the determined or By substituting the value of into Equation 9, prediction samples can be calculated.

[0221] For example, when M is 2 (when there are 2 reference channels), , , Based on mathematical equation 8, the relationship between samples belonging to the first restored interval of each channel can be modeled for multiple channels and the current channel. In this case, the multiple channels may be the first reference channel or the second reference channel. For any natural number M, the multiple channels may include the first reference channel through the Mth reference channel.

[0222] In the same way, based on the samples belonging to the restored second interval of the first reference channel and the samples belonging to the restored second interval of the second reference channel, samples belonging to the restored second interval of the current channel can be derived.

[0223] As another example, when M is 1, to 1, When signaled as 0, samples belonging to the second interval of the current channel can be predicted to be identical to samples belonging to the second interval of the reference channel.

[0224] Predicted samples and residual samples are generated as a result of performing inverse transformation and prediction.

[0225] The decoder can generate a finally restored signal by performing entropy decoding, inverse quantization, and restoration on the predicted sample and residual sample.

[0226] In the entropy decoding process, the decoder can receive quantized transform samples or residual transform samples. The compressed statistical information within the quantized transform samples or residual transform samples can be restored using variable-length coding or arithmetic coding methods.

[0227] In the inverse quantization process, the decoder can inversely quantize the transform sample or the residual transform sample. Inverse quantization can dequantize the sign part and the level part of the transform sample or the residual transform sample, respectively, through a finite state machine based on parity bits.

[0228] Finally, the decoder can restore the biomedical waveform signal by performing restoration on the transformed sample or the residual transformed sample.

[0229] FIG. 8 is a block diagram illustrating a signal encoding method according to the present invention.

[0230] A signal encoding method performed in a signal encoding device according to the present invention may include a sub-block and sub-sub-block derivation step (S800), a prediction unit and conversion unit derivation step (S810), and a conversion and prediction execution step (S820).

[0231] The method for deriving sub-blocks and sub-sub-blocks, deriving prediction units and transformation units, and performing transformation and prediction is as described with reference to Fig. 1.

[0232] The various embodiments of the present disclosure are not intended to list all possible combinations but to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.

[0233] In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, it may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0234] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating system, application, firmware, program, etc.) that enable an operation according to a method of various embodiments to be executed on a device or computer, and a non-transitory computer-readable medium on which such software or instructions, etc. are stored and executable on a device or computer.

Claims

1. A step of inducing a subblock or a lower subblock of the current block by a predetermined division method; A step of deriving a prediction unit and an inverse transformation unit based on at least one of the above subblock or the above sub-subblock; A step of performing a prediction based on the above prediction unit; and The method includes a step of performing an inverse transformation based on the above inverse transformation unit, An image decoding method in which the above-mentioned sub-blocks are divided into a prediction sub-block and a transformation sub-block.

2. In Paragraph 1, The above prediction subblock is divided from the above subblock, and An image decoding method in which the above-mentioned conversion subblock is divided from at least one of the above-mentioned subblock or the above-mentioned prediction subblock.

3. In Paragraph 1, The above-described segmentation method is an image decoding method that is performed recursively on a sub-block of the current block or on a sub-block of the current block.

4. In Paragraph 3, The above division method is determined based on block division information, and An image decoding method wherein the block division information includes at least one of whether the block is allowed to be divided, the division depth, or the division ratio.

5. In Paragraph 1, An image decoding method in which the prediction unit or the inverse transformation unit is derived based on at least one of the prediction subblock or the inverse transformation subblock.

6. In Paragraph 5, The above prediction is performed based on at least one of a reference sample or a pre-recovery sample, and The above prediction method is an image decoding method comprising at least one of weighted average prediction, extrapolation prediction, or prediction unit matching prediction.

7. In Paragraph 6, The above reference sample is derived based on at least one of a specific sample or sample index among the above-mentioned reconstructed samples, and An image decoding method in which the above sample index is an index representing the size of the timestamp of a reference sample.

8. In Paragraph 7, A video decoding method that generates the above-mentioned 9. In Paragraph 6, The above prediction method is cross-channel prediction, and An image decoding method in which the above-mentioned cross-channel prediction is performed based on at least one of a single or multiple reference samples, pre-restored samples, or regression parameters.

10. In Paragraph 9, An image decoding method that generates the previously restored samples through padding when the number of previously restored samples is smaller than the sample index.

11. A step of inducing a subblock or a lower subblock of the current block by a predetermined division method; A step of deriving a prediction unit or a transformation unit based on at least one of the above subblock or the above sub-subblock; and The method includes the step of performing a prediction based on the above-mentioned prediction unit or performing a transformation based on the above-mentioned transformation unit, wherein An image encoding method in which the above-mentioned sub-blocks are divided into a prediction sub-block and a transformation sub-block.

12. A computer-readable storage medium for storing a bitstream generated by the image encoding method according to paragraph 11.

13. A method for transmitting a bitstream generated by the video encoding method according to paragraph 11.

Images