Coding / Decoding Apparatus, Coding / Decoding Method, and Program
The encoding/decoding device improves data restoration accuracy by adaptively determining quantization precision for each encoded feature, optimizing the quantization process to enhance data restoration precision.
Patent Information
- Application Number
- JP2023520630
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-05-11
- Publication Date
- 2024-11-20
- Estimated Expiration
- 2041-05-11
AI Technical Summary
Existing encoding/decoding devices using machine-learned neural networks face limitations in improving the precision of restoring input data from encoded data due to fixed quantization precision for each encoded feature.
An encoding/decoding device that adaptively determines quantization precision for each encoded feature based on the encoded code amount, utilizing a neural network to optimize the quantization process and improve the accuracy of data restoration.
Enhances the accuracy of restoring input data from encoded data by dynamically adjusting quantization precision, leading to improved performance in data restoration.
Smart Images

Figure 0007587187000001 
Figure 0007587187000002 
Figure 0007587187000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a coding / decoding device, a coding / decoding method, and a program. [Background technology]
[0002] There is known a device (encoding / decoding device) that compresses input data using a machine-learned neural network (see Non-Patent Document 1). Fig. 8 is a diagram showing a configuration example of a coding / decoding device 10. The coding / decoding device 10 includes an encoding unit 11, a quantization unit 12, a binarization unit 13, and a decoding unit 14 as functional units of an autoencoder that uses a neural network.
[0003] The encoding unit 11 converts input data into a vector (hereinafter referred to as an "encoding feature vector") having N (N is an integer equal to or greater than 1) encoding features as elements. The quantization unit 12 executes a quantization process on the encoding feature vector based on a vector (hereinafter referred to as a "quantization accuracy vector") having quantization accuracy as elements. Here, a fixed quantization accuracy is determined for each encoding feature (element of the encoding feature vector).
[0004] The binarization unit 13 performs a binarization process on the quantized encoded feature vector (hereinafter referred to as the "quantized encoded feature vector") to generate a binarized quantized encoded feature vector (hereinafter referred to as the "encoded data"). The decoding unit 14 performs a decoding process on the encoded data to generate decoded data. [Prior art documents] [Non-patent literature]
[0005] [Non-Patent Document 1] Eirikur Agustsson, et al. ,"Generative Adversarial Networks for Extreme Learned Image Compression," ICCV2019. Summary of the Invention [Problem to be solved by the invention]
[0006] The encoding / decoding device generates encoded data having a predetermined code amount. The predetermined code amount is the code amount obtained by multiplying the number of encoding features "N" by the quantization accuracy. However, such an encoding / decoding device has a problem in that it is not possible to improve the accuracy of restoring input data from the encoded data.
[0007] In view of the above circumstances, an object of the present invention is to provide a coding / decoding device, a coding / decoding method, and a program that are capable of improving the accuracy of restoring input data from coded data. [Means for solving the problem]
[0008] One aspect of the present invention is a coding / decoding device that includes an encoding unit that converts input data into an encoding feature vector, a quantization accuracy derivation unit that derives a quantization accuracy for each encoding feature that is an element of the encoding feature vector in accordance with an encoding code amount, a quantization unit that performs a quantization process on the encoding feature vector based on the quantization accuracy to generate a quantized encoding feature vector of a size of a quantized code amount that targets the encoding code amount, a binarization unit that performs a binarization process on the quantized encoding feature vector to generate encoded data, and a decoding unit that performs a decoding process on specified data in accordance with the encoding data.
[0009] One aspect of the present invention is an encoding / decoding method executed by an encoding / decoding device, the encoding / decoding method including: an encoding step of converting input data into an encoding feature vector; a quantization accuracy derivation step of deriving a quantization accuracy for each encoding feature, which is an element of the encoding feature vector, in accordance with the encoding code amount; a quantization step of generating a quantized encoding feature vector of a size of quantized code amount targeting the encoding code amount by performing a quantization process on the encoding feature vector based on the quantization accuracy; a binarization step of generating encoded data by performing a binarization process on the quantized encoding feature vector; and a decoding step of performing a decoding process on specified data in accordance with the encoding data.
[0010] One aspect of the present invention is a program for causing a computer to function as the encoding / decoding device described above. Effect of the Invention
[0011] According to the present invention, it is possible to improve the accuracy of restoring input data from encoded data. [Brief description of the drawings]
[0012] [Figure 1] FIG. 2 is a diagram illustrating an example of the configuration of a coding / decoding device in the first embodiment. [Diagram 2] 4 is a flowchart showing an example of the operation of the encoding / decoding device in the first embodiment. [Diagram 3] FIG. 4 is a diagram illustrating an example of the relationship between the compression ratio and the peak signal-to-noise ratio in the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of rate control in the first embodiment. [Diagram 5] FIG. 11 is a diagram illustrating an example of the configuration of a coding / decoding device in a second embodiment. [Figure 6] FIG. 11 is a diagram illustrating an example of scalable decoding in the second embodiment. [Figure 7] FIG. 2 is a diagram illustrating an example of a hardware configuration of an encoding / decoding device in each embodiment. [Figure 8] FIG. 1 illustrates an example of the configuration of a coding / decoding device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings. (First embodiment) 1 is a diagram showing an example of the configuration of a coding / decoding device 1a. The coding / decoding device 1a is a system that performs coding processing (data compression processing) on input data and performs decoding processing on the coded data.
[0014] The encoding / decoding device 1a includes an autoencoder 2 and a learning device 3. The autoencoder 2 includes an encoding unit 20, a quantization unit 21, a binarization unit 22, an extraction and shaping unit 23a, a debinarization unit 24, and a decoding unit 25. The learning device 3 includes a reconstruction error derivation unit 30, a quantization accuracy derivation unit 31, a code amount derivation unit 32, a code amount error derivation unit 33, and an optimization unit 34.
[0015] First, an overview of autoencoder 2 will be provided. The encoding unit 20 has a neural network for performing encoding processing (hereinafter referred to as "encoding neural network"). The decoding unit 25 has a neural network for performing decoding processing (hereinafter referred to as "decoding neural network"). The quantization accuracy derivation unit 31 has a neural network for deriving a quantization accuracy vector (hereinafter referred to as "quantization neural network"). Each of the encoding neural network, the decoding neural network, and the quantization neural network is a neural network to be learned (optimized).
[0016] The autoencoder 2 converts the input data into an encoded feature vector by executing an encoding process (data compression process) on the input data using an encoding neural network.
[0017] In the following, each element (encoding feature) of the encoding feature vector is associated with an element (quantization accuracy) of the quantization accuracy vector. The quantization accuracy is adaptively updated by the learning device 3 according to the code amount (hereinafter referred to as the "encoding code amount") (compression rate) of one or more encoding features.
[0018] The autoencoder 2 performs a quantization process on the encoding feature vector based on the quantization precision vector. The autoencoder 2 converts the encoding feature vector into a quantized encoding feature vector through the quantization process. The autoencoder 2 generates encoded data by performing a binarization process on the quantized encoding feature vector. In the binarization process, the autoencoder 2 deletes binarized data outside the range of the quantization precision from the encoded data.
[0019] Hereinafter, the code amount of the binary data extracted from the encoded data is referred to as the "decoded code amount". In the first embodiment, the encoded code amount and the decoded code amount are equal. The autoencoder 2 extracts binary data having a size of the decoded code amount from the encoded data. The autoencoder 2 performs a shaping process on the binary data having a size of the decoded code amount. Here, the autoencoder 2 generates decoding data in a shaped format (shaped decoding data) by shaping the format of the extracted binary data into the format of a quantized encoding feature vector. Here, the autoencoder 2 complements the binary data deleted from the encoded data with a predetermined value (e.g., 0) in the decoding data in the shaped format.
[0020] The autoencoder 2 generates debinarized decoding data by performing a debinarization process on the decoded data in the shaped format. The autoencoder 2 generates decoded data by performing a decoding process using a decoding neural network on the debinarized decoding data.
[0021] Next, the autoencoder 2 will be described in detail. The encoding unit 20 acquires the encoded code amount and input data from, for example, an information processing device (not shown). The encoding unit 20 converts the input data into an encoded feature vector based on the encoded code amount. The quantization unit 21 derives the result of an integer rounding process using a sigmoid function and a quantization precision vector as a quantized encoding feature vector of a size of a quantized code amount targeting the encoded code amount for each element of the encoded feature vector. The quantized code amount is the sum of the elements in the quantization precision vector. The binarization unit 22 generates encoded data by executing a binarization process on the quantized encoding feature vector based on the quantization precision vector. Here, the binarization unit 22 generates encoded data by deleting binarized data outside the range of the quantization precision from the quantization precision vector.
[0022] The extracting and shaping unit 23a extracts binary data of the size of the decoded code amount from the acquired coded data. The extracting and shaping unit 23a shapes the format of the binary data extracted from the acquired coded data into the format of the quantized coding feature vector based on the quantization precision vector. Here, the extracting and shaping unit 23a complements the binary data outside the range of the quantization precision with a predetermined value based on the quantization precision in the data for decoding. In this way, the extracting and shaping unit 23a generates data for decoding in a shaped format.
[0023] The debinarization unit 24 performs a debinarization process on the decoded data in the shaped format to generate debinarized decoding data. The decoding unit 25 performs a decoding process on the debinarized decoding data based on the decoded code amount. In this way, the decoding unit 25 converts the debinarized decoding data into decoded data.
[0024] Next, an overview of the learning device 3 will be described. The learning device 3 is a device that executes a learning process (machine learning). The learning device 3 derives a difference (distance between vectors) between input data and decoded data. The difference between the input data and decoded data is expressed using, for example, a mean square error. The learning device 3 derives a difference between a quantized code amount, which is the sum of elements in a quantization precision vector, and an encoded code amount (compression rate). The learning device 3 generates an objective function based on each difference.
[0025] The learning device 3 updates at least one of the parameters of the encoding neural network of the encoding unit 20, the parameters of the decoding neural network of the decoding unit 25, and the parameters of the quantization neural network of the quantization accuracy derivation unit 31 so as to reduce the difference between the input data and the decoded data (so as to reduce the value of the objective function). In this way, the learning device 3 adaptively updates the elements of the quantization accuracy vector (quantization accuracy) according to the amount of encoded code.
[0026] The learning device 3 (optimization device) outputs the updated parameters of the encoding neural network to the encoding unit 20. The learning device 3 outputs the updated parameters of the decoding neural network to the decoding unit 25. The learning device 3 outputs the updated parameters of the quantization neural network to the quantization accuracy derivation unit 31.
[0027] Next, the learning device 3 will be described in detail. The reconstruction error derivation unit 30 derives a reconstruction error, which is an error of decoded data with respect to input data. The quantization accuracy derivation unit 31 derives a quantization accuracy vector according to the amount of encoded code. Here, the quantization accuracy derivation unit 31 derives the quantization accuracy vector using a quantization neural network for the amount of encoded code. The parameters of the quantization neural network are updated by the optimization unit 34.
[0028] The code amount derivation unit 32 derives the quantized code amount [bit] which is the sum of the "N" elements in the quantization precision vector. The code amount error derivation unit 33 derives the code amount error which is the error of the quantized code amount with respect to the encoded code amount (the difference between the encoded code amount and the quantized code amount).
[0029] The optimization unit 34 derives an objective function based on the reconstruction error and the code amount error. The optimization unit 34 executes an optimization process on the objective function. The optimization unit 34 updates at least one of the parameters of the encoding neural network of the encoding unit 20, the parameters of the decoding neural network of the decoding unit 25, and the parameters of the quantization neural network of the quantization accuracy derivation unit 31 by, for example, executing an error backpropagation method on the minimized objective function.
[0030] Next, an example of the operation of the encoding / decoding device 1a will be described. FIG. 2 is a flowchart showing an example of the operation of the encoding / decoding device 1a. enc The encoding unit 20 obtains the input data "x" from, for example, an information processing device (not shown). The encoding unit 20 converts the input data "x" into an encoded feature vector "z=[z 1 ,…,z N ]" to convert the encoded feature "z n The value of " represents the feature amount to be encoded (step S101).
[0031] The quantization accuracy derivation unit 31 calculates the coding code amount "R enc " is obtained from, for example, an information processing device (not shown). The quantization precision derivation unit 31 obtains the coding code amount "R enc " is used to obtain the quantization precision vector "B = [B 1 ,…,B N Here, the element of the quantization precision vector "B n The value of " is, for example, an integer between 0 and 64 (step S102).
[0032] Thus, the element "B n" may be 0. The quantization precision derivation unit 31 changes the quantization precision according to the amount of encoded code, thereby making it possible to control the number "N" of quantized encoding features included in the encoded data.
[0033] The quantization unit 21 obtains the encoding feature vector "z" from the encoding unit 20. The quantization unit 21 obtains the quantization precision vector "B" from the quantization precision derivation unit 31. The quantization unit 21 obtains the quantization encoding feature vector "z" of the size of the quantization code amount that is the target encoding code amount. q =[z 1 q ,…,z N q ]" and the sigmoid function "sigmoid(z n ) and the quantization precision vector B are used to calculate the integer rounding process Q. n q =Q(sigmoid(z n )(2 Bn -1))" to the element "z n " is derived for each (step S103).
[0034] The binarization unit 22 converts the quantized encoded feature vector “z q " from the quantization unit 21. The binarization unit 22 obtains the quantization precision vector from the quantization precision derivation unit 31. The binarization unit 22 performs binarization processing on the quantization encoding feature vector based on the quantization precision vector, thereby obtaining the encoded data "z enc Here, the binarization unit 22 converts the binarized data outside the range of the quantization accuracy into the encoded data “z enc " (step S104).
[0035] The extraction and shaping unit 23a extracts the decoded code amount "R dec " is obtained from, for example, an information processing device (not shown). The extraction and shaping unit 23a obtains the encoded data "z enc " from the binarization unit 22. The extraction and shaping unit 23a obtains the decoded code amount "R dec The binary data of size "z enc" (step S105).
[0036] The extraction and shaping unit 23a acquires the quantization precision vector "B" from the quantization precision derivation unit 31. The extraction and shaping unit 23a extracts the acquired encoded data "z enc The format of the binary data extracted from z q Here, the extraction and shaping unit 23a complements the binarized data outside the range of quantization accuracy with a predetermined value (for example, 0) in the data for decoding. As a result, the extraction and shaping unit 23a converts the data for decoding in the shaped format into "z dec " is generated (step S106).
[0037] In the following, symbols placed above letters in mathematical formulas are written immediately before the letters. For example, the letter "z q The symbol "^" above "^z q " q " is written immediately before.
[0038] The debinarization unit 24 converts the decryption data “z dec By performing the debinarization process on ", the debinarization decoding data "^z q " (step S107). The decoding unit 25 generates the debinarization decoding data "^z q ", the decoded code amount "R dec ” is used to perform the decoding process. In this way, the decoding unit 25 obtains the debinarization decoding data “^z q " is converted into decoded data "^x" (step S108).
[0039] The reconstruction error derivation unit 30 acquires input data from, for example, an information processing device (not shown). The reconstruction error derivation unit 30 acquires decoded data (reconstructed data) from the decoding unit 25. The reconstruction error derivation unit 30 calculates a reconstruction error "L rec=d(x, ^x)". The function "d" is an arbitrary function for deriving the distance between vectors, and is, for example, the sum of mean squared errors or binary cross entropy (step S109).
[0040] The code amount derivation unit 32 acquires the quantization precision vector “B” from the quantization precision derivation unit 31. The code amount derivation unit 32 calculates the “N” elements “B n The quantized code amount "R = ΣB n " [bit] is derived (step S110).
[0041] The code amount error derivation unit 33 calculates the coding code amount "R enc The code amount error derivation unit 33 obtains the quantized code amount "R=ΣB n " from the quantization precision derivation unit 31. The code amount error derivation unit 33 obtains the coding code amount "R enc Quantized code amount for "R=ΣB n The error in the amount of code "L rate =d(R enc , R) is derived (step S111).
[0042] The optimization unit 34 calculates the reconstruction error "L rec " and the code amount error "L rate Based on this, the objective function "L = L rec +λL rate " is derived. The weight "λ" is an arbitrary value (step S112).
[0043] The optimization unit 34 executes an optimization process for the objective function "L." That is, the optimization unit 34 solves a minimization problem of the objective function "L" by, for example, executing a gradient method (step S113).
[0044] The optimization unit 34 updates at least one of the parameters of the encoding neural network of the encoding unit 20, the parameters of the decoding neural network of the decoding unit 25, and the parameters of the quantization neural network of the quantization accuracy derivation unit 31, for example by performing backpropagation on the minimized objective function “L”.
[0045] The optimization unit 34 outputs the updated parameters of the encoding neural network to the encoding unit 20. The optimization unit 34 outputs the updated parameters of the quantization neural network to the quantization precision derivation unit 31. The optimization unit 34 outputs the updated parameters of the decoding neural network to the decoding unit 25 (step S114).
[0046] The optimization unit 34 determines whether or not to end the process shown in Fig. 2 based on a predetermined condition. For example, the optimization unit 34 ends the process when a predetermined condition is satisfied that the process shown in Fig. 2 is executed a predetermined number of times or more. For example, the optimization unit 34 ends the process when a predetermined condition is satisfied that the value of the objective function "L" is equal to or less than a predetermined value (step S115).
[0047] If it is determined that the process should be continued (step S115: NO), the optimization unit 34 returns the process to step S101. If it is determined that the process should be ended (step S115: YES), the optimization unit 34 ends the process shown in FIG.
[0048] As described above, the encoding unit 20 converts input data into an encoded feature vector. The quantization accuracy derivation unit 31 derives the quantization accuracy for each encoding feature, which is an element of the encoded feature vector, according to the encoded code amount. The quantization unit 21 performs a quantization process on the encoded feature vector based on the quantization accuracy, thereby generating a quantized encoded feature vector having a size of a quantized code amount that is a target encoded code amount. The binarization unit 22 performs a binarization process on the quantized encoded feature vector, thereby generating encoded data. The decoding unit 25 performs a decoding process on predetermined data according to the encoded data.
[0049] The extracting and shaping unit 23a extracts binary data having a size of the decoded code amount from the coded data. The extracting and shaping unit 23a shapes the format of the extracted binary data based on the quantization accuracy to generate shaped decoding data. The inverse binarization unit 24 generates inverse binarization decoding data by executing an inverse binarization process on the shaped decoding data. The decoding unit 25 converts the inverse binarization decoding data into decoded data by executing a decoding process on the inverse binarization decoding data (predetermined data) based on the decoded code amount.
[0050] The optimization unit 34 updates at least one of the parameters used in the encoding process that converts the input data into an encoded feature vector, the parameters used in the decoding process, and the parameters used to derive the quantization accuracy, based on an objective function.
[0051] In this way, the number of coding features "N" and the quantization precision "B n " and " are not fixed, and the quantization accuracy "B n " is derived. The quantization precision "B n Since the number of coding features "N" is determined according to ", the input data is coded with the optimal expression (combination of the number of coding features and quantization accuracy) according to the coding code amount. This makes it possible to improve the accuracy of restoring the input data from the coded data.
[0052] Fig. 3 is a diagram showing an example of the relationship between the compression rate (code amount) and the peak signal-to-noise ratio. Fig. 3 shows an example of the relationship between the compression rate and the peak signal-to-noise ratio for seismic wave data as an example. The horizontal axis shows the compression rate. The vertical axis shows the peak signal-to-noise ratio (PSNR) [dB].
[0053] In Fig. 3, "1 bit", "2 bits", "3 bits", "4 bits", and "8 bits" indicate fixed quantization precision. The graphs for "1 bit", "2 bits", "3 bits", "4 bits", and "8 bits" are graphs for conventional autoencoders. In each graph for conventional autoencoders, the quantization precision associated with each encoding feature is uniformly X [bit] for all encoding features. In each graph for conventional autoencoders, points are plotted for each number of encoding features.
[0054] In contrast, "AdaptiveBits" shown in Fig. 3 indicates adaptively changed quantization accuracy (quantization accuracy according to the amount of encoded code). The graph of "AdaptiveBits" is a graph related to the encoding / decoding device 1a. In the graph related to the encoding / decoding device 1a, a point is plotted for each amount of encoded code. In this way, the encoding / decoding device 1a can improve the accuracy (peak signal-to-noise ratio) of restoring input data from encoded data.
[0055] Next, the binarization unit 22 and the extraction and shaping unit 23a will be described in detail. Fig. 4 is a diagram showing an example of rate control, in which the quantized coding feature vector 210 includes, as an example, quantized coding features from element 211-1 to element 211-5.
[0056] The binarization unit 22 acquires the quantized encoded feature vector 210 from the quantization unit 21. The binarization unit 22 executes a binarization process on the quantized encoded feature vector 210 to generate encoded data 220 including binarized data.
[0057] The binarization unit 22 obtains the quantization precision vector 310 from the quantization precision derivation unit 31. In Fig. 4, the quantization precision vector 310 is, for example, "[2, 1, 4, 3, 0]." The binarization unit 22 obtains the encoded data 220 including the binarized data of each element 211 from the binarization unit 22.
[0058] The quantization precision associated with the binary data "...0010" of the element 211-1 is "2" in the quantization precision vector 310. The quantization precision associated with the binary data "...0000" of the element 211-2 is "1" in the quantization precision vector 310. The quantization precision associated with the binary data "...0101" of the element 211-3 is "4" in the quantization precision vector 310. The quantization precision associated with the binary data "...0111" of the element 211-4 is "3" in the quantization precision vector 310. The quantization precision associated with the binary data "...0000" of the element 211-5 is "0" in the quantization precision vector 310.
[0059] The binarization unit 22 deletes binarized data outside the range of quantization accuracy (outside the rectangular frame indicated by the dashed line in FIG. 4) from the coded data 220. In this way, the binarization unit 22 generates coded data 220 having a size of the quantization code amount specified using the quantization accuracy vector 310.
[0060] Here, the binarization unit 22 scans the binarized data of all elements 211. The binarization unit 22 scans the binarized data of all elements 211 in the order from the most significant bit to the least significant bit of the binarized data. The binarization unit 22 also scans the binarized data of all elements 211 in the order from element 211-1 to element 211-5, for example. Each arrow of the dashed dotted line shown in the encoded data 220 in FIG. 4 indicates such a scanning order.
[0061] By scanning from element 211-1 to element 211-5 in order, the binarization unit 22 obtains the most significant bit "0" within each quantization accuracy range from the binarized data. The binarization unit 22 obtains the most significant bits "1" and "1" on the lower side within each quantization accuracy range from the binarized data. The binarization unit 22 obtains the most significant bits "1", "0", and "1" on the further lower side within each quantization accuracy range from the binarized data. The binarization unit 22 obtains the most significant bits "0", "0", "1", and "1" within each quantization accuracy range from the binarized data.
[0062] In FIG. 4, the quantization precision associated with the binary data "...0000" of element 211-5 is "0". Therefore, the binary data of element 211-5 is outside the range of the quantization precision. For this reason, in the scan, the binarization unit 22 does not acquire the binary data "...0000" of element 211-5. In this way, the binarization unit 22 deletes the binary data "...0000" of element 211-5, whose quantization precision is "0", from the encoded data 220.
[0063] The binarization unit 22 combines the acquired binary data ("0", "11", "101", "0011") in the order in which the binary data was acquired, to generate rate-controlled coded data 220. In FIG. 4, the rate-controlled coded data 220 becomes "0111010011".
[0064] Among the binarized data of the coding feature, the binarized data outside the range of the quantization accuracy is deleted from the coded data as the rate control. In Fig. 4, only the binarized data within each rectangular frame represented by the dashed line is transmitted to the extraction and shaping unit 23a as the coded data 220.
[0065] The extracting and shaping unit 23a acquires the encoded data 220 from the binarization unit 22. The extracting and shaping unit 23a acquires the quantization precision vector 310 from the quantization precision derivation unit 31. The extracting and shaping unit 23a extracts, from the encoded data 220, binarized data of the size of the decoded code amount.
[0066] The extracting and shaping unit 23a performs a shaping process on the binarized data extracted from the rate-controlled encoded data 220. Here, the extracting and shaping unit 23a shapes the format of the extracted binarized data into the format of a quantized encoding feature vector, thereby generating data for decoding in a shaped format (shaped data for decoding).
[0067] The extracting and shaping unit 23a identifies the position of the binarized data deleted from the rate-controlled encoded data 220 by using the quantization precision vector 310. The extracting and shaping unit 23a complements the binarized data deleted from the rate-controlled encoded data 220 with a predetermined value (e.g., 0) in the shaped data for decoding.
[0068] As described above, the binarization unit 22 deletes binary data outside the range of the quantization accuracy from the encoded data 220 based on the quantization accuracy vector 310. The extraction and shaping unit 23a identifies the bit positions of the binary data deleted from the encoded data 220 based on the quantization accuracy. The extraction and shaping unit 23a complements the positions of the binary data deleted from the encoded data 220 with a predetermined value (e.g., 0) in the shaping decoding data.
[0069] In this way, it is possible to improve the accuracy of restoring input data from encoded data. Even if a coding / decoding device 1b is not prepared for each coding code amount (compression rate), the coding / decoding device 1b can execute rate control.
[0070] Second embodiment The second embodiment differs from the first embodiment in that the encoding / decoding device executes scalable decoding. Scalable decoding is decoding decoded data (reconstructed data of input data) of an arbitrary code amount less than the encoded code amount from encoded data. The second embodiment will be described focusing on the differences from the first embodiment.
[0071] 5 is a diagram showing a configuration example of a coding / decoding device 1b. The coding / decoding device 1b is a system that performs coding processing (data compression processing) on input data and performs decoding processing on the coded data. In the decoding processing, the coding / decoding device 1b performs scalable decoding on decoding data extracted from the coded data.
[0072] The encoding / decoding device 1b includes an autoencoder 2 and a learning device 3. The autoencoder 2 includes an encoding unit 20, a quantization unit 21, a binarization unit 22, an extraction and shaping unit 23b, a debinarization unit 24, and a decoding unit 25. The learning device 3 includes a reconstruction error derivation unit 30, a quantization accuracy derivation unit 31, a code amount derivation unit 32, a code amount error derivation unit 33, and an optimization unit 34.
[0073] Fig. 6 is a diagram showing an example of scalable decoding. The extracting and shaping unit 23b acquires a quantization precision vector 310 from the quantization precision derivation unit 31. In Fig. 6, the quantization precision vector 310 is, for example, "[2, 1, 4, 3, 0]". That is, in this example, the quantization code amount "R" = 10 bits.
[0074] The extraction and shaping unit 23b extracts the decoded code amount "R dec " is acquired from, for example, an information processing device (not shown). In the second embodiment, the decoded code amount "R dec " is equal to or smaller than the quantized code amount "R". The extracting and shaping unit 23b acquires the coded data 220 from the binarization unit 22. The extracting and shaping unit 23b extracts, from the coded data 220, binarized data having a size of the decoded code amount specified using the quantization precision vector 310.
[0075] The extracting and shaping unit 23b performs a shaping process on the binarized data extracted from the encoded data 220. Here, the extracting and shaping unit 23b generates decoding data 230 in a shaped format by shaping the format of the extracted binarized data into the format of a quantized encoded feature vector.
[0076] In FIG. 6, the debinarization decoding data 240 includes, as an example, each of the debinarization data from element 241-1 to element 241-5.
[0077] The quantization precision associated with the binarized data "...0010" of the element 241-1 is "2" in the quantization precision vector 310. The quantization precision associated with the binarized data "...0000" of the element 241-2 is "1" in the quantization precision vector 310. The quantization precision associated with the binarized data "...0101" (in scalable decoding, "...0100") of the element 241-3 is "4" in the quantization precision vector 310. The quantization precision associated with the binarized data "...0111" (in scalable decoding, "...0110") of the element 241-4 is "3" in the quantization precision vector 310. The quantization precision associated with the binarized data "...0000" of the element 241-5 is "0" in the quantization precision vector 310.
[0078] The extracting and shaping unit 23b deletes the binarized data outside the range of the quantization accuracy (outside the rectangular frame indicated by the dashed line in FIG. 6) from the shaped format of the decoding data 230. In this way, the extracting and shaping unit 23b generates the decoding data 230 having the size of the decoding code amount specified by using the quantization accuracy vector 310.
[0079] Here, the extracting and shaping unit 23b scans the binarized data of all elements 241. The extracting and shaping unit 23b scans the binarized data of all elements 241 in the order from the most significant bit to the least significant bit of the binarized data. The extracting and shaping unit 23b also scans the binarized data of all elements 241 in the order from element 241-1 to element 241-5, for example. Each arrow of the dashed dotted line shown in the data for decoding 230 in FIG. 6 indicates such a scanning order. The extracting and shaping unit 23b ... calculates the decoded code amount "R dec The extracting and shaping unit 23b acquires elements of the size of ". The extracting and shaping unit 23b sets the remaining elements that have not been acquired to a predetermined value (for example, 0).
[0080] In the second embodiment, the decoded code amount “R dec" is 8 bits, for example. By scanning in order from element 241-1 to element 241-5, the extraction and shaping unit 23b obtains the most significant bit "0" within each quantization precision range from the binarized data. The extraction and shaping unit 23b obtains the most significant bits "1" and "1" on the lower side within each quantization precision range from the binarized data. The extraction and shaping unit 23b obtains the most significant bits "1", "0" and "1" on the further lower side within each quantization precision range from the binarized data. The extraction and shaping unit 23b obtains the most significant bits "0" and "0" within each quantization precision range from the binarized data. At this point, the decoded code amount "R dec " is extracted as 8-bit binary data. Therefore, the extraction and shaping unit 23b does not acquire the remaining binary data within the range of the quantization accuracy, but sets it to a predetermined value (for example, 0). In FIG. 6, the extraction and shaping unit 23b sets each value "1" of the remaining binary data within the range of the quantization accuracy to each value "0" as each value is surrounded by a solid-line rectangle in FIG. 6.
[0081] In FIG. 6, the quantization precision associated with the binary data "...0000" of element 241-5 is "0". Therefore, the binary data of element 241-5 is outside the range of the quantization precision. For this reason, in the scan, the extraction and shaping unit 23b does not acquire the binary data "...0000" of element 241-5. In this way, the extraction and shaping unit 23b deletes the binary data "...0000" of element 241-5, which has a quantization precision of "0", from the decoding data 230.
[0082] The extracting and shaping unit 23b generates the decoding data 230 having the size of the decoding code amount specified by using the quantization precision vector 310 by combining each of the acquired binary data ("0", "11", "101", "0000") in the order in which the binary data was acquired. In FIG. 6, the decoding data 230 having the size of the decoding code amount is "0111010000". In this way, only the binary data within each rectangular frame represented by a dashed line in FIG. 6 is transmitted to the debinarization unit 24 as the decoding data 230 having the size of the decoding code amount.
[0083] The debinarization unit 24 acquires the decoding data 230 having the size of the decoding code amount from the extraction and shaping unit 23b. The debinarization unit 24 generates the debinarized decoding data 240 by performing a debinarization process on the decoding data 230 having the size of the decoding code amount.
[0084] As described above, the extracting and shaping unit 23b obtains binary data within the range of quantization accuracy from the extracted binary data. The extracting and shaping unit 23b generates the decoding data 230 in a shaped format (shaped decoding data) by shaping the format of the binary data within the range of quantization accuracy. Here, the extracting and shaping unit 23b generates the decoding data 230 in a shaped format by extracting binary data of a size of the decoding code amount from the binary data within the range of quantization accuracy.
[0085] This makes it possible to improve the accuracy of restoring input data from encoded data. Even if a coding / decoding device 1b is not prepared for each coding code amount (compression rate), the coding / decoding device 1b can perform scalable decoding.
[0086] (Hardware configuration example) FIG. 7 is a diagram showing an example of a hardware configuration of a coding / decoding device 1 (coding device) (decoding device) (data compression device) in each embodiment. The coding / decoding device 1 corresponds to each of the coding / decoding device 1a and the coding / decoding device 1b. A part or all of each functional unit of the coding / decoding device 1 is realized as software by a processor 100 such as a CPU (Central Processing Unit) executing a program stored in a storage device 101 having a non-volatile recording medium (non-transient recording medium) and a memory 102. The program may be recorded in a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM (Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or a non-transient recording medium such as a storage device built into a computer system, such as a hard disk.
[0087] Some or all of the functional units of the encoding / decoding device 1 may be realized using hardware including electronic circuits (electronic circuits or circuitry) using, for example, an LSI (Large Scale Integrated circuit), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array).
[0088] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Industrial Applicability]
[0089] The present invention is applicable to devices that execute predetermined data processing. [Explanation of symbols]
[0090] Reference Signs List 1a, 1b... encoding / decoding device, 10... encoding / decoding device, 11... encoding unit, 12... quantization unit, 13... binarization unit, 14... decoding unit, 20... encoding unit, 21... quantization unit, 22... binarization unit, 23a, 23b... extraction and shaping unit, 24... inverse binarization unit, 25... decoding unit, 30... reconstruction error derivation unit, 31... quantization accuracy derivation unit, 32... code amount derivation unit, 33... code amount error derivation unit, 34... optimization unit, 100... processor, 101... storage device, 102... memory, 210... quantization encoding feature vector, 211... element, 220... encoding data, 230... decoding data, 240... inverse binarization decoding data, 241... element, 310... quantization accuracy vector
Claims
1. an encoding unit for converting input data into an encoded feature vector; a quantization accuracy derivation unit that derives a quantization accuracy for each encoding feature that is an element of the encoding feature vector in accordance with an encoding code amount; a quantization unit that performs a quantization process on the encoded feature vector based on the quantization accuracy to generate a quantized encoded feature vector having a size of a quantized code amount that is a target encoded code amount; a binarization unit that generates encoded data by performing a binarization process on the quantized encoded feature vector; a decoding unit that executes a decoding process on predetermined data corresponding to the encoded data; an extracting and shaping unit that extracts binary data of a size of a decoded code amount from the encoded data, and shapes the format of the extracted binary data into the format of the quantized encoded feature vector based on the quantization accuracy, thereby generating shaped decoding data; a debinarization unit that generates debinarized decoding data by performing a debinarization process on the shaping decoding data; Equipped with the decoding unit converts the inverse binarization decoding data into decoded data by executing the decoding process on the inverse binarization decoding data based on the decoded code amount. Encoding and decoding device.
2. a reconstruction error derivation unit that derives a reconstruction error, which is an error of the decoded data with respect to the input data; an optimization unit that derives a code amount error that is a difference between the encoded code amount and the quantized code amount that is the sum of the quantization accuracy, derives an objective function based on the reconstruction error and the code amount error, and updates at least one of a parameter used in an encoding process that converts the input data into the encoded feature vector, a parameter used in the decoding process, and a parameter used to derive the quantization accuracy based on the objective function. The encoding / decoding device according to claim 1 .
3. the binarization unit deletes binarized data outside the range of the quantization accuracy from the encoded data; the extracting and shaping unit complements the binarized data deleted from the encoded data with a predetermined value in the shaping decoding data based on the quantization accuracy.
3. The encoding / decoding device according to claim 1.
4. the extraction and shaping unit acquires binary data within the range of the quantization accuracy from the extracted binary data, and generates the shaped decoding data by shaping a format of the binary data within the range of the quantization accuracy.
3. The encoding / decoding device according to claim 1.
5. A coding / decoding method executed by a coding / decoding device, comprising: an encoding step for converting input data into an encoded feature vector; a quantization accuracy derivation step of deriving a quantization accuracy for each encoding feature, which is an element of the encoding feature vector, in accordance with an encoding code amount; a quantization step of generating a quantized encoded feature vector having a size of a quantized code amount that is a target encoded code amount by performing a quantization process on the encoded feature vector based on the quantization accuracy; a binarization step of generating encoded data by performing a binarization process on the quantized encoded feature vector; a decoding step of executing a decoding process on predetermined data corresponding to the encoded data; an extracting and shaping step of extracting binary data of a size of a decoded code amount from the encoded data, and shaping a format of the extracted binary data into a format of the quantized encoded feature vector based on the quantization accuracy, thereby generating shaped decoding data; a de-binarization step of generating de-binarized decoding data by performing a de-binarization process on the shaped decoding data; Including, the decoding step includes converting the inverse binarization decoding data into decoded data by performing the decoding process on the inverse binarization decoding data based on the decoded code amount. Encoding and decoding methods.
6. A program for causing a computer to function as the encoding / decoding device according to any one of claims 1 to 4.
Citation Information
Patent Citations
Information processing method and information processor
JP2020149311A
Encoding device, encoding method and program
WO2020230188A1
