Learning device, learning method, and program

Abstract

This training device comprises a training unit that updates through training encoding and decoding processes in an auto-encoding process that: involves vector quantization; uses both of main data features, which are features to be auto-encoded, and auxiliary features, which are features of the main data features; and performs vector quantization of the main data features and scalar quantization of the auxiliary features. In the training, the training unit executes a Straight Through Estimator for the main data features and executes a process of adding noises to the main data features.

Description

[Technical Field]

[0001] The present invention relates to a learning device, a learning method, and a program. [Background technology]

[0002] There are two types of quantization processes for data compression using deep learning: scalar quantization and vector quantization, with vector quantization providing better compression performance. One attempt at vector quantization is described in the following Non-Patent Document 1. The technology described in Non-Patent Document 1 is a method known as universal quantization, which performs encoding by adding uniform noise to the object to be encoded both during learning and during inference.

[0003] This method ensures that the coding process is the same during training and inference, preventing a decrease in the decoding performance, which is the accuracy of the self-encoding. However, uniform noise can increase quantization error, which reduces coding efficiency.

[0004] To improve this, a method has been proposed in Non-Patent Document 2. The technology described in Non-Patent Document 2 uses a Straight Through Estimator (STE) during both learning and inference. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Yoojin Choi, et al., “Variable Rate Deep Image Compression With a Conditional Autoencoder,” 2019. [Non-patent document 2] Mu Li, et al., “Learning Content-Weighted Deep Image Compression,” 2019. Summary of the Invention [Problem to be solved by the invention]

[0006] The technology in Non-Patent Document 2 also uses the same coding process during learning and inference, so degradation of decoding performance is suppressed. However, the technology described in Non-Patent Document 2 uses a mathematical model in which the gradient is fixed to 1 during backward STE, which may increase the frequency of overfitting.

[0007] In view of the above circumstances, an object of the present invention is to provide a technology that can simultaneously suppress a decrease in accuracy of auto-encoding and suppress the occurrence of over-learning. [Means for solving the problem]

[0008] One aspect of the present invention is a learning device that includes a learning unit that updates, through learning, encoding and decoding processes in a self-encoding process using vector quantization, the self-encoding process using main data features, which are features to be self-encoded, and auxiliary features, which are features of the main data features, and that performs vector quantization of the main data features and scalar quantization of the auxiliary features, and that, during the learning, the learning unit executes a straight-through estimator on the main data features and performs a process of adding noise to the main data features.

[0009] One aspect of the present invention is a learning method for a self-encoding process using vector quantization, which uses main data features, which are features to be self-encoded, and auxiliary features, which are features of the main data features, and which performs vector quantization of the main data features and scalar quantization of the auxiliary features, and which includes a learning step of updating encoding and decoding processes in the self-encoding process by learning, wherein the learning step executes a straight-through estimator on the main data features and performs a process of adding noise to the main data features.

[0010] One aspect of the present invention is a program for causing a computer to function as the learning device described above. [Effects of the Invention]

[0011] According to the present invention, it is possible to suppress a decrease in the accuracy of auto-encoding and to suppress the occurrence of over-learning. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is an explanatory diagram illustrating an overview of a learning device according to an embodiment. [Figure 2] FIG. 2 is an explanatory diagram illustrating LatticeVQ according to an embodiment. [Figure 3] FIG. 3 is a first explanatory diagram illustrating an example of adding noise according to the embodiment. [Figure 4] FIG. 2 is a second explanatory diagram illustrating an example of adding noise in the embodiment. [Figure 5] FIG. 4 is an explanatory diagram illustrating the relationship between a cumulative distribution function and occurrence probability in the embodiment. [Figure 6] FIG. 2 is an explanatory diagram illustrating hyper-rectangular parallelepiped division according to the embodiment. [Figure 7] FIG. 4 is a diagram showing an example of a processing flow executed by a learning unit in the embodiment. [Figure 8] FIG. 1 is a first explanatory diagram illustrating an overview of a self-encoding device according to an embodiment. [Figure 9] FIG. 2 is a second explanatory diagram illustrating an overview of the self-encoding device according to the embodiment. [Figure 10] 10 is a flowchart showing an example of the flow of processing executed by an encoder in the embodiment. [Figure 11] 10 is a flowchart showing an example of the flow of processing executed by a decoder in the embodiment. [Figure 12] FIG. 2 is a diagram illustrating an example of a hardware configuration of a learning device according to an embodiment. [Figure 13] FIG. 2 is a diagram showing an example of the configuration of a control unit included in the learning device according to the embodiment. [Figure 14]FIG. 2 is a diagram illustrating an example of a hardware configuration of a self-encoding device according to an embodiment. [Figure 15] FIG. 2 is a diagram showing an example of the configuration of a control unit included in the self-encoding device according to the embodiment. [Figure 16] FIG. 10 is an explanatory diagram illustrating an example of the configuration of a learning network in a modified example. [Figure 17] 10 is a flowchart showing an example of the flow of processing executed by a learning unit in a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0013] (Embodiment) FIG. 1 is an explanatory diagram illustrating an overview of a learning device 1 according to an embodiment. The learning device 1 includes a learning unit 10. The learning unit 10 performs learning so as to improve the performance of auto-encoding using vector quantization of data represented as tensors. The performance of auto-encoding is evaluated based on the smallness of the RD cost (D+λR), which is the sum of the error D between the original data and the restored data and the code amount R of the data, weighted by the Lagrange constant λ. The smaller the RD cost, the better the RD performance. The term "learning" refers to machine learning. The term "learning" refers to, for example, deep learning. As is well known, data auto-encoding refers to data compression.

[0014] The learning unit 10 includes a learning network 100 and an optimization unit 113. The learning network 100 is a neural network. The learning network 100 includes a main data acquisition unit 101, a main data encoding unit 102, an auxiliary data encoding unit 103, a main data noise addition unit 104, an auxiliary data noise addition unit 105, an auxiliary data probability estimation unit 106, an auxiliary data decoding unit 107, an auxiliary entropy acquisition unit 108, a main data probability estimation unit 109, a main entropy acquisition unit 110, a main data decoding unit 111, and a reconstruction error calculation unit 112. As will be described in detail later, the optimization unit 113 updates the learning network 100 based on the output of the learning network 100.

[0015] The main data acquisition unit 101 acquires data expressed as tensors as main data. The data expressed as tensors is, for example, image data. The data expressed as tensors may be, for example, time-series audio data of one or more channels. The data expressed as tensors may be, for example, point cloud data. The data expressed as tensors may be, for example, sensor data used in machine learning. Hereinafter, the data acquired by the main data acquisition unit 101 will be referred to as main data.

[0016] The main data encoding unit 102 executes a main data feature acquisition process. The main data feature acquisition process is a process of encoding main data. Since encoding is a process of acquiring information indicating the features of the object of encoding, encoding is a process of acquiring information indicating the features.

[0017] Therefore, the encoding of the main data by the main data side encoding unit 102 is a process of acquiring the feature of the main data. Therefore, the main data feature acquisition process is a process of acquiring the main data feature. The main data feature is encoded main data. The content of the main data feature acquisition process is updated by learning. In other words, the content of the process executed by the main data side encoding unit 102 is updated by learning.

[0018] The auxiliary data-side encoder 103 further encodes the main data features. As described above, encoding is a process of acquiring information indicating features, so encoding of the main data features by the auxiliary data-side encoder 103 is a process of acquiring the features of the main data features. Therefore, hereinafter, information obtained by further encoding the encoded main data will be referred to as auxiliary features. In other words, auxiliary features are information indicating the features of the main data features. Because auxiliary features are information obtained by encoding main data features, the entropy of auxiliary features is information with a lower entropy than main data features.

[0019] Hereinafter, the process of encoding the main data features will be referred to as the auxiliary feature acquisition process. The details of the auxiliary feature acquisition process are updated through learning. In other words, the details of the process executed by the auxiliary data encoding unit 103 are updated through learning.

[0020] The content of the auxiliary feature acquisition process is updated through learning so that the auxiliary features include more information in the main data statistics information. The main data statistics information is information indicating the statistics of each probability distribution to which the values ​​of each element of the tensor representing the main data follow. The statistics of the probability distribution are, for example, scatter. The statistics of the probability distribution may be not only the scatter but also a pair of the scatter and a representative value.

[0021] As explained in the fields of information processing and information theory, data generally appear according to a probability distribution, such as the probability distribution of each Roman character that appears in an English sentence. The values ​​of each element of a tensor that represents the main data also follow a probability distribution.

[0022] The main data-side noise adding unit 104 executes a noise-added main data feature acquisition process. The noise-added main data feature acquisition process is a process of executing a vector noise addition process on the main data feature. The vector noise addition process is a process of processing a K-dimensional vector (hereinafter referred to as a "noise-added vector") having a predetermined number K of elements (K is an integer equal to or greater than 2) as a processing target, and is a process of adding noise to the processing target. Therefore, the noise-added main data feature acquisition process is a process of acquiring information in which noise has been added to the main data feature (hereinafter referred to as a "noise-added main data feature").

[0023] Specifically, the noise-added vector is a K-dimensional vector included in a tensor that represents the main data feature. A K-dimensional vector included in a tensor means a vector whose k-th element (k is an integer between 1 and K) is the k-th element among K consecutive elements of all elements of the tensor.

[0024] The number of elements K of a noise-added vector is the number of elements mapped to one code by vector quantization by a device that performs vector quantization using the results of learning by the learning unit 10. Hereinafter, the result of learning by the learning unit 10 is referred to as a network learning result. Therefore, when K elements are collectively mapped to one code by vector quantization using the results of network learning, the number of elements of the noise-added vector is K. Note that when K elements are collectively mapped to one code by quantization, the resulting code is an index indicating a K-dimensional vector.

[0025] Therefore, the number of elements K of the noise-added vector is a predetermined number. Note that vector quantization of data is encoding of data, so vector quantization using the network learning results means encoding of data using the network learning results.

[0026] Here, a method for adding noise will be explained, but before explaining the addition of noise, LatticeVQ will be explained.

[0027] <latticevq> Vector quantization requires a representative vector. A lattice is a set of lattice points in a vector space. LatticeVQ is vector quantization in which the representative vectors are arranged in a lattice pattern in a vector space. In other words, LatticeVQ is vector quantization that satisfies the condition that the representative vectors are arranged in a lattice pattern in a vector space. LatticeVQ is known to have better RD performance than scalar quantization, except under certain conditions.

[0028] Fig. 2 is an explanatory diagram illustrating LatticeVQ in an embodiment. More specifically, Fig. 2 is an explanatory diagram illustrating LatticeVQ when the number of elements of the noise-added vector is two.

[0029] Figure 2 shows an example of an A2 lattice. In two dimensions, the A2 lattice is one type of lattice. In eight dimensions, the E8 lattice is one type of lattice. In 24 dimensions, the Reach lattice is one type of lattice. These are LatticeVQ lattices that maximize RD performance for a uniform distribution.

[0030] Hereinafter, a space in which representative vectors are arranged in a lattice pattern will be referred to as a lattice space. In other words, the representative vectors are located at each lattice point of the lattice space. Figure 2 is also a diagram showing an example of a lattice space. In the example of Figure 2, the lattice space is two-dimensional, but if the representative vectors are K-dimensional, the lattice space will also be K-dimensional.

[0031] <Adding noise> An example of adding noise will now be described using Fig. 3 and Fig. 4, taking the case where the number of elements is 2 as an example. Fig. 3 is a first explanatory diagram illustrating an example of adding noise in an embodiment. Fig. 4 is a second explanatory diagram illustrating an example of adding noise in an embodiment.

[0032] When adding noise, first, multiple random K-dimensional vectors are generated that are uniformly distributed within a (K-1)-dimensional sphere circumscribing the lattice unit area of ​​the origin lattice (hereinafter referred to as the "target lattice unit area"), and then placed in the lattice space. For example, in the example of Figure 4, when adding noise, first, they are generated until the number of internal points in Figure 4 becomes at least one. The lattice unit areas are each area that results from dividing the lattice space into multiple areas so that the division conditions are met.

[0033] The division conditions are that each region has the same size and shape and each region contains one lattice point in the lattice space. Therefore, a lattice unit region is a region that divides a vector space in which representative vectors are arranged in a lattice pattern and contains one lattice point in the vector space.

[0034] The (K-1)-dimensional sphere is a circle if K = 2, a sphere if K = 3, and a hypersphere if K is 4 or more. Region B1 in Figure 3 is an example of a target lattice unit region.

[0035] 3 and 4, the lattice unit region is a Voronoi region, which is each of the regions obtained by dividing a metric space such as a lattice space into a plurality of regions by Voronoi division.

[0036] The noise points in Figure 3 are an example of samples placed within the circumscribing circle of a target lattice unit area. Specifically, the process of placing samples within lattice space is the process of acquiring coordinates within the lattice space. Since a lattice unit area in lattice space is an area within the lattice space, the process of placing samples within the circumscribing circle of any one lattice unit area is the process of acquiring coordinates within that circumscribing circle.

[0037] Next, of the placed noise points, noise points located within the target lattice unit area (hereinafter referred to as "internal points") and noise points located outside the target lattice unit area (hereinafter referred to as "exclusion points") are distinguished, as shown in the example of Figure 4. Specifically, the process of distinguishing between internal points and exclusion points is a process of determining, for each noise point, based on its coordinates, whether the coordinates are within the target lattice unit area or outside the target lattice unit area. Specifically, the determination process is a process of vector quantizing the noise points using equation (11), and determining that points that match the origin lattice have coordinates within the area.

[0038] Next, one of the multiple noise points located within the target lattice unit area is randomly selected. The selected noise point is then added to one noise-added vector. Specifically, the addition process means adding the noise-added vector and a position vector indicating the selected noise point.

[0039] In this way, the position vector indicating the selected noise point is a quantity determined by a random number, and is therefore a type of noise. Furthermore, since the position vector indicating the selected noise point is expressed as a vector, the position vector indicating the selected noise point is noise expressed as a vector. Therefore, hereinafter, the position vector indicating the selected noise point will be referred to as the noise vector. In this way, noise is added to the noise-adding target vector.

[0040] Returning to the description of FIG. 1, the auxiliary data-side noise adding unit 105 executes noise-added auxiliary feature acquisition processing. The noise-added auxiliary feature acquisition processing is processing for adding noise to auxiliary features. In other words, the noise-added auxiliary feature acquisition processing is processing for acquiring information in which scalar noise has been added to auxiliary features (hereinafter referred to as "noise-added auxiliary features"). More specifically, the auxiliary data-side noise adding unit 105 adds scalar noise to each element of a tensor representing an auxiliary feature. The scalar noise is, for example, uniform noise between -½ and ½.

[0041] The auxiliary data-side probability estimation unit 106 executes auxiliary data-side probability estimation processing. The auxiliary data-side probability estimation processing is processing for estimating auxiliary data-side probabilities based on noise-added auxiliary features. The auxiliary data-side probabilities are information indicating the occurrence probability of each element of a tensor representing the auxiliary features. Information on the probability distribution of each element of the tensor representing the auxiliary features is used to estimate the occurrence probability of each element of the tensor representing the auxiliary features.

[0042] When the probability distribution of each element of the tensor representing the auxiliary feature is a predetermined probability distribution, the auxiliary data-side probability estimation unit 106 acquires the predetermined probability distribution by, for example, reading it from a predetermined storage device such as the storage unit 14 described below. Then, the auxiliary data-side probability estimation unit 106 estimates the occurrence probability of each element of the tensor representing the auxiliary feature based on the acquired probability distribution. Note that the predetermined probability distribution is a probability distribution expressed using, for example, a cumulative distribution function.

[0043] When the probability distribution of each element of a tensor representing an auxiliary feature is expressed by a parameterized auxiliary feature cumulative distribution function, the auxiliary data probability estimation unit 106 estimates the occurrence probability of each element of the tensor representing the auxiliary feature based on the parameterized auxiliary feature cumulative distribution function. The parameterized auxiliary feature cumulative distribution function is a parameterized function that indicates the probability distribution of each element of the tensor representing the auxiliary feature. Specifically, the parameters of the parameterized auxiliary feature cumulative distribution function are parameters that change according to statistics that represent the probability distribution of each element of the tensor representing the auxiliary feature.

[0044] The parameter values ​​of the parameterized auxiliary feature cumulative distribution function are updated through learning, i.e., the content of the processing performed by the auxiliary data-side probability estimation unit 106 is updated through learning.

[0045] The parameterized cumulative distribution function is, for example, a parameterized sigmoid function or a softplus function. As described above, the parameter values ​​of the parameterized auxiliary feature cumulative distribution function are updated by learning, and therefore the parameter values ​​of the parameterized cumulative distribution function are updated by learning.

[0046] Here, the relationship between the cumulative distribution function and occurrence probability will be explained using an example where the random variable is one-dimensional. FIG. 5 is an explanatory diagram illustrating the relationship between the cumulative distribution function and occurrence probability in an embodiment. Image G1 is an image showing an example of a probability density function indicating the probability density of random variable q. Δ indicates the quantization step size. Image G1 shows that the value obtained by integrating the probability density function over a range Δ within a domain centered on the value q is the occurrence probability p(q) of value q. In the one-dimensional case as in FIG. 5, the step size Δ is, for example, the size of the closed interval [-1 / 2, 1 / 2] (i.e., 1).

[0047] Image G2 is a cumulative distribution function cdf(q) obtained as a result of integrating the probability density function of image G1. In this way, the result of integrating the probability density function is expressed as a monotonically increasing function such as a sigmoid function. Note that the result of integrating the probability density function is expressed as a monotonically increasing function such as a sigmoid function regardless of the shape of the probability density function. More specifically, a cumulative distribution function is a function that satisfies the conditions cdf(-∞) = 0, cdf(∞) = 1, and the differential value of cdf(q) with respect to q being 0 or greater.

[0048] Image G2 shows that the probability of occurrence p(q) is equal to cdf(q+Δ / 2)-cdf(q-Δ / 2). Returning to the description of FIG.

[0049] The auxiliary data-side decoding unit 107 executes auxiliary feature decoding processing. The auxiliary feature decoding processing targets information obtained based on auxiliary features, and is a process of decoding the target. The target of the auxiliary feature decoding processing executed by the auxiliary data-side decoding unit 107 is noise-added auxiliary features. Hereinafter, information obtained by decoded noise-added auxiliary features will be referred to as auxiliary data. Therefore, the auxiliary data-side decoding unit 107 acquires auxiliary data by decoding the noise-added auxiliary features.

[0050] As described above, the RD cost decreases as the content of the auxiliary feature acquisition process is updated through learning. Therefore, the maximum amount of main data statistics information that the auxiliary data can contain increases as the content of the auxiliary feature acquisition process is updated.

[0051] The details of the auxiliary feature decoding process are updated through learning so as to reduce the RD cost, i.e., the details of the process executed by the auxiliary data side decoding unit 107 are updated through learning.

[0052] The auxiliary entropy acquisition unit 108 acquires auxiliary entropy based on the auxiliary data-side probability that is the estimation result of the auxiliary data-side probability estimation unit 106. The auxiliary entropy is the entropy of the auxiliary feature.

[0053] The main data side probability estimation unit 109 executes a main data side probability estimation process. The main data side probability estimation process is a process for estimating a main data side probability based on noise-added main data features and auxiliary data. The main data side probability is information indicating the occurrence probability of each element of a tensor indicating the main data features. Information on the probability distribution of each element of the tensor indicating the main data features is used to estimate the occurrence probability of each element of the tensor indicating the main data features.

[0054] When the probability distribution of each element of a tensor representing a main data feature is expressed using a parameterized main data feature cumulative distribution function, the main data side probability estimation unit 109 estimates the occurrence probability of each element of the tensor representing a main data feature based on the parameterized main data feature cumulative distribution function.

[0055] The parameterized main data feature cumulative distribution function is a parametrized cumulative distribution function that indicates the probability distribution of each element of a tensor that indicates the main data feature. The probability distribution is, for example, a Gaussian distribution.

[0056] The parameters of the parameterized main data feature cumulative distribution function are, specifically, statistics that represent the probability distribution of each element of the tensor that represents the main data feature. Therefore, in learning, values ​​indicated by the auxiliary data are used as the parameter values ​​of the parameterized auxiliary feature cumulative distribution function.

[0057] For example, if the probability distribution is a Gaussian distribution, the auxiliary data obtained in the trained learning network 100 indicates the representative value and the degree of dispersion of the Gaussian distribution.

[0058] <Estimation of occurrence probability> An example of the main data side probability estimation process will be described. More specifically, an example of the main data side probability estimation process when the above-mentioned LatticeVQ is used will be described. When vector quantization is performed, each vector of the main data is represented by one of the representative vectors. Therefore, the description of the main data side probability estimation process when LatticeVQ is used is, more specifically, a description of the process of estimating the occurrence probability of the representative vector when LatticeVQ is used. Hereinafter, the process of estimating the occurrence probability of the representative vector will be referred to as the representative vector occurrence probability estimation process. The representative vector occurrence probability estimation process is an example of the main data side probability estimation process.

[0059] <An example of representative vector occurrence probability estimation processing> As described above, the main data-side probability estimation process uses a parameterized main data feature cumulative distribution function. As described above, the parameterized main data feature cumulative distribution function is, specifically, a parameterized cumulative distribution function. As described above, a cumulative distribution function is the result of integrating a probability density function. Therefore, the representative vector occurrence probability estimation process is a process that uses a cumulative distribution function. Using a cumulative distribution function in the representative vector occurrence probability estimation process means using the result of integrating a parameterized probability density function over a lattice unit region.

[0060] As mentioned above, a Voronoi region is one of the lattice unit regions. Since Voronoi tessellation is a well-known method of region division, it is often used to obtain lattice unit regions. However, when the lattice space is two-dimensional, the shape of the Voronoi region is hexagonal, as mentioned above.

[0061] However, when integrating a function on a manifold, if the shape of the region representing the integration domain is hexagonal, it is not necessarily easy to perform the integration. Note that "easy to perform the integration" means that the amount of calculation required to obtain the integration result with a predetermined accuracy or higher is small.

[0062] Furthermore, when a manifold has dimensions higher than two, integrating a function over the region obtained as a result of the Voronoi tessellation is even more difficult than in the two-dimensional case. For example, in the four-dimensional case, it is often difficult to draw the shape of the Voronoi region in two-dimensional space, and integrating a function over such a manifold is also not easy. Thus, it is not necessarily easy to integrate a function over the region obtained as a result of the Voronoi tessellation as an integral region.

[0063] Therefore, it may not be easy to obtain a parameterized cumulative distribution function used in the representative vector occurrence probability estimation process.

[0064] Therefore, in the representative vector occurrence probability estimation process, for example, a parametrized cumulative distribution function obtained using the results of hypercuboid division instead of Voronoi division is used. Hypercuboid division is a process that divides a lattice space into lattice unit areas whose shape is a hypercuboid. Note that a two-dimensional hypercuboid is a rectangle, and a three-dimensional hypercuboid is a rectangular parallelepiped.

[0065] Fig. 6 is an explanatory diagram for explaining hyper-rectangular parallelepiped division in an embodiment. More specifically, Fig. 6 is a diagram showing an example of the result of hyper-rectangular parallelepiped division of a two-dimensional lattice space, together with an example of the result of Voronoi division. The "true region" and "approximate region" in Fig. 6 are both examples of lattice unit regions.

[0066] The "true region" is the region resulting from the Voronoi tessellation. In other words, the "true region" is the Voronoi region. The "true region" is a hexagonal grid unit region. The "approximate region" is a grid unit region resulting from the hypercuboid tessellation.

[0067] The shape of the "approximation region" is a rectangular parallelepiped. Of the two types of division results shown in Figure 6, an example of the result of the hyper-rectangular parallelepiped division performed by the main data side probability estimation unit 109 is the result of division using the "approximation region". S1 in Figure 6 is the length of the side of the rectangular parallelepiped, which is the shape of the "approximation region", and indicates the length of the side in the first dimension of the two-dimensional lattice space, while S2 indicates the length of the side in the second dimension.

[0068] The length of the first-dimensional side refers to the length of the rectangular parallelepiped in one of the oblique spaces when a rectangular parallelepiped in a lattice space is projected onto two orthogonal oblique spaces. The length of the second-dimensional side refers to the length of the rectangular parallelepiped in the other oblique space. In the following description, the length of the n-th dimension side of a hyper-rectangle in an N-dimensional lattice space refers to the length of the hyper-rectangle in the n-th oblique space when the hyper-rectangle is projected onto N one-dimensional oblique spaces that are orthogonal to each other. The hyper-rectangle in the oblique space is a straight line.

[0069] Incidentally, integrals with an integration domain of two or more dimensions can be obtained by iterative integration. When integrating a function with an N-dimensional hyper-rectangle as its integration domain, an iterative integration is performed by integrating the function N times. In the case of a hyper-rectangle, it is possible to perform an iterative integration with the integration domain of each of the N integrals being the edges of the hyper-rectangle.

[0070] When performing an iterative integral in which the integration domain of each of the N integrals is the edge of a hyper-rectangle, each integral is not affected by the results of the other integrals. On the other hand, in the case of a hyper-polyhedron that is not a hyper-rectangle, each integral in the iterative integral is affected by the results of the other integrals. Therefore, when the shape of the lattice unit region is a hyper-rectangle, it is easier to perform the integral than when the shape of the lattice unit region is a hyper-rectangle.

[0071] In the example of Figure 6, the "true domain" is hexagonal in shape and the "approximate domain" is rectangular, so performing integration over the "approximate domain" is easier than integration over the "true domain".

[0072] When obtaining a parametrized cumulative distribution function using the hyper-rectangle obtained as a result of the hyper-rectangle division as the integration domain, in each iteration of the integral, a one-dimensional probability density function that is not affected by other dimensions is integrated.

[0073] In the learning network 100, the probability density function that is the subject of the iterative integration is a function that indicates a predetermined type of distribution and is a parametrized function. The predetermined type of distribution is a Gaussian distribution. The parameter values ​​of the probability density function are values ​​that correspond to the auxiliary data. The parametrized cumulative distribution function is obtained by performing iterative integration on the probability density function, with the hyperrectangle obtained by the hyperrectangle division being the integration domain. In the learning network 100, the parameter values ​​of the probability density function are values ​​of the auxiliary data, so the parametrized cumulative distribution function obtained in this manner is a function that corresponds to the auxiliary data.

[0074] In the representative vector occurrence probability estimation process, the values ​​of the auxiliary data are substituted into the parameters of the cumulative distribution function obtained in advance in this way, and the representative vector occurrence probability estimation process estimates the occurrence probability of the representative vector using the cumulative distribution function obtained in this way.

[0075] In the representative vector occurrence probability estimation process, the occurrence probability of the representative vector is estimated by executing the processes shown in the following equations (1) and (2). Note that the cumulative distribution function cdf in equation (2) represents each cumulative distribution function obtained by each integral of the iterative integral. The result of integrating an m-dimensional (m is a natural number) cumulative distribution function for one dimension is an (m-1)-dimensional cumulative distribution function.

[0076] The right-hand side of equation (1) is an example of an occurrence probability obtained using a parameterized main data feature cumulative distribution function. Equation (1) is a function parameterized via the cumulative distribution function cfd of equation (2). Equation (2) is a cumulative distribution function obtained by integrating the probability density function in the one-dimensional direction of the lattice unit area. Equation (1) is the product of the occurrence probabilities of each dimension obtained by equation (2). Therefore, equation (1) is an occurrence probability expressed using the result of integrating the probability density function over the entire lattice unit area.

[0077]

number

[0078]

number

[0079] i is an identifier that identifies each lattice point. j represents each dimension of the lattice space. The left side of equation (1) indicates the occurrence probability of the representative vector. Since each lattice point represents a representative vector, the occurrence probability p i is the probability of occurrence of each representative vector p i It means. Δ j means the length of the dimension represented by the identifier j of the lattice unit area. Since the shape and size of the lattice unit area are predetermined, Δ j is a predetermined length.

[0080] Note that the hat-marked y in the following formula (3) means the noise-added main data feature. Hereinafter, the hat-marked A will be written as A^. Therefore, for example, the hat-marked y in the following formula (3) is y^.

[0081]

number

[0082] Equation (1) is the probability of occurrence p i is the probability p obtained for each dimension i If the shape of the lattice unit area is not a super-rectangular prism, the probability of occurrence p i is the probability p obtained for each dimension i Therefore, since the shape of the lattice unit area is a super-rectangular parallelepiped, the main data side probability estimation unit 109 calculates the occurrence probability p i It is easy to obtain.

[0083] Note that the above equations (1) and (2) are obtained under the assumption that the covariance between dimensions is 0. However, if the shape of the lattice unit area is a hypercube, the covariance between dimensions can be made 0 by rotating the coordinate axes of the lattice space so that they are parallel to each edge of the hypercube. Therefore, even if the covariance between dimensions is not 0, equations (1) and (2) are still valid by rotating the coordinate axes. Note that, as is well known in linear algebra, rotating the coordinate axes is a unitary transformation, and therefore does not change the contents of equations (1) and (2). The process of making the covariance 0 is the process of diagonalizing the matrix that represents the variance between dimensions.

[0084] In this way, the main data-side probability estimation process estimates the occurrence probability of each element of a tensor representing the main data feature quantity by using the result of integrating the parameterized probability density function with a lattice unit region whose shape is a hyper-rectangular parallelepiped as the integration region. Note that the occurrence probability of each element of a tensor representing the main data feature quantity is the occurrence probability of a representative vector representing the main data feature quantity.

[0085] <Processing for determining the range of the hypercuboid> So far, we have explained the use of a hyper-rectangle as a lattice unit area, but have not touched on the process of determining the shape and size of the hyper-rectangle. Here, we will explain an example of the process of determining the shape and size of a lattice unit area that at least satisfies the condition of being a hyper-rectangle (hereinafter referred to as the "hyper-rectangle determination process").

[0086] In the hyper-rectangle determination process, first, the adjacent lattices of the origin lattice are calculated. The origin lattice is the lattice point located at the origin of the lattice space. The adjacent lattices are the lattice points next to the origin. Next, in the hyper-rectangle determination process, a hyper-rectangle that satisfies the hyper-rectangle determination conditions is calculated. The hyper-rectangle determination conditions include the condition that the lattice unit area of ​​the origin lattice does not overlap with the lattice unit areas of each adjacent lattice, and the condition that the volume of the hyper-rectangle is equal to the volume of the Voronoi region.

[0087] As a specific example of the hypercuboid determination process, we will explain the conditions used to determine the shape and size of a hypercuboid in the 8-dimensional E8 lattice space. There are two types of adjacent lattices of the origin lattice. One is [±1 2 , 0 6 ]. The other one is [±(1 / 2) 8 ], where the superscript indicates the number of dimensions. For example, [1 2 , 0 6 ] means [1, 1, 0, 0, 0, 0, 0, 0]. Note that the square brackets [] above represent a vector.

[0088] <Notation for indicating a hypercuboid> Hypercuboid [s1 a , s2 b s3 c ] indicates that the length of each side from the 1st dimension to the ath dimension is s1, and the length of each side from the (a+1)th dimension to the (a+b)th dimension is s2. In this way, a superscript in a notation that uses superscripts indicates that the length of the sides of the hypercuboid is the same length s for a series of dimensions equal to the number of superscripts, with s being the character to which the superscript is attached. In other words, s a indicates that the length of the edge of the hypercuboid is s for a series of a dimensions. Note that information on the order of the dimensions is required to determine whether the dimensions are continuous or not, but the order of the dimensions is predetermined.

[0089] Furthermore, [±1 2 , 0 6 The "±" in [±1] indicates that there are cases of +1 and -1. 2 , 0 6 ] is specifically [1, 1, 0 6 ] and [-1, 1, 0 6 ] and [1, -1, 0 6 ] and [-1, -1, 0 6 ] means four vectors.

[0090] Returning to the explanation of determining the shape and size of a hypercube in an eight-dimensional lattice space, there are two conditions for the hypercube of each adjacent lattice not to overlap with the hypercube of the origin lattice. One is that at least seven of the sides (elements) of the hypercube must be 1 or less. The other is that at least one of the sides (elements) of the hypercube must be 1 / 2 or less.

[0091] Furthermore, since the volume of the Voronoi region of the E8 lattice is 1, the hyper-rectangular parallelepiped satisfies the condition of the following formula (4).

[0092]

number

[0093] In the case of the 8-dimensional E8 lattice space, for example, s=[1 / 2, 1 6 , 2] is obtained.

[0094] Here is an example for other dimensions. In the case of a two-dimensional A2 lattice space, the range of the lattice unit area is the range enclosed by a hyper-rectangular prism expressed by the following equation (5).

[0095]

number

[0096] In the case of an eight-dimensional E8 lattice space, the range of the lattice unit area is the range enclosed by a hyper-rectangular parallelepiped expressed by the following equation (6).

[0097]

number

[0098] In the case of a 24-dimensional Reach lattice space, the range of the lattice unit area is the range enclosed by a hyper-rectangular parallelepiped expressed by the following equation (7).

[0099]

number

[0100] In this way, the range of the hyper-rectangular parallelepiped (the shape and size of the lattice unit area) is determined. Returning to the explanation of FIG.

[0101] The main entropy acquisition unit 110 acquires the entropy of the main data feature amount based on the estimation result of the main data side probability estimation unit 109. Hereinafter, the entropy of the main data feature amount will be referred to as the main entropy.

[0102] The main data side decoding unit 111 executes a main data feature decoding process. The main data feature decoding process is a process for processing information obtained based on the main data feature, and is a process for decoding the processing object. The processing object of the main data feature decoding process executed by the main data side decoding unit 111 is the noise-added main data feature. The content of the decoding process of the main data side decoding unit 111 is updated by learning.

[0103] The reconstruction error calculation unit 112 calculates the difference between the decoding result of the main data side decoding unit 111 and the main data acquired by the main data acquisition unit 101. Hereinafter, the difference between the decoding result of the main data side decoding unit 111 and the main data acquired by the main data acquisition unit 101 is referred to as the reconstruction error. The difference between the decoding result of the main data side decoding unit 111 and the main data acquired by the main data acquisition unit 101 may be expressed, for example, by the sum of mean squared errors or by binary cross entropy.

[0104] The optimization unit 113 updates the learning network 100 based on the auxiliary entropy, the principal entropy, and the reconstruction error. The auxiliary entropy, the principal entropy, and the reconstruction error are all examples of outputs from the learning network 100. Specifically, the optimization unit 113 updates the learning network 100 so as to reduce the optimization error, the principal entropy, and the auxiliary entropy. The objective function used by the optimization unit 113 is, for example, L=D+λ(R y +R z )

[0105] The symbol L represents the objective function. The symbol D represents the reconstruction error. The symbol lambda is a predetermined constant. The symbol R y represents the auxiliary entropy. The symbol R z represents the principal entropy.

[0106] Since small entropy means a short code length, the optimization unit 113 updates the learning network 100 to reduce the entropy. Also, since a smaller optimization error means higher accuracy of auto-encoding, the optimization unit 113 updates the learning network 100 to reduce the optimization error.

[0107] For example, the learning network 100 is updated by using a gradient method to solve the minimization problem of the objective function L. That is, the learning network 100 is updated by updating the values ​​of each parameter of the learning network 100 using, for example, the backpropagation method.

[0108] Specifically, updating the learning network 100 means updating the processing contents of the main data side encoding unit 102, auxiliary data side encoding unit 103, auxiliary data side probability estimating unit 106, auxiliary data side decoding unit 107, and main data side decoding unit 111.

[0109] <Relationship between autoencoding and learning network 100> The learning network 100 includes a main data-side noise adding unit 104. The main data-side noise adding unit 104 itself does not perform quantization processing. However, it is precisely because the processing of the main data-side noise adding unit 104 is included that learning is performed, thereby improving the performance of self-encoding using vector quantization. The reason for this is explained below.

[0110] It is known that if vector quantization is performed during learning, the gradient becomes 0, making it impossible to perform learning that improves the efficiency of the auto-encoding process using vector quantization. Therefore, in order to improve the efficiency of the auto-encoding process through learning, a process of adding noise to the vectors is performed during learning instead of vector quantization itself.

[0111] By performing a process of adding noise instead of vector quantization itself, the performance of other processes that generate the information used in vector quantization is improved, rather than the process of performing vector quantization itself. As a result, even if vector quantization is performed instead of a process of adding noise during auto-encoding, auto-encoding can be performed more efficiently than before learning.

[0112] Regarding the learning unit 10, the main data-side noise adding unit 104 adds noise to the noise-adding target vector. Therefore, by performing learning including the main data-side noise adding unit 104, the gradient does not become 0. As a result, it is possible to update the contents of the main data-side encoding unit 102, auxiliary data-side encoding unit 103, auxiliary data-side probability estimating unit 106, auxiliary data-side decoding unit 107, and main data-side decoding unit 111, which are also used in auto-encoding using vector quantization.

[0113] Therefore, what has been described up to this point as the network learning result specifically refers to the peripheral processing updated by the learning of the neural network (i.e., learning network 100) that adds noise instead of vector quantization. The peripheral processing is processing that generates information used in vector quantization. Specifically, the peripheral processing is processing that is executed by each of main data side encoding unit 102, auxiliary data side encoding unit 103, auxiliary data side probability estimating unit 106, auxiliary data side decoding unit 107, and main data side decoding unit 111.

[0114] 7 is a diagram showing an example of the flow of processing executed by the learning unit 10 in the embodiment. N ] is acquired (step S101). The main data x is obtained from x1 to x N It is an N-dimensional vector with N elements from x1 to x N Each element from x1 to x is a tensor. N Each element up to may be a scalar or a vector.

[0115] Next, the main data encoding unit 102 executes a main data feature acquisition process (step S102). That is, the main data encoding unit 102 encodes the main data x. By encoding the main data, the main data feature y=f enc (x) is obtained. Function f enc (x) is a function (hereinafter referred to as "main data encoding function") that expresses the encoding process of the main data x. Note that the main data feature y is a tensor composed of k K-dimensional vectors.

[0116] Next, the auxiliary data encoding unit 103 executes an auxiliary feature acquisition process (step S103). By executing the auxiliary feature acquisition process, the auxiliary feature z=g enc (y) is obtained. Function g enc (y) is a function (hereinafter referred to as the "main data feature encoding function") that expresses the encoding process of the main data feature y. The auxiliary feature z is a tensor such as a vector.

[0117] Next, the main data-side noise adding unit 104 executes a noise-added main data feature acquisition process (step S104). By executing the noise-added main data feature acquisition process, noise is added to the main data feature. That is, by executing the noise-added main data feature acquisition process, the noise-added main data feature y^=[y1^, y2^, . . . , y k ^] is obtained, where y i ^=y i +u y y i represents the i-th vector element of the main data feature y. y represents noise.

[0118] Next, the auxiliary data-side noise adding unit 105 executes noise-added auxiliary feature acquisition processing (step S105). By executing the noise-added auxiliary feature acquisition processing, noise is added to the auxiliary feature. That is, by executing the noise-added auxiliary feature acquisition processing, the noise-added auxiliary feature z^=[z1^, z2^, . . . , z w ^] is obtained. Note that w is an integer equal to or greater than 1. However, z i ^=z i +u z z i represents the i-th element of the auxiliary feature z. z represents noise.

[0119] Next, the auxiliary data-side probability estimation unit 106 executes the auxiliary data-side probability estimation process (step S106). The auxiliary data-side probability estimation unit 106 executes the auxiliary data-side probability estimation process to estimate the auxiliary data-side probability. Specifically, the auxiliary data-side probability is expressed by the following equation (8).

[0120]

number

[0121] The symbol on the left side of equation (8) represents the probability of the auxiliary data, and the symbol h represents the cumulative distribution function of the parametrized auxiliary feature.

[0122] Next, the auxiliary data side decoding unit 107 performs auxiliary feature decoding processing on the noise-added auxiliary feature (step S107). By executing the auxiliary feature decoding processing, the noise-added auxiliary feature is decoded. That is, by executing the auxiliary feature decoding processing, the auxiliary data θ=g dec (z^) is obtained. Function g dec (ẑ) is a function (hereinafter referred to as the “auxiliary feature decoding function”) that expresses the decoding process of the noisy auxiliary feature ẑ. Note that the auxiliary data θ is a tensor such as a vector.

[0123] Next, the auxiliary entropy acquisition unit 108 acquires auxiliary entropy based on the auxiliary data-side probability (step S108). Specifically, the auxiliary entropy acquisition unit 108 acquires auxiliary entropy by executing the process expressed by the following equation (9).

[0124]

number

[0125] The symbol on the left side of equation (9) represents auxiliary entropy.

[0126] Next, the main data-side probability estimation unit 109 executes a main data-side probability estimation process (step S109). The main data-side probability estimation unit 109 executes the main data-side probability estimation process to estimate the main data-side probability based on the noise-added main data feature quantity and the auxiliary data. Specifically, the main data-side probability is expressed by the above-mentioned formula (1).

[0127] Next, the main entropy acquisition unit 110 acquires the main entropy based on the main data side probability (step S110). Specifically, the main entropy acquisition unit 110 acquires the main entropy by executing the process expressed by the following equation (10).

[0128]

number

[0129] The symbol on the left side of equation (10) represents the principal entropy.

[0130] Next, the main data side decoding unit 111 executes a main data feature decoding process on the noise-added main data feature (step S111). By executing the main data feature decoding process, the noise-added main data feature is decoded. Hereinafter, information obtained by decoded the noise-added main data feature is referred to as decoded main data. Therefore, the main data side decoding unit 111 executes the main data feature decoding process to decode the decoded main data x^=f dec (y^) is obtained by the function f dec (y^) is a function (hereinafter referred to as the "main data feature decoding function") that expresses the process of decoding the noise-added main data feature y^. Note that the decoded main data x^ is a tensor such as a vector.

[0131] Next, the reconstruction error calculation unit 112 acquires the difference between the decoded main data and the main data acquired by the main data acquisition unit 101 (step S112). The difference between the decoded main data and the main data acquired by the main data acquisition unit 101 is the reconstruction error.

[0132] Next, the optimization unit 113 updates the training network 100 based on the auxiliary data entropy, the main data entropy, and the reconstruction error (step S113). Next, the optimization unit 113 determines whether a predetermined termination condition for training (hereinafter referred to as the "training termination condition") is satisfied (step S114). The training termination condition is, for example, a condition that the training network 100 has been updated a predetermined number of times.

[0133] If the learning termination condition is satisfied (step S114: YES), the process ends. On the other hand, if the learning termination condition is not satisfied (step S114: NO), the process returns to step S101. The peripheral process at the time when the learning termination condition is satisfied is used for vector quantization as the learned peripheral process.

[0134] The processes in steps S101 to S114 may be executed in any order as long as they do not violate the law of causality.

[0135] As described above, the auxiliary data probability estimation unit 106 may acquire a predetermined probability distribution by reading it from a predetermined storage device. In such a case, the content of the auxiliary data probability estimation process is not updated by learning. Therefore, the learned auxiliary data probability estimation process is the same as the auxiliary data probability estimation process before learning.

[0136] More specifically, updating the content of the auxiliary data-side probability estimation process is updating the parameterized auxiliary feature cumulative distribution function h. Therefore, when the auxiliary data-side probability estimation unit 106 acquires a predetermined probability distribution by, for example, reading it from a predetermined storage device, the learned parametrized auxiliary feature cumulative distribution function h is the same as the parametrized auxiliary feature cumulative distribution function h before learning.

[0137] In this way, the learning unit 10 updates the encoding and decoding processes in the auto-encoding process using vector quantization through learning. The auto-encoding process using vector quantization uses main data features, which are features to be auto-encoded, and auxiliary features, which are features of the main data features. Furthermore, the auto-encoding process using vector quantization performs entropy coding on the result of vector quantizing the main data features and entropy coding on the result of scalar quantizing the auxiliary features.

[0138] The encoding process in such a self-encoding process using vector quantization specifically comprises a main data feature acquisition process executed by the main data side encoding unit 102 and an auxiliary feature acquisition process executed by the auxiliary data side encoding unit 103. Furthermore, the decoding process in such a self-encoding process using vector quantization specifically comprises an auxiliary feature decoding process executed by the auxiliary data side decoding unit 107 and a main data feature decoding process executed by the main data side decoding unit 111.

[0139] Auto-encoding using vector quantization using trained peripheral processing will be described with reference to Figs. 8 and 9. "Trained" means that the learning termination condition has been met. More specifically, an auto-encoding device 2 that performs encoding and decoding processes will be described as an example of a device that performs auto-encoding using vector quantization using trained peripheral processing. The auto-encoding device 2 is a type of auto-encoder (autoencoder).

[0140] Fig. 8 is a first explanatory diagram illustrating an overview of the self-encoding device 2 in the embodiment. Fig. 9 is a second explanatory diagram illustrating an overview of the self-encoding device 2 in the embodiment. More specifically, Fig. 8 is an explanatory diagram illustrating encoding processing executed by the self-encoding device 2, and Fig. 9 is an explanatory diagram illustrating decoding processing executed by the self-encoding device 2.

[0141] The auto-encoding device 2 is a type of auto-encoder and therefore includes an encoder and a decoder. Specifically, the auto-encoding device 2 includes an encoder 200 and a decoder 212. The encoder 200 includes an auto-encoding target acquisition unit 201, a learned main data side encoding unit 202, a learned auxiliary data side encoding unit 203, a vector quantization unit 204, a scalar quantization unit 205, a learned auxiliary data side probability estimation unit 206, a learned auxiliary data side decoding unit 207, an auxiliary entropy encoding unit 208, a main data side probability estimation unit 209, a main entropy encoding unit 210, and a data multiplexing unit 211.

[0142] The decoder 212 includes an encoded data acquisition unit 213 , a data separation unit 214 , an auxiliary entropy decoding unit 215 , a learned auxiliary data side decoding unit 216 , a main entropy decoding unit 217 , and a learned main data side decoding unit 218 .

[0143] The auto-encoding target acquisition unit 201 acquires data to be subjected to auto-encoding as main data. Hereinafter, the data to be subjected to auto-encoding will be referred to as the auto-encoding target.

[0144] The learned main data side encoding unit 202 executes a learned main data feature acquisition process on the target for auto-encoding. By executing the learned main data feature acquisition process, the learned main data side encoding unit 202 acquires the main data feature of the target for auto-encoding.

[0145] The learned auxiliary data-side encoder 203 executes a learned auxiliary feature acquisition process on the main data features to be auto-encoded. By executing the learned auxiliary feature acquisition process, the learned auxiliary data-side encoder 203 acquires the auxiliary features to be auto-encoded.

[0146] The vector quantization unit 204 performs vector quantization processing on the main data feature to be self-encoded. By performing the vector quantization processing, the vector quantization unit 204 acquires the vector-quantized main data feature to be self-encoded (hereinafter referred to as "vector-quantized feature").

[0147] The scalar quantization unit 205 performs scalar quantization on the auxiliary features to be auto-encoded. By performing the scalar quantization, the scalar quantization unit 205 acquires scalar-quantized auxiliary features to be auto-encoded (hereinafter referred to as "scalar quantized features").

[0148] The learned auxiliary data-side probability estimation unit 206 executes learned auxiliary data-side probability estimation processing to estimate auxiliary data-side probabilities of the auto-encoding target based on the scalar quantized features.

[0149] The learned auxiliary data-side decoding unit 207 executes learned auxiliary feature decoding processing on the scalar quantized features. That is, the learned auxiliary data-side decoding unit 207 decodes the scalar quantized features. Hereinafter, information obtained by decoded scalar quantized features is referred to as quantized auxiliary data. Therefore, the learned auxiliary data-side decoding unit 207 obtains quantized auxiliary data by decoding the scalar quantized features.

[0150] The auxiliary entropy coding unit 208 performs entropy coding of the scalar quantized feature based on the scalar quantized feature and the auxiliary data side probability to be self-encoded. The entropy coding is, for example, arithmetic coding.

[0151] The main data side probability estimation unit 209 estimates the main data side probability of the auto-encoding target based on the vector quantization feature amount and the quantized auxiliary data.

[0152] The main entropy coding unit 210 performs entropy coding of the vector quantized feature quantity based on the vector quantized feature quantity and the main data side probability of the self-encoding target. The entropy coding is, for example, arithmetic coding.

[0153] The data multiplexing unit 211 outputs the entropy-encoded vector quantized feature and the entropy-encoded scalar quantized feature to the decoder 212. In this manner, the encoder 200 encodes the object to be auto-encoded.

[0154] The coded data acquisition unit 213 acquires the entropy-coded vector quantized feature amount and the entropy-coded scalar quantized feature amount.

[0155] The data separation unit 214 acquires the entropy-coded vector quantization features and the entropy-coded scalar quantization features acquired by the encoded data acquisition unit 213. The data separation unit 214 outputs the entropy-coded scalar quantization features to the auxiliary entropy decoding unit 215, and outputs the entropy-coded vector quantization features to the main entropy decoding unit 217.

[0156] The trained auxiliary entropy decoding unit 215 performs entropy decoding on the entropy-encoded scalar quantized features using the trained parametrized auxiliary feature cumulative distribution function.

[0157] The learned auxiliary data side decoding unit 216 executes learned auxiliary feature decoding processing on the result of entropy decoding by the learned auxiliary entropy decoding unit 215 .

[0158] The main entropy decoding unit 217 performs entropy coding of the entropy coded vector quantization features based on the entropy coded vector quantization features and the results of the learned auxiliary feature decoding process by the learned auxiliary data side decoding unit 216. More specifically, the main entropy decoding unit 217 performs entropy decoding of the entropy coded vector quantization features using a decoding cumulative distribution function. The decoding cumulative distribution function is a parameterized main data feature cumulative distribution function whose parameter values ​​are values ​​indicated by the results of the learned auxiliary feature decoding process by the learned auxiliary data side decoding unit 216.

[0159] The learned main data side decoding unit 218 executes a learned main data feature decoding process on the result of decoding by the main entropy decoding unit 217.

[0160] In this way, the decoder 212 decodes the auto-encoding target encoded by the encoder 200. Also, in this way, the auto-encoding device 2 performs auto-encoding of the auto-encoding target.

[0161] 10 is a flowchart showing an example of the flow of processing executed by the encoder 200 according to the embodiment. N ] is obtained (step S201). The autoencoding target X is X1 to X N is an N-dimensional vector with N elements from X1 to X N Each element from X1 to X is a tensor. N Each element up to may be a scalar or a vector.

[0162] Next, the learned main data side encoding unit 202 executes a learned main data feature acquisition process (step S202). That is, the learned main data side encoding unit 202 encodes the auto-encoding target X. By encoding the auto-encoding target, the main data feature Y of the auto-encoding target is obtained as Y=F. enc (Y) is obtained. Function F enc (X) is the trained main data encoding function. The main data feature Y is a tensor such as a vector.

[0163] Next, the learned auxiliary data side encoding unit 203 executes a learned auxiliary feature acquisition process (step S203). By executing the learned auxiliary feature acquisition process, the auxiliary feature Z to be auto-encoded is obtained as Z=G enc (Y) is obtained. Function G enc (Y) is the trained main data feature encoding function. The auxiliary feature Z is a tensor such as a vector.

[0164] Next, the vector quantization unit 204 performs vector quantization on the main data feature Y to be auto-encoded (step S204). By performing vector quantization, the vector quantized feature Y^=[Y1^, Y2^, . . . , Y k ^]=[Q(Y1), Q(Y2),..., Q(Y k )] is obtained. Y i represents the i-th element of the main data feature Y to be auto-encoded. i represents the i-th element of the vector quantized feature Ŷ. Q is a function expressed by the following equation (11).

[0165]

number

[0166] The symbol Λ denotes the set of all lattice points.

[0167] Next, the scalar quantization unit 205 performs scalar quantization on the auxiliary feature Z to be auto-encoded (step S205). By executing the scalar quantization process, the scalar quantization unit 205 obtains the scalar quantized feature Z^=round(Z), where round represents a rounding process.

[0168] Next, the learned auxiliary data side probability estimation unit 206 executes learned auxiliary data side probability estimation processing (step S206). The learned auxiliary data side probability estimation unit 206 executes learned auxiliary data side probability estimation processing to estimate the auxiliary data side probability of the object to be auto-encoded based on the scalar quantized feature Z^. Specifically, the auxiliary data side probability of the object to be auto-encoded is expressed by the following equation (12).

[0169]

number

[0170] The symbol on the left side of equation (12) represents the probability of the auxiliary data to be auto-encoded. Symbol H is the cumulative distribution function of the trained parametrized auxiliary features.

[0171] The learned auxiliary data-side decoder 207 performs a learned auxiliary feature decoding process on the scalar quantized feature Ẑ (step S207). That is, the learned auxiliary data-side decoder 207 decodes the scalar quantized feature to generate quantized auxiliary data Θ=G dec (Z^) is obtained. Function G dec (Z^) is the trained auxiliary feature decoding function. The quantized auxiliary data Θ is a tensor such as a vector.

[0172] Next, the auxiliary entropy coding unit 208 performs entropy coding of the scalar quantized feature Z^ based on the scalar quantized feature Z^ and the auxiliary data side probability to be auto-encoded (step S208).

[0173] Next, the main data side probability estimation unit 209 estimates the main data side probability of the auto-encoding target based on the vector quantized feature value Y^ and the quantized auxiliary data Θ (step S209).

[0174] Next, the main entropy coding unit 210 performs entropy coding of the vector quantized feature value Y^ based on the vector quantized feature value Y^ and the main data side probability to be self-encoded (step S210).

[0175] Next, the data multiplexing unit 211 outputs the entropy-encoded vector quantized feature quantity and the entropy-encoded scalar quantized feature quantity to the decoder 212 (step S211).

[0176] The series of processes from step S201 to step S211 is an example of encoding processing by the encoder 200. Note that the processes from step S201 to step S211 may be executed in any order as long as they do not violate the law of causality.

[0177] 11 is a flowchart showing an example of the flow of processing executed by the decoder 212 in this embodiment. The encoded data acquisition unit 213 acquires the results of encoding by the encoder 200 (step S301). Specifically, the results of encoding by the encoder 200 are the entropy-encoded vector quantized features and entropy-encoded scalar quantized features output in step S211.

[0178] Next, the data separating unit 214 separates the entropy-coded scalar quantized feature obtained in step S301 from the entropy-coded vector quantized feature obtained in step S302 (step S302). Specifically, separating means outputting the entropy-coded scalar quantized feature obtained in step S301 to the auxiliary entropy decoding unit 215, and outputting the entropy-coded vector quantized feature obtained in step S302 to the main entropy decoding unit 217.

[0179] Next, the trained auxiliary entropy decoding unit 215 performs entropy decoding on the entropy-encoded scalar quantized feature using the trained parametrized auxiliary feature cumulative distribution function (step S303).

[0180] Next, the learned auxiliary data side decoding unit 216 executes learned auxiliary feature decoding processing on the result of entropy decoding by the learned auxiliary entropy decoding unit 215 (step S304).

[0181] Next, the main entropy decoding unit 217 performs entropy decoding of the entropy-encoded vector quantization feature quantity using the decoded cumulative distribution function (step S305).

[0182] Next, the learned main data side decoding unit 218 executes a learned main data feature decoding process on the result of decoding by the main entropy decoding unit 217 (step S306).

[0183] The series of processes from step S301 to step S306 is an example of the decoding process by the decoder 212. Note that each of the processes from step S301 to step S306 is executed after the encoding process by the encoder 200, such as step S211, and may be executed in any order as long as it does not violate the law of causality.

[0184] <Hardware Description> 12 is a diagram showing an example of the hardware configuration of a learning device 1 according to an embodiment. The learning device 1 includes a control unit 11 having a processor 91 such as a CPU (Central Processing Unit) and a memory 92 connected via a bus, and executes a program. By executing the program, the learning device 1 functions as a device including the control unit 11, input unit 12, communication unit 13, storage unit 14, and output unit 15.

[0185] More specifically, the processor 91 reads out a program stored in the storage unit 14 and stores the read out program in the memory 92. When the processor 91 executes the program stored in the memory 92, the learning device 1 functions as a device including a control unit 11, an input unit 12, a communication unit 13, a storage unit 14, and an output unit 15.

[0186] The control unit 11 controls the operation of various functional units included in the learning device 1. The control unit 11 controls, for example, the operation of the output unit 15. The control unit 11 records, in the storage unit 14, various pieces of information generated by learning, for example.

[0187] Input unit 12 includes input devices such as a mouse, keyboard, and touch panel. Input unit 12 may be configured as an interface that connects these input devices to learning device 1. Input unit 12 accepts input of various types of information to learning device 1.

[0188] The communication unit 13 includes a communication interface for connecting the learning device 1 to an external device. The communication unit 13 communicates with the external device via wired or wireless communication. The external device is, for example, a device that transmits main data used for learning. The communication unit 13 acquires the main data used for learning by communicating with the device that transmits the main data. The external device is, for example, the autoencoding device 2. The communication unit 13 transmits the network learning results to the autoencoding device 2 by communicating with the autoencoding device 2. Note that the main data does not necessarily have to be input via the communication unit 13, but may be input to the input unit 12.

[0189] The storage unit 14 is configured using a computer-readable storage medium device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 14 stores various information related to the learning device 1. The storage unit 14 stores information input via, for example, the input unit 12 or the communication unit 13. The storage unit 14 stores various information generated by, for example, the execution of learning.

[0190] The storage unit 14 stores in advance, for example, a probability distribution used to acquire the occurrence probability of each element of a tensor representing an auxiliary feature. The storage unit 14 stores in advance, for example, a parametrized auxiliary feature cumulative distribution function. The storage unit 14 stores in advance, for example, a parametrized main data feature cumulative distribution function. The storage unit 14 stores in advance, for example, representative vector information. The storage unit 14 stores, for example, the result of hyper-rectangular parallelepiped division.

[0191] The storage unit 14 stores, in advance, for example, the initial values ​​of the parameters of the training network 100. The initial values ​​are, for example, random values. The storage unit 14 stores, for example, the network training results.

[0192] The output unit 15 outputs various types of information. The output unit 15 includes a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display. The output unit 15 may be configured as an interface that connects these display devices to the learning device 1. The output unit 15 outputs, for example, information input to the input unit 12. The output unit 15 may also display, for example, the results of learning.

[0193] 13 is a diagram showing an example of the configuration of the control unit 11 provided in the learning device 1 in this embodiment. The control unit 11 includes a learning unit 10, a memory control unit 120, a communication control unit 130, and an output control unit 140. The memory control unit 120 records various information in the memory unit 14. The communication control unit 130 controls the operation of the communication unit 13. The output control unit 140 controls the operation of the output unit 15.

[0194] 14 is a diagram showing an example of the hardware configuration of an autoencoding device 2 in an embodiment. The autoencoding device 2 includes a control unit 21 having a processor 93 such as a CPU (Central Processing Unit) and a memory 94 connected via a bus, and executes a program. By executing the program, the autoencoding device 2 functions as a device including the control unit 21, an input unit 22, a communication unit 23, a storage unit 24, and an output unit 25.

[0195] More specifically, the processor 93 reads the program stored in the storage unit 24 and stores the read program in the memory 94. When the processor 93 executes the program stored in the memory 94, the autoencoding device 2 functions as a device including a control unit 21, an input unit 22, a communication unit 23, a storage unit 24, and an output unit 25.

[0196] The control unit 21 controls the operation of various functional units included in the self-encoding device 2. The control unit 21 controls, for example, the operation of the output unit 25. The control unit 21 records, in the storage unit 24, various pieces of information generated by encoding by the encoder 200 and decoding by the decoder 212, for example.

[0197] The input unit 22 includes input devices such as a mouse, a keyboard, and a touch panel. The input unit 22 may be configured as an interface that connects these input devices to the autoencoding device 2. The input unit 22 accepts input of various types of information to the autoencoding device 2.

[0198] The communication unit 23 includes a communication interface for connecting the autoencoding device 2 to an external device. The communication unit 23 communicates with the external device via wired or wireless communication. The external device is, for example, a device that is the sender of the autoencoding target. The communication unit 23 acquires the autoencoding target by communicating with the device that is the sender of the autoencoding target. The external device is, for example, the learning device 1. The communication unit 23 receives the network learning result by communicating with the learning device 1. Note that the autoencoding target does not necessarily have to be input via the communication unit 23, but may be input to the input unit 22.

[0199] The storage unit 24 is configured using a computer-readable storage medium device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 24 stores various information related to the self-encoding device 2. The storage unit 24 stores information input via, for example, the input unit 22 or the communication unit 23. The storage unit 24 stores various information generated by, for example, encoding by the encoder 200 and decoding by the decoder 212.

[0200] The storage unit 24 stores, for example, the network learning results, the representative vector information in advance, and the hyper-rectangular parallelepiped division results.

[0201] The output unit 25 outputs various types of information. The output unit 25 includes a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display. The output unit 25 may be configured as an interface that connects these display devices to the learning device 1. The output unit 25 outputs, for example, information input to the input unit 22. The output unit 25 may also output, for example, the result of auto-encoding of the auto-encoding target.

[0202] 15 is a diagram showing an example of the configuration of the control unit 21 included in the auto-encoding device 2 in the embodiment. The control unit 21 includes an auto-encoding execution unit 20, a storage control unit 220, a communication control unit 230, and an output control unit 240. The auto-encoding execution unit 20 performs auto-encoding on the auto-encoding target. The auto-encoding execution unit 20 includes an encoder 200 and a decoder 212. The auto-encoding execution unit 20 performs encoding on the auto-encoding target by using the encoder 200 and decoding by using the decoder 212.

[0203] The storage control unit 220 records various information in the storage unit 24. The communication control unit 230 controls the operation of the communication unit 23. The output control unit 240 controls the operation of the output unit 25.

[0204] The learning device 1 configured in this way performs learning of peripheral processing for vector quantization using representative vector information, which is information indicating the positions of representative vectors arranged in a lattice pattern in a vector space such as LatticeVQ. The learning device 1 then estimates the occurrence probability of the representative vector using the results of the hypercuboid division. As mentioned above, when using representative vectors, the occurrence probability of the representative vector may be estimated using Voronoi division, but integration is not easy.

[0205] Therefore, the learning device 1 that estimates the occurrence probability of a representative vector using a hyper-rectangle obtained by hyper-rectangle division can reduce the burden required for achieving the process of auto-encoding using vector quantization. This reduces the burden from the learning stage required to realize auto-encoding using vector quantization. Therefore, the learning device 1 can reduce the burden required for auto-encoding using vector quantization.

[0206] Furthermore, the learning device 1 configured in this manner uses representative vectors to learn peripheral processing for vector quantization, thereby reducing the burden required for learning representative vectors. Furthermore, the learning device 1 configured in this manner uses representative vectors to learn peripheral processing for vector quantization, thereby eliminating the use of memory that would be required when learning representative vectors. Therefore, the learning device 1 can reduce the frequency with which memory shortages occur and can process main data with larger dimensions.

[0207] As mentioned above, in training vector quantization, it is necessary to add noise instead of quantization. However, when using representative vector information such as LatticeVQ, the range of the Voronoi region becomes a quantization error, so it is not easy to generate noise with a Gaussian probability distribution.

[0208] Meanwhile, the learning device 1 configured in this manner uses, as noise during learning, only samples within the Voronoi region among samples randomly generated within a (K-1)-dimensional sphere circumscribing a Voronoi region in a K-dimensional vector space. This makes it possible to generate noise that follows a Gaussian distribution. Therefore, the learning device 1 can reduce the burden required for performing auto-encoding using vector quantization. Therefore, the learning device 1 can reduce the burden required for auto-encoding using vector quantization.

[0209] The auto-encoding device 2 configured in this manner performs auto-encoding using vector quantization using the results of learning by the learning device 1. This reduces the burden required for auto-encoding using vector quantization.

[0210] (Variation) In adding noise to the main data feature quantity, samples uniformly generated within a (K-1)-dimensional sphere whose volume is approximated to the volume of a Voronoi region in a K-dimensional vector space may be used as noise points. In adding noise to the main data feature quantity, samples uniformly generated within a hypercuboid obtained by hypercuboid division may be used as noise points.

[0211] Therefore, the process of adding noise to the main data feature (i.e., vector noise adding process) may be any one of the first noise adding process, the second noise adding process, and the third noise adding process. The first noise adding process is a process of adding, as noise, samples within a Voronoi region among samples randomly generated within a (K-1)-dimensional sphere circumscribing a Voronoi region in a K-dimensional vector space. In other words, the first noise adding process is the process described using Figures 3 and 4.

[0212] The second noise-adding process adds, as noise, samples uniformly generated within a (K-1)-dimensional sphere whose volume approximates the volume of a Voronoi region in a K-dimensional vector space. The third noise-adding process adds, as noise, samples uniformly generated within a region that divides a vector space in which representative vectors are arranged in a lattice pattern, includes one lattice point in the vector space, and has a shape of a hypercuboid.

[0213] <Another example of the probability estimation process on the main data side> As described above, it is not always easy to obtain a parameterized cumulative distribution function. Therefore, it is not always easy to estimate the occurrence probability of a representative vector representing a main data feature quantity by a main data-side probability estimation process. Therefore, in the above example, the occurrence probability of a representative vector representing a main data feature quantity was obtained by obtaining a parameterized cumulative distribution function using hyper-rectangle division.

[0214] Here, we will explain another example of a process for estimating the occurrence probability of a representative vector representing a main data feature by explaining another example of obtaining a parameterized cumulative distribution function. Specifically, we will explain a main data side probability estimation process (hereinafter referred to as "Monte Carlo occurrence probability estimation process") that estimates the occurrence probability of a representative vector representing a main data feature by using Monte Carlo integration.

[0215] For ease of explanation, the position in the lattice space is represented by x, and the probability density function at the position x in the lattice space is represented by f(x). The probability density function f(x) is a parameterized probability density function as described above. In this case, the cumulative distribution function in the region V in the lattice space is expressed by the following equation (13).

[0216]

number

[0217] Using Monte Carlo integration, equation (13) can be approximated by the following equation (14). The right-hand side of equation (14) is the Monte Carlo integral that expresses equation (13). In other words, the Monte Carlo integral on the right-hand side of equation (14) represents the integral of the probability density function f(x) in lattice space.

[0218]

number

[0219] q(x i ) is a probability that follows a uniform distribution and is used in the Monte Carlo method to determine the position x in the lattice space. i is the probability that q(x i The value of (14) is equal to the reciprocal of the size of the region V. In equation (14), i is a position x in the lattice space, and is an identifier that identifies each sampled position x. The sum of the right side of equation (14) is the number of sampled positions x in the region V by the Monte Carlo method. i From the expressions on the right and left sides of equation (14), it is clear that the error between the right and left sides of equation (14) converges to 0 as N → infinity.

[0220] When region V is one Voronoi region, the value of the cumulative distribution function of formula (13) is the occurrence probability of the representative vector belonging to the Voronoi region indicated by region V. Therefore, in such a case, the value of the Monte Carlo integral on the right side of formula (14) is the occurrence probability of the representative vector belonging to the Voronoi region indicated by region V. Note that the cumulative distribution function is the result of integrating the parametrized probability density function as shown in formula (13).

[0221] A more specific process of the Monte Carlo type occurrence probability estimation process will be described. In the Monte Carlo type occurrence probability estimation process, first, the position x that indicates the average of the probability density function f(x) among the positions x in the lattice space is selected. i The mean position is, in other words, the center of the distribution of the probability density expressed by the probability density function f(x). Hereinafter, the position x that indicates the mean of the probability density function f(x) is i is called the average position. In the Monte Carlo occurrence probability estimation process, next, the value of the Monte Carlo integral expressed by equation (14) is obtained in the Voronoi region (hereinafter referred to as the "initial region") that includes the estimated average position.

[0222] Next, in the Monte Carlo occurrence probability estimation process, the occurrence probabilities are added up in order from the representative vector closest to the average position until a predetermined termination condition (hereinafter referred to as the "addition termination condition") is met. As described above, from equations (13) and (14), the occurrence probability of each representative vector is the Monte Carlo integral in the Voronoi region to which each representative vector belongs. Therefore, the value of the Monte Carlo integral in the initial region is the occurrence probability of the representative vector belonging to the initial region.

[0223] Furthermore, since the distance between adjacent representative vectors is the same as the distance between adjacent Voronoi regions, adding up the occurrence probabilities of representative vectors in order of proximity to the mean position means adding up the values ​​of the Monte Carlo integrals in order of proximity to the mean position. The distance between Voronoi regions is, for example, the distance between the centers of the Voronoi regions.

[0224] The addition termination condition is, for example, a condition that the first sub-condition is satisfied. The first sub-condition is a condition that the result of adding up the occurrence probabilities is greater than 1. The addition termination condition may be, for example, a condition that the second sub-condition is satisfied. The second sub-condition is a condition that the distance between the average position and the representative vector is longer than the distance between the average position and a position in the lattice space that represents the main data feature (hereinafter referred to as the "main lattice space position"). The addition termination condition may be a condition that either the first sub-condition or the second sub-condition is satisfied.

[0225] The main lattice space position is not necessarily the position of the representative vector. The main lattice space position is a mathematical model that is updated through learning, and is the result of estimation by the mathematical model that estimates the position in lattice space from the main data feature amount.

[0226] The determination as to whether the addition end condition is satisfied is made, for example, by the main data side probability estimation unit 109. The determination as to whether the addition end condition is satisfied is made, for example, every time the occurrence probability of one representative vector is added.

[0227] If the first sub-condition is satisfied, the Monte Carlo occurrence probability estimation process next acquires the occurrence probability of the nearest representative vector as the occurrence probability of the representative vector representing the main data feature. The nearest representative vector is the representative vector closest to the main lattice space position that is included in a Voronoi region that does not include the main lattice space position, among the representative vectors whose occurrence probabilities have been added. Furthermore, the Monte Carlo occurrence probability estimation process determines that the nearest representative vector is the representative vector representing the main data feature.

[0228] By definition, the nearest representative vector is a representative vector that belongs to the Voronoi region (hereinafter referred to as the "nearest Voronoi region") that is closest to the position of the vector indicating the main data feature among the Voronoi regions obtained by adding up the Monte Carlo integral values ​​of the Voronoi regions in order of closest to the average position and the result is greater than 1. Therefore, when the first sub-condition is satisfied, in the Monte Carlo occurrence probability estimation process, the value of the Monte Carlo integral of the nearest Voronoi region is estimated as the occurrence probability of the vector indicating the main data feature.

[0229] If the second sub-condition is satisfied, the Monte Carlo occurrence probability estimation process next estimates the occurrence probability of the representative vector of the Voronoi region including the primary lattice spatial position as the occurrence probability of the representative vector representing the primary data feature. Furthermore, the Monte Carlo occurrence probability estimation process determines that the representative vector of the Voronoi region including the primary lattice spatial position is the representative vector representing the primary data feature.

[0230] In this way, the learning device 1, which estimates the occurrence probability of a representative vector representing a main data feature using a Monte Carlo-type occurrence probability estimation process, estimates the occurrence probability of a representative vector representing a main data feature using Monte Carlo integration. Incidentally, when performing auto-encoding using hypercuboid division, the Voronoi region is approximated by a hypercuboid. However, such approximation is not necessarily accurate. As a result, if the accuracy of the hypercuboid approximation is poor, it may become necessary to perform calculations without using the approximation, which increases the burden.

[0231] On the other hand, when Monte Carlo integration is used, as shown in equation (14), the accuracy of approximation to the cumulative distribution function is high and does not depend on the shape of the Voronoi region. Therefore, the learning device 1 that estimates the occurrence probability of the representative vector representing the main data feature using Monte Carlo-type occurrence probability estimation processing can reduce the burden required to obtain the process of auto-encoding using vector quantization. This reduces the burden from the learning stage required to realize auto-encoding using vector quantization. Therefore, the learning device 1 can reduce the burden required for auto-encoding using vector quantization.

[0232] Furthermore, the auto-encoding device 2 configured in this manner performs auto-encoding using vector quantization using the results of learning by the learning device 1. This reduces the burden required for auto-encoding using vector quantization.

[0233] (Another example of a learning network 100) The learning unit 10 may include a learning network 100a instead of the learning network 100. In such a case, the optimization unit 113 updates the learning network 100a instead of the learning network 100 based on the output of the learning network 100a. Hereinafter, functional units having the same functions as those in the learning network 100 are denoted by the same reference numerals as those in FIG. 1, and their explanations will be omitted.

[0234] 16 is an explanatory diagram illustrating an example of the configuration of a learning network 100a in a modified example. Learning network 100a differs from learning network 100 in that it includes an STE execution unit 114 and a main-data-side decoding unit 111a instead of main-data-side decoding unit 111. Learning network 100a also differs from learning network 100 in that main-data-side noise addition unit 104 of learning network 100a is not output to main-data-side decoding unit 111a.

[0235] The STE execution unit 114 executes a Straight Through Estimator (STE). The STE is a mathematical model that performs vector quantization and has a gradient fixed to 1 during error backpropagation. The target of the STE execution by the STE execution unit 114 is the execution result of the main data side encoding unit 102. Specifically, the execution result of the main data side encoding unit 102 is the main data feature.

[0236] The main data-side decoding unit 111a executes a main data feature decoding process. The main data-side decoding unit 111a differs from the main data-side decoding unit 111 in that the main data feature decoding process is executed on the result of execution by the STE execution unit 114, not on the result of execution by the main data noise addition unit 104.

[0237] 17 is a flowchart showing an example of the flow of processing executed by the learning unit 10 in the modified example. To simplify the following explanation, the same processes as those executed by the learning unit 10 in the embodiment are assigned the same reference numerals as in FIG. 7, and the explanation thereof will be omitted.

[0238] After the processes from step S101 to step S103 are performed, the STE execution unit 114 executes STE on the main data feature (step S115). After step S115, step S104 is executed. After the processes from step S104 to step S110 are performed, the processes from step S112 to step S114 are executed.

[0239] The process of step S115 may be performed at any timing as long as it is performed after step S102 and before step S112.

[0240] <Effects of the learning network 100a> The effects of the learning network 100a will be described. As described above, during autoencoding (i.e., during inference) by the autoencoding device 2, vector quantization is performed without adding noise to the encoding target. The learning network 100, which is a mathematical model used during learning, includes an STE execution unit 114.

[0241] The encoding performed by the STE execution unit 114 is encoding that quantizes the encoding target without adding noise. Therefore, the learning of the learning network 100a is performed by the same encoding as the autoencoding during inference.

[0242] However, STE is a mathematical model that fixes the gradient at 1 during backward processing. As a result, the distribution of quantization errors may differ between training and inference, and the accuracy of the probability distribution estimation may decrease due to overfitting. In other words, because STE is a process that does not add variance, it is prone to overfitting.

[0243] To prevent this situation from occurring, the learning network 100 uses noise-added values ​​to estimate probability distributions during learning. Specifically, the main data probability estimation unit 109 uses the results of the main data noise addition unit 104 to estimate the occurrence probability of representative vectors representing the main data features.

[0244] As a result, the learning network 100a can suppress a decrease in the accuracy of autoencoding due to differences in processing between learning and estimation, and can also suppress overlearning by using noise.

[0245] Therefore, the modified learning device 1 including the learning network 100a configured in this way can simultaneously suppress a decrease in the accuracy of autoencoding and the occurrence of overlearning.

[0246] Furthermore, the modified learning device 1 equipped with the learning network 100a configured in this manner can suppress a decrease in the accuracy of autoencoding due to differences in processing between learning and estimation, and can also suppress overlearning by using noise. Therefore, by using the learning network 100a, it is possible to suppress a decrease in the accuracy of autoencoding while reducing the effort required to generate training data, such as preparing training data with large variations in order to suppress overlearning.

[0247] Therefore, the modified learning device 1 including the learning network 100a can reduce the burden required to achieve the process of auto-encoding using vector quantization. This means that the burden from the learning stage required to achieve auto-encoding using vector quantization is reduced. Therefore, the modified learning device 1 including the learning network 100a can reduce the burden required for auto-encoding using vector quantization.

[0248] Furthermore, the autoencoding device 2 that uses the learning results of the modified learning device 1 configured in this way performs autoencoding using a mathematical model that suppresses a decrease in autoencoding accuracy and that suppresses the occurrence of overlearning, thereby reducing the burden required for autoencoding using vector quantization.

[0249] The learning device 1 may be implemented using multiple information processing devices connected to each other via a network, in which case the functional units of the learning device 1 may be distributed and implemented across the multiple information processing devices.

[0250] The self-encoding device 2 may be implemented using a plurality of information processing devices communicably connected via a network. In this case, the functional units of the self-encoding device 2 may be distributed and implemented in the plurality of information processing devices.

[0251] All or part of the functions of the learning device 1 and the autoencoding device 2 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The program may be transmitted via a telecommunications line.

[0252] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0253] 1...Learning device, 10...Learning unit, 100, 100a...Learning network, 101...Main data acquisition unit, 102...Main data side encoding unit, 103...Auxiliary data side encoding unit, 104...Main data side noise adding unit, 105...Auxiliary data side noise adding unit, 106...Auxiliary data side probability estimating unit, 107...Auxiliary data side decoding unit, 108...Auxiliary entropy acquisition unit, 109...Main data side probability estimating unit, 110...Main entropy acquisition unit, 111, 111a...Main data side decoding unit, 112...Reconstruction error calculation unit, 113...Optimization unit, 114...STE execution unit, 2...Auto-encoding device, 200...Encoder, 201...Auto-encoding target acquisition unit, 202...Learned main data side encoding unit, 203...Learned auxiliary data side encoding unit, 204...Vector quantization unit, 205...scalar quantization unit, 206...learned auxiliary data side probability estimation unit, 207...learned auxiliary data side decoding unit, 208...auxiliary entropy coding unit, 209...main data side probability estimation unit, 210...main entropy coding unit, 211...data multiplexing unit, 213...encoded data acquisition unit, 214...data separation unit, 215...auxiliary entropy decoding unit, 216...learned auxiliary data side decoding unit, 217...main entropy decoding unit, 218...learned main data side decoding unit, 11...control unit, 12...input unit, 13...communication unit, 14...storage unit, 15...output unit, 120...storage control unit, 130...communication control unit, 140...output control unit, 21...control unit, 22...input unit, 23...communication unit, 24...storage unit, 25...output unit, 20...self-encoding execution unit, 220...storage control unit, 230...communication control unit, 240...output control unit, 91...processor, 92...memory, 93...processor, 94...memory< / latticevq>

Claims

1. a learning unit that updates, through learning, encoding and decoding processes in the auto-encoding process using vector quantization, the auto-encoding process using main data features that are features to be auto-encoded and auxiliary features that are features of the main data features, and that performs vector quantization of the main data features and scalar quantization of the auxiliary features; Equipped with In the learning, the learning unit executes a straight-through estimator on the main data feature quantity and a process of adding noise to the main data feature quantity. Learning device.

2. The vector quantization is LatticeVQ. The learning device according to claim 1 .

3. the learning unit further executes a main data side probability estimation process in the learning to estimate an occurrence probability of a vector indicating the main data feature amount; the main data-side probability estimation process is a Monte Carlo integral value representing an integral of a parameterized probability density function in a vector space in which representative vectors are arranged in a lattice pattern, the Monte Carlo integral value being a result of adding up values ​​of the Monte Carlo integrals of Voronoi regions in the vector space, and when a result of adding up the Monte Carlo integral values ​​of Voronoi regions in order of proximity to an average position, which is a position in the vector space representing an average of the probability density function, is greater than 1, the Monte Carlo integral value of the Voronoi region closest to the position of the vector representing the main data feature amount among the added Voronoi regions is estimated as the occurrence probability of the vector representing the main data feature amount; The learning device according to claim 1 .

4. The noise adding process includes: A process of adding, as noise, samples within a Voronoi region among samples randomly generated within a (K-1)-dimensional sphere circumscribing the Voronoi region in a K-dimensional (K is an integer equal to or greater than 2) vector space; A process of adding samples uniformly generated within a (K-1)-dimensional sphere whose volume is approximated to the volume of a Voronoi region in a K-dimensional vector space as noise; A process of adding samples uniformly generated within a region that divides a vector space in which representative vectors are arranged in a lattice pattern, that includes one lattice point of the vector space, and that has a shape of a hypercuboid, as noise; Either one of the following: The learning device according to claim 1 .

5. a learning step of updating encoding and decoding processes by learning, the self-encoding process using vector quantization, which uses main data features that are features to be self-encoded and auxiliary features that are features of the main data features, and which performs vector quantization of the main data features and scalar quantization of the auxiliary features; and the learning step includes, in the learning, executing a straight-through estimator on the main data feature quantity and performing a process of adding noise to the main data feature quantity. How to learn.

6. A program for causing a computer to function as the learning device according to any one of claims 1 to 4.

Images