Information processing device and method

Abstract

The present disclosure relates to an information processing device and method which make it possible to suppress a decrease in encoding efficiency. The present invention uses a predetermined quantization step to quantize a depth value that wraps around in a predetermined range. A difference value between the quantized depth value and a prediction value of the depth value is derived, and the quantization step is used to quantize a wrap-around value of the depth value. The quantized wrap-around value is used to correct the derived difference value as appropriate, and the difference value which has been corrected as appropriate is encoded. The present disclosure can be applied to, for example, an information processing device, an image processing device, an electronic device, an information processing method, an image processing method, or a program.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method that can suppress a decrease in coding efficiency. [Background technology]

[0002] In recent years, the use of image sensors for sensing purposes has been increasing. For example, a technology has been devised to measure the distance to a target object by detecting the phase difference between a sine wave emitted from a light source and a reflected wave (see, for example, Non-Patent Document 1).

[0003] In this distance measurement method, distance measurement using light of multiple modulation frequencies can achieve more accurate distance measurement without reducing the maximum measurable distance. When distance measurement is performed using multiple modulation frequencies in this way, the distance measurement result (depth value) using light of a high modulation frequency wraps around within a predetermined value range.

[0004] As a method for encoding such depth values, for example, there is a method such as DPCM (Differential Pulse Code Modulation) that derives the difference between the depth value to be processed and its predicted value, and encodes the difference value. Meanwhile, there has conventionally been a method that uses wraparound to suppress a decrease in data encoding efficiency (see, for example, Patent Documents 1 and 2). [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Keita Yasutomi, Shoji Kawato, "Technical Explanation of Time-of-Flight Camera", Journal of the Institute of Image Information and Television Engineers, Vol. 70, No. 11, 2016, pp. 880-885, received August 29, 2016 [Patent documents]

[0006] [Patent Document 1] Patent No. 6045123 [Patent Document 2] Japanese Patent Application Laid-Open No. 2002-344477 Summary of the Invention [Problem to be solved by the invention]

[0007] However, when encoding depth values ​​that wrap around as described above, in a method of deriving the difference between a depth value and its predicted value as described above and encoding the difference value, the difference value increases due to the depth value wrapping around, which may result in a decrease in encoding efficiency.In contrast, the methods described in Patent Documents 1 and 2 wrap around the difference value or encoded data to fit within a predetermined value range, making it difficult to suppress a decrease in encoding efficiency of the wraparound data.

[0008] The present disclosure has been made in light of such circumstances, and makes it possible to suppress a decrease in coding efficiency. [Means for solving the problem]

[0009] An information processing device according to one aspect of the present technology is an information processing device including: a first quantization unit that quantizes depth values ​​that wrap around within a predetermined value range using a predetermined quantization step; a differential value derivation unit that derives a differential value between the depth value quantized by the first quantization unit and a predicted value of the depth value; a second quantization unit that quantizes a folding value of the depth value using the quantization step; a correction unit that appropriately corrects the differential value derived by the differential value derivation unit using the folding value quantized by the second quantization unit; and an encoding unit that encodes the differential value appropriately corrected by the correction unit.

[0010] An information processing method according to one aspect of the present technology is an information processing method that quantizes depth values ​​that wrap around within a predetermined value range using a predetermined quantization step, derives a difference value between the quantized depth value and a predicted value of the depth value, quantizes a folding value of the depth value using the quantization step, appropriately corrects the derived difference value using the quantized folding value, and encodes the appropriately corrected difference value.

[0011] According to another aspect of the present technology, an information processing device includes: a decoding unit that decodes encoded data to derive a difference value between coefficient data relating to a depth value that wraps around in a predetermined value range and a predicted value of the coefficient data; a coefficient data derivation unit that derives the coefficient data by adding the predicted value to the difference value derived by the decoding unit; a quantization unit that quantizes a folding value of the depth value using a predetermined quantization step; a correction unit that appropriately corrects the coefficient data derived by the coefficient data derivation unit using the folding value quantized by the quantization unit; and an inverse quantization unit that derives the depth value by inverse quantizing the coefficient data appropriately corrected by the correction unit using the quantization step.

[0012] An information processing method according to another aspect of the present technology is an information processing method that, by decoding encoded data, derives a difference value between coefficient data relating to a depth value that wraps around in a predetermined value range and a predicted value of the coefficient data, derives the coefficient data by adding the predicted value to the derived difference value, quantizes a folding value of the depth value using a predetermined quantization step, appropriately corrects the derived coefficient data using the quantized folding value, and dequantizes the appropriately corrected coefficient data using the quantization step, thereby deriving the depth value.

[0013] In an information processing device and method according to one aspect of the present technology, a depth value that wraps around within a predetermined value range is quantized using a predetermined quantization step, a difference value between the quantized depth value and a predicted value of that depth value is derived, a folding value of the depth value is quantized using the quantization step, the derived difference value is appropriately corrected using the quantized folding value, and the appropriately corrected difference value is encoded.

[0014] In an information processing device and method according to another aspect of the present technology, encoded data is decoded to derive a difference value between coefficient data relating to a depth value that wraps around within a predetermined value range and a predicted value of that coefficient data, the predicted value is added to the derived difference value to derive coefficient data, the folding value of the depth value is quantized using a predetermined quantization step, the derived coefficient data is appropriately corrected using the quantized folding value, and the appropriately corrected coefficient data is inverse-quantized using the quantization step to derive a depth value. [Brief explanation of the drawings]

[0015] [Figure 1] 1A and 1B are diagrams illustrating an example of distance measurement using an indirect ToF method. [Figure 2] 1A and 1B are diagrams illustrating an example of distance measurement using an indirect ToF method. [Figure 3] 1A and 1B are diagrams illustrating an example of distance measurement using an indirect ToF method. [Figure 4] 1A and 1B are diagrams illustrating an example of distance measurement using an indirect ToF method. [Figure 5] 10A and 10B are diagrams illustrating an example of distance measurement by an indirect ToF method using multiple modulation frequencies. [Figure 6] 10A and 10B are diagrams illustrating an example of distance measurement by an indirect ToF method using multiple modulation frequencies. [Figure 7] FIG. 10 is a diagram illustrating an example of measurement values. [Figure 8] FIG. 10 is a diagram illustrating an example of measurement values. [Figure 9] FIG. 1 is a block diagram illustrating an example of the main configuration of an encoding device. [Figure 10] FIG. 10 is a diagram illustrating a wraparound value. [Figure 11] 10A and 10B are diagrams illustrating an example of how a difference value is corrected. [Figure 12] 10 is a flowchart illustrating an example of the flow of an encoding process. [Figure 13] FIG. 2 is a block diagram illustrating an example of the main configuration of a decoding device. [Figure 14] FIG. 10 is a diagram illustrating an example of how decoded data is corrected. [Figure 15] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 16] FIG. 1 is a block diagram illustrating an example of the main configuration of an encoding device. [Figure 17] 10 is a flowchart illustrating an example of the flow of an encoding process. [Figure 18] FIG. 2 is a block diagram illustrating an example of the main configuration of a decoding device. [Figure 19] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 20] FIG. 1 is a block diagram illustrating an example of the main configuration of a distance measuring device. [Figure 21] 10 is a flowchart illustrating an example of the flow of a distance measurement process. [Figure 22] 10A and 10B are diagrams illustrating an example of distance measurement by an indirect ToF method using multiple modulation frequencies. [Figure 23] FIG. 1 is a block diagram illustrating an example of the main configuration of a distance measuring device. [Figure 24] FIG. 10 is a diagram illustrating an example of how a return value is supplied. [Figure 25] FIG. 10 is a diagram illustrating an example of how a return value is supplied. [Figure 26] 10 is a flowchart illustrating an example of the flow of a distance measurement process. [Figure 27] FIG. 1 is a diagram illustrating P2 prediction. [Figure 28] FIG. 1 is a block diagram illustrating an example of the main configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1. Encoding and decoding wraparound data 2. First embodiment (encoding device) 3. Second embodiment (decoding device) 4. Third embodiment (encoding device) 5. Fourth embodiment (decoding device) 6. Fifth embodiment (range measuring device) 7. Sixth embodiment (range measuring device) 8. Notes

[0017] <1. Encoding and decoding of wraparound data> <itof> In recent years, the use of image sensors for sensing purposes has been increasing. For example, as described in Non-Patent Document 1, a technology has been devised to measure the distance to a target object by detecting the phase difference between a sine wave emitted from a light source and a reflected wave.

[0018] For example, as shown in Fig. 1, ToF (Time-of-Flight) has been considered, in which light (e.g., infrared light) is emitted from a light source 1 toward a subject 3, the time it takes for the reflected light to be received by a distance measuring sensor 2 is measured, and the distance to the subject 3 is derived based on that time. In addition, two ToF methods have been considered: a direct ToF method (also called dToF (direct Time of Flight)) and an indirect ToF method (also called iToF (indirect Time of Flight)).

[0019] In the case of the direct ToF method, it is difficult to increase the number of pixels because a TDC (Time-to-Digital Converter) is used. In the case of the indirect ToF method, a time calculation circuit such as a TDC is not required within the pixel, and an increase in the number of elements within the pixel can be suppressed. Therefore, it is easy to achieve an increase in the number of pixels.

[0020] In the indirect ToF method, the photocharge generated by the photodiode is modulated by a lock-in pixel using a time window (clock) synchronized with the light source. At this time, the time information is reflected in the signal amount, so the time of flight can be derived.

[0021] Modulation methods used in indirect ToF methods include, for example, continuous wave modulation and pulse wave modulation. An example of continuous wave modulation is shown in Figure 2. In the graph shown in Figure 2, sine wave 11 indicates emitted light, and sine wave 12 indicates reflected light. As shown in Figure 2, in the case of continuous wave modulation, the phase difference between emitted light and reflected light can be derived by performing lock-in detection using four time windows. Lock-in detection used here refers to the operation of accumulating signals of the same phase multiple times using a short electronic shutter synchronized with the light source. When modulated with a sine wave, the phase difference φ TOF is derived as shown in the following equation (1) using signals A0, A1, A2, and A3 accumulated in four time windows TW1, TW2, TW3, and TW4.

[0022] TIFF0007802753000001.tif2357

[0023] Modulation frequency f m Since is known, the phase (φ TOF ) to time (t TOF ) can be converted into

[0024] TIFF0007802753000002.tif2328

[0025] The received light contains a DC component other than that of the light source, i.e., a background light component, but this background light component is canceled out by the calculation of the above equation (1). Therefore, as long as the sensor is not saturated, distance can be estimated without being affected by background light.

[0026] In contrast, an example of pulse wave modulation is shown in Figure 3. In the case of pulse wave modulation, if the signals accumulated by TW1 and TW2 are A0 and A1, respectively, the time of flight t TOF is derived as shown in the following equation (3).

[0027] TIFF0007802753000003.tif2439

[0028] TD is an emission window that emits excess background light. By setting the number of time windows TW to three or more, it is possible to know only the background light component, making it possible to estimate distance without being affected by background light. With pulse wave modulation, by setting a high duty ratio for the light source, it is possible to achieve imaging that is robust against background light.

[0029] <Depth value derivation device> 4 is a block diagram showing an example of the main configuration of a depth value derivation device that derives depth values ​​using the above-described indirect ToF method. The depth value derivation device 30 shown in FIG. 4 includes a sensor 31, an i signal generator 32, a q signal generator 33, a phase difference detector 34, and a depth calculator 35.

[0030] The sensor 31 receives light and measures the intensity of the received light by combining shutters of 1 / 4 cycle of the modulated wave. The sensor 31 then supplies data of these four sections (N1, N2, N3, N4) to the i signal generation unit 32 and the q signal generation unit 33. The i signal generation unit 32 generates an i (In-Phase) signal by subtracting data N1 from data N3. The i signal generation unit 32 supplies the i signal to the phase difference detection unit 34. The q signal generation unit 33 generates a q (Quadrature) signal by subtracting data N4 from data N2. The q signal generation unit 33 supplies the q signal to the phase difference detection unit 34.

[0031] The phase difference detection unit 34 uses the supplied i and q signals to derive the phase difference φ as shown in the following equation (4).

[0032] TIFF0007802753000004.tif2775

[0033] The phase difference detection unit 34 supplies the derived phase difference φ to the depth calculation unit 35 .

[0034] The depth calculation unit 35 derives a depth value (depth) d from the phase difference φ as shown in the following equation (5).

[0035] TIFF0007802753000005.tif2531

[0036] In this manner, the depth value d can be derived from the sensor output.

[0037] <iToF with multiple modulation frequencies> In iToF, increasing the modulation frequency of the irradiated light enables more accurate distance measurement. However, in this case, the wavelength becomes shorter, which narrows the measurable range. For example, assume that it is possible to measure range X from a point at distance X1 from the sensor to a point at distance X2 from the sensor. In this case, by increasing the modulation frequency, it is possible to measure distances within range X with higher accuracy, but the length of range X becomes shorter (the difference between distance X1 and distance X2 becomes smaller). Conversely, lowering the modulation frequency reduces the accuracy of distance measurement, but the length of range X becomes longer (the difference between distance X1 and distance X2 becomes larger).

[0038] Therefore, by combining iToF with multiple modulation frequencies, it is possible to improve the accuracy of ranging while preventing the measurable range from being narrowed (while preventing the length of range X from becoming shorter) (Dual Frequency iToF). For example, by merging the depth value derived by iToF with a low modulation frequency as the most significant bit, the depth value derived by iToF with a low modulation frequency as the most significant bit, and the depth value derived by iToF with a high modulation frequency as the least significant bit, it is possible to obtain a more accurate depth value with a wider range of possible values.

[0039] Such a method (Dual Frequency iToF) can be realized by preparing a device (light source and sensor) for each modulation frequency, and measuring distances using irradiated light of the respective modulation frequency on each device, as shown in Figure 5.

[0040] Furthermore, as shown in the example of Figure 6, it is also possible to prepare a single device (light source and sensor) and use it in a time-sharing manner to achieve distance measurement using light of each modulation frequency. That is, at one timing, distance measurement is performed using irradiated light with a low modulation frequency, and at another timing, distance measurement is performed using irradiated light with a high modulation frequency. By switching the modulation frequency along the time axis and repeating distance measurement in this way, distance measurement using irradiated light of each modulation frequency can be achieved. In this case, the number of components can be reduced compared to the example of Figure 5. For example, it can be achieved with a single chip.

[0041] When using one device in a time-sharing manner like this, in order to merge the distance measurement results of each modulation frequency, it is necessary to store the distance measurement result (depth value) data of at least one modulation frequency in a memory, etc. When storing the distance measurement result data in memory, the data can be encoded (compressed) to prevent an increase in the memory capacity required for storage.

[0042] While any method for encoding such data may be used, the more efficient the encoding and the less delay, the better. For example, a method such as DPCM (Differential Pulse Code Modulation) can be used, in which the difference between samples is derived and the difference is encoded.

[0043] <Depth Value Encoding> Therefore, let us consider encoding of depth values. For example, when combining iToFs with multiple modulation frequencies as described above, the relationship between the depth value (measurement value) obtained by distance measurement using illumination light with the lowest modulation frequency and the actual distance (real distance) from the sensor to the object being measured is as shown in A of Figure 7. In other words, there is a one-to-one correspondence between the measurement value and the actual distance. In contrast, the relationship between the measurement value obtained by distance measurement using illumination light with a higher modulation frequency and the actual distance is as shown in B of Figure 7. In other words, the measurement value wraps around (folds back) within a certain value range relative to the actual distance. This is because the range in which distance can be measured is narrower than the range in which the actual distance can be measured.

[0044] For example, as shown in A of FIG. 8, suppose that distance measurement is performed from a sensor 41 to an object 42 using a combination of iToF with a low modulation frequency and iToF with a high modulation frequency. The sensor 41 has multiple pixels, like a so-called image sensor, and receives reflected light or the like at each pixel. That is, a depth value for each pixel of the sensor 41 is obtained as the distance measurement result. The range measurable by the sensor 41 in this distance measurement is indicated by a double-headed arrow 43. That is, the range measurable by iToF with a low modulation frequency is indicated by the double-headed arrow 43.

[0045] 8A, the object 42 is a rectangle with its longitudinal direction extending in the depth direction as viewed from the sensor 41. In other words, the object 42 exists from close to far away from the sensor 41. Therefore, the actual distance from the sensor 41 to the surface of the object 42 changes smoothly along the longitudinal direction of the object 42 from close to far away from the sensor 41.

[0046] The depth image 44 shown in FIG. 8B shows the distance measurement results (i.e., the depth value of each pixel) of the iToF with the low modulation frequency. The depth image 44 indicates the depth value by color, with whiter indicating a farther distance (larger depth value) and darker indicating a closer distance (smaller depth value). As shown in FIG. 8B, in this depth image 44, the depth values ​​of pixels corresponding to the surface of an object 42 whose actual distance changes gradually change gradually along the longitudinal direction of the object 42, just like the actual distance. Therefore, by taking the difference with neighboring pixels as in DPCM during encoding and encoding the difference value, it is possible to improve encoding efficiency by utilizing the correlation of depth values ​​between pixels.

[0047] The depth image 45 shown in FIG. 8C shows the distance measurement results (i.e., the depth value of each pixel) of the iToF with a high modulation frequency. As with the depth image 44, the depth values ​​are represented by color in the depth image 45. As shown in FIG. 7B, in the case of the iToF with a high modulation frequency, the measured values ​​wrap around within a predetermined range. Therefore, even in areas where the actual distance changes smoothly in the depth image 45, the depth values ​​change significantly in areas where the depth values ​​fold over. For example, even if the actual distances of the pixels above and below the dotted line 45A are approximately the same, the depth values ​​differ significantly due to foldover. Therefore, if the difference between adjacent pixels is calculated and the difference value is encoded as in DPCM, the difference value may become large in areas where the depth values ​​fold over. In other words, in the case of depth values ​​that wrap around, the correlation with adjacent pixels decreases, which may result in reduced encoding efficiency.

[0048] In addition, Patent Documents 1 and 2 disclose methods for suppressing a decrease in encoding efficiency by wrapping around differential values ​​and encoded data to keep them within a specified value range, but these methods make it difficult to suppress a decrease in encoding efficiency for data that wraps around, such as the depth values ​​mentioned above.

[0049] <Correction of difference values> Therefore, the differential value is appropriately corrected before encoding. For example, in an information processing method, a predetermined quantization step is used to quantize a depth value that wraps around in a predetermined value range, a differential value between the quantized depth value and a predicted value of the depth value is derived, the foldback value of the depth value is quantized using the above-mentioned quantization step, the quantized foldback value is used to appropriately correct the derived differential value, and the appropriately corrected differential value is encoded.

[0050] For example, an information processing device may include a first quantization unit that quantizes depth values ​​that wrap around within a predetermined value range using a predetermined quantization step, a differential value derivation unit that derives a differential value between the depth value quantized by the first quantization unit and a predicted value of that depth value, a second quantization unit that quantizes a folding value of the depth value using the above-mentioned quantization step, a correction unit that appropriately corrects the differential value derived by the differential value derivation unit using the folding value quantized by the second quantization unit, and an encoding unit that encodes the differential value appropriately corrected by the correction unit.

[0051] In this way, by appropriately correcting the difference value using the folding value of the depth value, it is possible to suppress an increase in the difference value due to folding of the depth value, and therefore to suppress a decrease in coding efficiency.

[0052] It should be noted that the difference value can be reduced by quantizing the depth value. However, simply quantizing the depth value makes it difficult to correct the difference value because it is no longer possible to establish correspondence with the foldback value. Therefore, the foldback value is also quantized using the same quantization step as the depth value. In this way, it is possible to maintain the correspondence between the depth value and the foldback value, and it is possible to correct the difference value. Therefore, it is possible to further suppress an increase in the difference value and further suppress a decrease in coding efficiency.

[0053] <Coefficient data correction> The coded data of the depth values ​​(difference values) described above is decoded and used. When the coded data is decoded, the coefficient data (quantized depth values) obtained by decoding the coded data are appropriately corrected.

[0054] For example, in an information processing method, coded data is decoded to derive a difference value between coefficient data relating to a depth value that wraps around within a predetermined value range and a predicted value of that coefficient data, the coefficient data is derived by adding the predicted value to the derived difference value, the folding value of the depth value is quantized using a predetermined quantization step, the derived coefficient data is appropriately corrected using the quantized folding value, and the corrected coefficient data is inversely quantized using the above-mentioned quantization step to derive a depth value.

[0055] For example, an information processing device may include a decoding unit that decodes encoded data to derive a difference value between coefficient data relating to a depth value that wraps around in a predetermined value range and a predicted value of that coefficient data, a coefficient data derivation unit that derives coefficient data by adding the predicted value to the difference value derived by the decoding unit, a quantization unit that quantizes a folding value of the depth value using a predetermined quantization step, a correction unit that appropriately corrects the coefficient data derived by the coefficient data derivation unit using the folding value quantized by the quantization unit, and an inverse quantization unit that derives a depth value by inverse quantizing the coefficient data appropriately corrected by the correction unit using the above-mentioned quantization step.

[0056] In this way, by appropriately correcting coefficient data obtained by decoding encoded data using the folding value of the depth value, it is possible to appropriately decode the encoded data and obtain depth values. In other words, it is possible to suppress an increase in difference values ​​due to folding of depth values, and to suppress a decrease in encoding efficiency.

[0057] In addition, when depth values ​​are quantized and encoded, the coefficient data obtained by decoding the encoded data or the like cannot correspond to the fold-over values ​​of the depth values, making it difficult to correct the coefficient data using the fold-over values ​​of the depth values. Therefore, the fold-over values ​​of the depth values ​​are quantized using a quantization step similar to the quantization step used for inverse quantization of the coefficient data. In this way, the correspondence between the coefficient data and the fold-over values ​​can be maintained, and the correction of the coefficient data can be achieved. Therefore, it is possible to suppress the increase in the difference value and further suppress the decrease in encoding efficiency.

[0058] 2. First Embodiment <Encoding device> Fig. 9 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an information processing device to which the present technology is applied. The encoding device 100 shown in Fig. 9 is a device that encodes depth data (depth images, depth values) obtained by iToF. Note that Fig. 9 shows main processing units, data flows, etc., and does not necessarily show all of them. In other words, in this encoding device 100, there may be processing units that are not shown as blocks in Fig. 9, or there may be processing and data flows that are not shown as arrows, etc. in Fig. 9.

[0059] 9, the encoding device 100 includes a quantization unit 111, a quantization unit 112, a DPCM processing unit 113, an arithmetic unit 114, an arithmetic unit 115, a selection control unit 116, a selection unit 117, and an encoding unit 118. The arithmetic unit 114, the arithmetic unit 115, the selection control unit 116, and the selection unit 117 configure a correction unit 121.

[0060] The quantization unit 111 acquires a value at which the wrap-around depth value turns around. This turn-around value is the upper limit value of the wrap-around depth value (measured value), as shown in FIG. 10, for example. In other words, the wrap-around depth value (measured value) takes a value smaller than this turn-around value. In the example of FIG. 10, "1500" is the turn-around value. This turn-around value is supplied from outside the encoding device 100. For example, this turn-around value may be supplied from a system controller or the like.

[0061] The quantization unit 111 quantizes the acquired aliasing value using a quantization step. This quantization step is set for each arbitrary data unit. For example, this quantization step may be set for each block (multiple pixels of a depth image). The quantization unit 111 also shares this quantization step with the quantization unit 112. That is, the quantization unit 111 quantizes the aliasing value using the same quantization step as the quantization unit 112. This makes it possible to maintain the correspondence between the aliasing value and the depth value. For example, the quantization unit 111 quantizes the aliasing value by dividing the aliasing value by the quantization step. After deriving the quantized aliasing value (for example, (aliasing value ÷ quantization step)), the quantization unit 111 supplies the quantized aliasing value to the correction unit 121 (the calculation units 114 and 115 thereof).

[0062] The quantization unit 112 acquires a depth value as input data. This depth value is information indicating the distance from a sensor used for distance measurement to a subject including the target object of distance measurement. Any distance measurement method may be used. For example, this depth value may be derived by indirect time-of-flight (ToF) distance measurement. For example, IR laser light may be emitted from an optical output unit, and the sensor may detect reflected light of the emitted light reflected by a subject including the target object of distance measurement, and the depth value may be derived from the phase difference between the emitted light and the reflected light. The sensor may have multiple pixels, and the depth value may be derived for each pixel of the sensor. In other words, the quantization unit 112 may acquire a depth image composed of the depth values ​​of each pixel.

[0063] The depth value wraps around within a predetermined range. For example, the depth value may be derived by indirect ToF distance measurement using light with a high modulation frequency. For example, the depth value may be derived by irradiating the object with light of a modulation frequency that narrows the measurable range beyond the range of possible values ​​for the distance to the object and detecting the reflected light.

[0064] The quantization unit 112 quantizes the depth values ​​acquired as described above using a quantization step. This quantization step is set for each arbitrary data unit. For example, this quantization step may be set for each block (multiple pixels of a depth image). The quantization unit 112 also shares this quantization step with the quantization unit 111. That is, the quantization unit 112 quantizes the depth values ​​using the same quantization step as the quantization unit 111. This makes it possible to maintain the correspondence between the foldback value and the depth value. For example, the quantization unit 112 quantizes the depth value by dividing the depth value by the quantization step.

[0065] After deriving the quantized depth value (for example, (depth value / quantization step)), the quantization unit 112 supplies the quantized depth value to the DPCM processing unit 113.

[0066] The DPCM processing unit 113 performs DPCM processing on the depth values ​​quantized by the quantization unit 112, and derives a difference value between the depth value and a predicted value of that depth value. That is, the DPCM processing unit 113 uses the depth value of the previous pixel to be processed, which is the pixel processed immediately before, as a predicted value for the current pixel to be processed, which is the pixel currently to be processed, and derives a difference value between the depth value of the current pixel to be processed and the depth value of the previous pixel to be processed. In this way, the DPCM processing unit 113 derives a difference value, thereby suppressing a decrease in encoding efficiency. The DPCM processing unit 113 supplies the difference value derived by DPCM processing to the correction unit 121 (the calculation unit 114, the calculation unit 115, the selection control unit 116, and the selection unit 117).

[0067] The correction unit 121 appropriately corrects the difference value derived by the DPCM processing unit 113 using the aliasing value quantized by the quantization unit 111 .

[0068] The calculation unit 114 acquires the difference value supplied from the DPCM processing unit 113. The calculation unit 114 also acquires the quantized aliasing value supplied from the quantization unit 111. The calculation unit 114 corrects the difference value by adding the quantized aliasing value to the difference value. The calculation unit 114 supplies the correction result (the addition result of the difference value and the quantized aliasing value) to the selection control unit 116 and the selection unit 117.

[0069] The calculation unit 115 obtains the difference value supplied from the DPCM processing unit 113. The calculation unit 115 also obtains the quantized aliasing value supplied from the quantization unit 111. The calculation unit 115 corrects the difference value by subtracting the quantized aliasing value from the difference value. The calculation unit 115 supplies the correction result (the result of subtracting the difference value from the quantized aliasing value) to the selection control unit 116 and the selection unit 117.

[0070] The selection control unit 116 acquires the difference value supplied from the DPCM processing unit 113. The selection control unit 116 also acquires the correction result of the difference value (i.e., the result of adding the difference value and the quantized aliasing value) supplied from the calculation unit 114. The selection control unit 116 also acquires the correction result of the difference value (i.e., the result of subtracting the difference value from the quantized aliasing value) supplied from the calculation unit 115. The selection control unit 116 controls the operation of the selection unit 117 based on these values.

[0071] The selection unit 117 acquires the difference value supplied from the DPCM processing unit 113. The selection unit 117 also acquires the correction result of the difference value (i.e., the addition result of the difference value and the quantized aliasing value) supplied from the calculation unit 114. The selection unit 117 also acquires the correction result of the difference value (i.e., the subtraction result of the difference value and the quantized aliasing value) supplied from the calculation unit 115. The selection unit 117 selects the one with the smallest absolute value from the acquired difference value, addition result, and subtraction result under the control of the selection control unit 116. The selection unit 117 supplies the selected value (one of the difference value, addition result, and subtraction result) to the encoding unit 118.

[0072] The encoding unit 118 acquires and encodes information supplied from (the selection unit 117 of) the correction unit 121. This information is a difference value appropriately corrected by the correction unit 121. That is, this information is one of the difference value, the addition result, and the subtraction result selected by the selection unit 117. Any encoding method may be used. For example, it may be variable length coding (VLC (Variable Length Code)) or fixed length coding (FLC (Fixed Length Coding)). The encoding unit 118 outputs encoded data derived by encoding the appropriately corrected difference value (the difference value, the addition result, or the subtraction result) to the outside of the encoding device 100.

[0073] <Processing example> An example of processing by the encoding device 100 will be described with reference to FIG. 11. For example, assume that the depth values ​​of four consecutive pixels in the input data are "1490," "0," "10," and "20." These depth values ​​are obtained by high-frequency iToF and wrap around at "1500." These depth values ​​are also assumed to be positive integers. That is, the range of these depth values ​​is assumed to be "0" to "1499." Furthermore, it is assumed that depth value wrapping occurs at the second pixel from the left of the four pixels. That is, the actual distances to the subject corresponding to these four pixels are assumed to be "1490," "1500," "1510," and "1520."

[0074] Also, as shown in FIG. 11, the quantization step is set to "10." The quantization unit 111 quantizes the folded value of the depth value by dividing the folded value by this quantization step. That is, in this example, the quantized folded value becomes "150" as shown in FIG. 11. The quantization unit 112 quantizes the input data (depth value) by dividing the input data by the quantization step "10." Therefore, the input data quantized by the quantization unit 112 becomes "149," "0," "1," and "2," as shown on the left side of FIG. 11.

[0075] Suppose the DPCM processing unit 113 calculates the difference [1] between the depth value of the pixel to the left as the predicted value. Since the DPCM processing unit 113 uses the quantized input data described above as the depth value, [1] becomes "-149", "1", and "1" as shown on the left side of Figure 11. In other words, the difference value increases at the pixel where aliasing occurs.

[0076] As described above, the actual distances corresponding to these four pixels are "1490," "1500," "1510," and "1520." If these depth values ​​were not folded over, the corrected input data (input data without folding over) would be "149," "150," "151," and "152." If the DPCM processing unit 113 derives [1] using these as depth values, [1] would be "1," "1," and "1." That is, in the example of FIG. 11, the difference value of the pixel where folding over occurs increases compared to when folding over does not occur. Therefore, if this [1] is encoded, there is a risk that the encoding efficiency will decrease.

[0077] Therefore, the calculation unit 114 adds the quantized folding value "150" to [1] and derives the correction result (addition result) [2]. In the example on the left side of FIG. 11, [2] becomes "1", "151", and "151". In addition, the calculation unit 115 subtracts the quantized folding value "150" from [1] and derives the correction result (subtraction result) [3]. In the example on the left side of FIG. 11, [3] becomes "-259", "-149", and "-149".

[0078] Then, the selection unit 117 selects, for each pixel, one of [1], [2], and [3] that has the smallest absolute value, under the control of the selection control unit 116. In the example on the left side of Fig. 11, "1", "1", and "1" are selected.

[0079] By appropriately correcting the difference values ​​in this way and selecting the one with the smallest absolute value from among them, it is possible to prevent an increase in the difference values ​​to be coded, and therefore to prevent a decrease in coding efficiency.

[0080] Another example is shown on the right side of Figure 11. In this example, the depth values ​​of four consecutive pixels, which are input data, are assumed to be "20," "10," "0," and "1490." These depth values ​​are obtained by high-frequency iToF and wrap around at "1500." These depth values ​​are assumed to be positive integers. In other words, the range of these depth values ​​is assumed to be "0" to "1499." Of the four pixels mentioned above, it is assumed that depth value wrapping occurs at the rightmost pixel. In other words, the actual distances to the subject corresponding to these four pixels are assumed to be "1520," "1510," "1500," and "1490."

[0081] Also, as shown in Fig. 11, the quantization step is "10." In this example, the quantized folded value is "150," as shown in Fig. 11. Also, the quantized input data is "2," "1," "0," and "149," as shown on the right side of Fig. 11.

[0082] In this example, [1] becomes "-1", "-1", and "-149". In other words, the difference value increases in pixels where aliasing occurs.

[0083] As described above, the actual distances corresponding to these four pixels are "1520," "1510," "1500," and "1490." If these depth values ​​were not folded, the corrected input data (input data without folding) would be "152," "151," "150," and "149." Therefore, [1] would be "-1," "-1," and "-1." In other words, in the example of FIG. 11, the difference value of the pixel where folding occurs increases compared to when folding does not occur. Therefore, encoding this [1] may result in reduced coding efficiency.

[0084] Therefore, the calculation unit 114 adds the quantized folding value "150" to [1] and derives the correction result (addition result) [2]. In the example on the right side of FIG. 11, [2] becomes "149", "149", and "299". Furthermore, the calculation unit 115 subtracts the quantized folding value "150" from [1] and derives the correction result (subtraction result) [3]. In the example on the right side of FIG. 11, [3] becomes "-151", "-151", and "-1".

[0085] Then, the selection unit 117 selects, for each pixel, one of [1], [2], and [3] that has the smallest absolute value, under the control of the selection control unit 116. In the example on the right side of Fig. 11, "-1", "-1", and "-1" are selected.

[0086] By appropriately correcting the difference values ​​in this way and selecting the one with the smallest absolute value from among them, it is possible to prevent an increase in the difference values ​​to be coded, and therefore to prevent a decrease in coding efficiency.

[0087] Furthermore, since the value at which the depth value folds back and the depth value are quantized, it is possible to further suppress an increase in the difference value, and it is possible to further suppress a decrease in coding efficiency.

[0088] <Encoding process flow> Next, an example of the flow of the encoding process executed by the encoding device 100 will be described with reference to the flowchart of FIG.

[0089] When the encoding process starts, in step S101, the quantization unit 111 of the encoding device 100 quantizes the aliased value by dividing the aliased value by a predetermined quantization step.

[0090] In step S102, the quantization unit 112 quantizes the input data (depth values ​​that wrap around within a predetermined range) by dividing the input data by the same quantization step that the quantization unit 111 used for quantization.

[0091] In step S103, the DPCM processing unit 113 performs DPCM processing on the quantized input data derived in step S102, and derives the difference value (difference data) between the depth value of the current pixel to be processed and the depth value of the previous pixel to be processed.

[0092] In step S104, the calculation unit 114 adds the quantized aliasing value derived in step S101 to the difference data derived in step S103.

[0093] In step S105, the calculation unit 115 subtracts the quantized aliasing value derived in step S101 from the difference data derived in step S103.

[0094] In step S106, the selection unit 117, under the control of the selection control unit 116, selects the differential data derived in step S103, the addition result derived in step S104, and the subtraction result derived in step S105, which has the smallest absolute value.

[0095] In step S107, the encoding unit 118 encodes the appropriately corrected difference data (the difference value, the addition result, or the subtraction result selected in step S106) to derive encoded data.

[0096] When the process of step S107 ends, the encoding process ends. The encoding device 100 performs such encoding process for each pixel (each depth value) of the depth image.

[0097] By doing so, as described above, it is possible to suppress an increase in the differential value to be coded, and therefore to suppress a decrease in coding efficiency.

[0098] Furthermore, since the value at which the depth value folds back and the depth value are quantized, it is possible to further suppress an increase in the difference value, and it is possible to further suppress a decrease in coding efficiency.

[0099] 3. Second Embodiment <Decryption device> Fig. 13 is a block diagram showing an example of the configuration of a decoding device, which is one aspect of an information processing device to which the present technology is applied. The decoding device 200 shown in Fig. 13 is a device that decodes coded data of depth data (depth images, depth values) obtained by iToF. The decoding device 200 acquires and decodes the coded data generated by the coding device 100, and outputs the depth values ​​as output data.

[0100] Note that Fig. 13 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, in this decoding device 200, there may be processing units that are not shown as blocks in Fig. 13, or there may be processing or data flows that are not shown as arrows, etc. in Fig. 13.

[0101] 13, the decoding device 200 includes a quantization unit 211, a decoding unit 1212, an inverse DPCM processing unit 213, an arithmetic unit 214, an arithmetic unit 215, a selection control unit 216, a selection unit 217, and an inverse quantization unit 218. The arithmetic unit 214, the arithmetic unit 215, the selection control unit 216, and the selection unit 217 configure a correction unit 221.

[0102] The quantization unit 211 acquires a value that the wrap-around depth value turns into. This turn-around value is the same as that in the encoding device 100 (the example in FIG. 10 ). This turn-around value is supplied from outside the decoding device 200. For example, this turn-around value may be supplied from a system controller or the like. Furthermore, for example, this turn-around value may be supplied from the encoding device 100 that generated the encoded data.

[0103] The quantization unit 211 quantizes the acquired aliasing value using a quantization step. This quantization step is set for each arbitrary data unit. For example, this quantization step may be set for each block (multiple pixels of a depth image). The quantization unit 211 also shares this quantization step with the inverse quantization unit 218. That is, the quantization unit 211 quantizes the aliasing value using the same quantization step as the inverse quantization unit 218. This makes it possible to maintain the correspondence between the aliasing value and the depth value. For example, the quantization unit 211 quantizes the aliasing value by dividing the aliasing value by the quantization step. After deriving the quantized aliasing value (for example, (aliasing value / quantization step)), the quantization unit 211 supplies the quantized aliasing value to the correction unit 221 (the calculation unit 214, the calculation unit 215, and the selection control unit 216).

[0104] The decoding unit 212 acquires coded data. This coded data is generated by the coding device 100 applying the present technology described in the first embodiment. The decoding unit 212 decodes the acquired coded data to generate differential data. The decoding unit 212 decodes the coded data using a decoding method corresponding to the coding method used by the coding unit 118. This differential data is a differential value between coefficient data relating to depth values ​​that wrap around in a predetermined value range and a predicted value of that coefficient data. For example, the differential data may be variable length coding (VLC (Variable Length Code)) or fixed length coding (FLC (Fixed Length Coding)). The decoding unit 212 supplies the derived differential data to the inverse DPCM processing unit 213.

[0105] The inverse DPCM processor 213 acquires differential data for the current pixel to be processed, which is supplied from the decoder 212. This differential data is a differential value between the coefficient data for the current pixel to be processed and the coefficient data for the previous pixel to be processed, which is the pixel to be processed immediately before the current pixel to be processed. The inverse DPCM processor 213 also acquires coefficient data for the previous pixel to be processed from the selector 217. The inverse DPCM processor 213 performs inverse DPCM processing, which is the inverse process of DPCM processing, using the coefficient data for the previous pixel to be processed as a predicted value for the current pixel to be processed. The inverse DPCM processing is a process of adding the predicted value for the current pixel to the differential data for the current pixel to be processed. That is, the inverse DPCM processor 213 derives coefficient data for the current pixel to be processed by adding the coefficient data for the previous pixel to the differential data for the current pixel to be processed. In this way, the inverse DPCM processor 213 derives coefficient data for the current pixel to be processed, thereby correctly decoding coded data in which the differential value is coded. That is, coding of the differential value can be achieved. Therefore, a decrease in coding efficiency can be suppressed. The inverse DPCM processing unit 213 supplies the coefficient data of the current pixel to be processed, which is derived by the inverse DPCM processing, to the correction unit 221 (the calculation unit 214, the calculation unit 215, the selection control unit 216, and the selection unit 217).

[0106] The correction unit 221 appropriately corrects the coefficient data derived by the inverse DPCM processing unit 1213 using the folding value quantized by the quantization unit 211 .

[0107] The calculation unit 214 acquires coefficient data supplied from the inverse DPCM processing unit 213. The calculation unit 214 also acquires the quantized aliasing value supplied from the quantization unit 211. The calculation unit 214 corrects the coefficient data by adding the quantized aliasing value to the coefficient data. The calculation unit 214 supplies the correction result (the addition result of the coefficient data and the quantized aliasing value) to the selection unit 217.

[0108] The calculation unit 215 obtains the coefficient data supplied from the inverse DPCM processing unit 213. The calculation unit 215 also obtains the quantized aliasing value supplied from the quantization unit 211. The calculation unit 215 corrects the coefficient data by subtracting the quantized aliasing value from the coefficient data. The calculation unit 215 supplies the correction result (the result of subtracting the coefficient data from the quantized aliasing value) to the selection unit 217.

[0109] The selection control unit 216 obtains the quantized folding value supplied from the quantization unit 211. The selection control unit 216 also obtains the coefficient data supplied from the inverse DPCM processing unit 213. The selection control unit 216 controls the operation of the selection unit 217 based on these values.

[0110] The selection unit 217 acquires coefficient data supplied from the inverse DPCM processing unit 213. The selection unit 217 also acquires the correction result of the coefficient data supplied from the calculation unit 214 (i.e., the addition result of the coefficient data and the quantized aliasing value). The selection unit 217 also acquires the correction result of the coefficient data supplied from the calculation unit 215 (i.e., the subtraction result of the coefficient data and the quantized aliasing value). The selection unit 217 selects, under the control of the selection control unit 216, the acquired coefficient data, addition result, and subtraction result whose value is within the range of depth values. The selection unit 217 supplies the selected value (one of the coefficient data, addition result, and subtraction result) to the inverse quantization unit 218 as appropriately corrected coefficient data.

[0111] The inverse quantization unit 218 acquires appropriately corrected coefficient data supplied from the selection unit 217. This coefficient data is a quantized depth value.

[0112] The inverse quantization unit 218 inverse quantizes the appropriately corrected coefficient data using a quantization step. For example, the inverse quantization unit 218 inverse quantizes the coefficient data, the addition result, or the subtraction result selected by the selection unit 217. This appropriately corrected coefficient data is a quantized depth value. In other words, the inverse quantization unit 218 derives a depth value by inverse quantizing the appropriately corrected coefficient data.

[0113] This quantization step is set for each arbitrary data unit. For example, this quantization step may be set for each block (multiple pixels of a depth image). Furthermore, the inverse quantization unit 218 shares this quantization step with the quantization unit 211. That is, the inverse quantization unit 218 inversely quantizes the coefficient data using the same quantization step as the quantization unit 211. This makes it possible to maintain the correspondence between the folding value and the depth value. For example, the inverse quantization unit 218 derives the depth value by multiplying the appropriately corrected coefficient data by the quantization step.

[0114] This depth value is information indicating the distance from the sensor used for distance measurement to a subject, including the target object of distance measurement. Any distance measurement method may be used. For example, this depth value may be derived by indirect Time-of-Flight (ToF) distance measurement. For example, an IR laser beam may be emitted from an optical output unit, and the sensor may detect the reflected light of the emitted light reflected by a subject, including the target object of distance measurement, and the depth value may be derived from the phase difference between the emitted light and the reflected light. The sensor may have multiple pixels, and the depth value may be derived for each pixel of the sensor. That is, the inverse quantization unit 218 derives such a depth value by inverse quantizing the coefficient data.

[0115] This depth value wraps around within a predetermined range. For example, this depth value may be derived by indirect ToF distance measurement using light with a high modulation frequency. For example, a depth value derived by irradiating the object with light of a modulation frequency that narrows the range of distances that can be measured compared to the range of possible values ​​for the distance to the object and detecting the reflected light may be obtained (restored) by this inverse quantization.

[0116] The inverse quantization unit 218 outputs the derived depth value as output data to the outside of the decoding device 200.

[0117] <Processing example> An example of processing by this decoding device 200 will be described with reference to Fig. 14. For example, as in the example on the left side of Fig. 14, suppose that the differential data between four consecutive pixels and their left adjacent pixels is "1", "1", "1". When this differential data is subjected to inverse DPCM processing, the coefficient data (decoded data) becomes "149", "150", "1", "2".

[0118] If this coefficient data falls outside the range (0 to 149), the selection unit 217 selects corrected coefficient data that falls within the range. In the example on the left side of Fig. 14, the decoded data of the second pixel from the left falls outside the range, so the selection unit 217 selects the corrected coefficient data. In this example, the selection unit 217 selects the subtraction result obtained by subtracting the quantized folding value from the decoded data.

[0119] The inverse quantization unit 218 then inverse quantizes the coefficient data thus appropriately corrected. This inverse quantization results in output data (depth values) of "1490", "0", "10", and "20".

[0120] In the example on the right side of Figure 14, the difference data between four consecutive pixels and their neighboring pixels on the left is "-1", "-1", "-1". When this difference data is subjected to inverse DPCM processing, coefficient data (decoded data) of "2", "1", "0", and "-1" are obtained.

[0121] If this coefficient data falls outside the range (0 to 149), the selection unit 217 selects corrected coefficient data that falls within that range. In the example on the right side of Fig. 14, the decoded data of the fourth pixel from the left falls outside the range, so the selection unit 217 selects the corrected coefficient data. In this example, the selection unit 217 selects the addition result of adding the quantized folding value to the decoded data.

[0122] The inverse quantization unit 218 then inverse quantizes the coefficient data thus appropriately corrected. This inverse quantization results in output data (depth values) of "20", "10", "0", and "1490".

[0123] By appropriately correcting coefficient data outside the range in this way, it is possible to suppress an increase in the difference value, and therefore a decrease in coding efficiency. Furthermore, since the value at which the depth value folds is quantized and used, it is possible to further suppress an increase in the difference value and a decrease in coding efficiency.

[0124] <Decryption process flow> Next, an example of the flow of the decoding process executed by the decoding device 200 will be described with reference to the flowchart of FIG.

[0125] When the decoding process starts, in step S201, the quantization unit 211 of the decoding device 200 quantizes the foldback value by dividing the foldback value by a predetermined quantization step.

[0126] In step S202, the decoding unit 212 decodes the coded data and derives the difference data.

[0127] In step S203, the inverse DPCM processing unit 213 performs inverse DPCM processing on the difference data derived in step S201 to derive coefficient data (decoded data).

[0128] In step S204, the calculation unit 214 adds the quantized folding value derived in step S201 to the coefficient data (decoded data) derived in step S203.

[0129] In step S205, the calculation unit 215 subtracts the quantized aliasing value derived in step S201 from the coefficient data (decoded data) derived in step S203.

[0130] In step S206, the selection unit 217, under the control of the selection control unit 216, selects the coefficient data (decoded data) derived in step S203, the addition result derived in step S204, and the subtraction result derived in step S205 that falls within the range of the depth value.

[0131] In step S207, the inverse quantization unit 218 inverse quantizes the coefficient data (decoded data) derived in step S203, the addition result derived in step S204, and the subtraction result derived in step S205, whichever is selected in step S206, to derive a depth value.

[0132] The decoding process ends when the process of step S207 ends. The decoding device 200 performs such a decoding process for each pixel (each depth value) of the depth image.

[0133] By doing so, as described above, it is possible to suppress an increase in the difference value, and therefore to suppress a decrease in coding efficiency. Furthermore, since the value at which the depth value folds over is quantized, it is possible to further suppress an increase in the difference value, and therefore to further suppress a decrease in coding efficiency.

[0134] <4. Third Embodiment> <Encoding device> 9, the encoding device 100 has been described as deriving the difference value, the addition result, and the subtraction result, and selecting the one with the smallest absolute value, as a method of appropriately correcting the difference value. However, any method may be used to appropriately correct the difference value, and is not limited to this example. For example, the correction method may be selected based on the difference value.

[0135] An example of the main configuration of the encoding device 100 in this case is shown in Fig. 16. In this case, the DPCM processing unit 113 supplies the difference value derived by the DPCM processing to the correction unit 121 (to the selection control unit 116 and selection unit 117).

[0136] The selection control unit 116 acquires the difference value supplied from the DPCM processing unit 113. The selection control unit 116 also acquires the quantized folding value supplied from the quantization unit 111. The selection control unit 116 controls the operation of the selection unit 117 based on these values.

[0137] The selection unit 117 acquires the differential value supplied from the DPCM processing unit 113. Under the control of the selection control unit 116, the selection unit 117 selects a correction method for the differential value based on the acquired differential value. Then, the selection unit 117 supplies the differential value to a processing unit corresponding to the selection. For example, the selection unit 117 selects, as the correction method for the differential value, whether to omit correction of the differential value, add a quantized aliasing value to the differential value, or subtract a quantized aliasing value from the differential value. Then, if omitting correction of the differential value is selected, the selection unit 117 supplies the differential value to the encoding unit 118. If adding a quantized aliasing value is selected, the selection unit 117 supplies the differential value to the calculation unit 114. If subtracting a quantized aliasing value is selected, the selection unit 117 supplies the differential value to the calculation unit 115.

[0138] When the selection unit 117 selects addition of the quantized folding value, the calculation unit 114 obtains the difference value supplied from the selection unit 117 and adds the quantized folding value supplied from the quantization unit 111 to the difference value. The calculation unit 114 supplies the correction result (the addition result of the difference value and the quantized folding value) to the encoding unit 118.

[0139] When subtraction of the quantized aliasing value is selected by the selection unit 117, the calculation unit 115 obtains the difference value supplied from the selection unit 117 and subtracts the quantized aliasing value supplied from the quantization unit 111 from the difference value. The calculation unit 115 supplies the correction result (the result of subtracting the difference value from the quantized aliasing value) to the encoding unit 118.

[0140] When the selection unit 117 selects to omit correction of the difference value, the encoding unit 118 acquires the difference value supplied from the selection unit 117 and encodes the difference value. Furthermore, when the selection unit 117 selects addition of a quantized foldback value, the encoding unit 118 encodes the addition result of the difference value derived by the calculation unit 114 and the quantized foldback value. Furthermore, when the selection unit 117 selects subtraction of a quantized foldback value, the encoding unit 118 encodes the subtraction result of the difference value derived by the calculation unit 115 and the quantized foldback value. Applicable encoding methods are the same as those in FIG. 9 . In this way, the encoding unit 118 outputs encoded data derived by encoding the appropriately corrected difference value (the difference value, the addition result, or the subtraction result) to the outside of the encoding device 100.

[0141] By selecting a correction method based on the difference value in this way, it is possible to suppress an increase in the difference value to be coded. Therefore, it is possible to suppress a decrease in coding efficiency. Also in this case, since the value at which the depth value folds back and the depth value are quantized, it is possible to further suppress an increase in the difference value and further suppress a decrease in coding efficiency.

[0142] <Encoding process flow> An example of the flow of the encoding process in this case will be described with reference to the flowchart in FIG.

[0143] In this case, when the encoding process starts, the quantization unit 111 of the encoding device 100 quantizes the aliased value by dividing the aliased value by a predetermined quantization step in step S301.

[0144] In step S302, the quantization unit 112 quantizes the input data (depth values ​​that wrap around within a predetermined range) by dividing the input data by the same quantization step that the quantization unit 111 used for quantization.

[0145] In step S303, the DPCM processing unit 113 performs DPCM processing on the quantized input data derived in step S102, and derives the difference value (difference data) between the depth value of the current pixel to be processed and the depth value of the previous pixel to be processed.

[0146] In step S304, the selection unit 117, under the control of the selection control unit 116, selects a correction method for the differential data based on the value of the differential data.

[0147] In step S305, the selection control unit 116 determines whether or not to add a quantized foldback value to the difference data. If it is determined that the quantized foldback value is to be added to the difference data, that is, if the selection unit 117 selects adding the quantized foldback value, the process proceeds to step S306.

[0148] In step S306, the calculation unit 114 adds the quantized aliasing value derived in step S301 to the difference data derived in step S303. When the process of step S306 ends, the process proceeds to step S309.

[0149] Also, if it is determined in step S305 that the quantized folding value is not to be added to the difference data, the process proceeds to step S307.

[0150] In step S307, the selection control unit 116 determines whether or not to subtract the quantized aliasing value from the difference data. If it is determined that the quantized aliasing value is to be subtracted from the difference data, that is, if the selection unit 117 selects subtraction of the quantized aliasing value, the process proceeds to step S308.

[0151] In step S308, the calculation unit 115 subtracts the quantized aliasing value derived in step S301 from the difference data derived in step S303. When the process of step S308 ends, the process proceeds to step S309.

[0152] Also, in step S307, if it is determined that the quantized folding value is not to be subtracted from the difference data, that is, if it is determined that correction of the difference data is to be omitted (skip), the process proceeds to step S309.

[0153] In step S309, the encoding unit 118 encodes the differential data appropriately corrected as described above (the differential value derived in step S303, the addition result derived in step S306, or the subtraction result derived in step S308) to derive encoded data.

[0154] When the process of step S309 ends, the encoding process ends. The encoding device 100 performs such encoding process for each pixel (each depth value) of the depth image.

[0155] By doing so, as described above, it is possible to suppress an increase in the differential value to be coded. Therefore, it is possible to suppress a decrease in coding efficiency. Also in this case, since the value at which the depth value folds back and the depth value are quantized, it is possible to further suppress an increase in the differential value and further suppress a decrease in coding efficiency.

[0156] 5. Fourth Embodiment <Decryption device> 13, the decoding device 200 has been described as deriving a difference value, an addition result, and a subtraction result, and selecting one of them that falls within the range of depth values, as a method for appropriately correcting coefficient data. However, the method for appropriately correcting coefficient data is arbitrary and is not limited to this example. For example, the correction method may be selected based on the value of the coefficient data.

[0157] An example of the main configuration of the decoding device 200 in this case is shown in Fig. 18. In this case, the inverse DPCM processing unit 213 supplies the coefficient data derived by the inverse DPCM processing to the correction unit 221 (the selection control unit 216 and the selection unit 217).

[0158] The selection control unit 216 acquires the coefficient data supplied from the inverse DPCM processing unit 213. The selection control unit 216 also acquires the quantized folding values ​​supplied from the quantization unit 211. The selection control unit 216 controls the operation of the selection unit 217 based on these values.

[0159] The selection unit 217 acquires coefficient data supplied from the inverse DPCM processing unit 213. The selection unit 217 selects a correction method for the coefficient data based on the value of the acquired coefficient data under the control of the selection control unit 216. Then, the selection unit 217 supplies the coefficient data to a processing unit according to the selection. For example, the selection unit 217 selects, as the correction method for the coefficient data, whether to omit correction of the coefficient data, add a quantized aliasing value to the coefficient data, or subtract a quantized aliasing value from the coefficient data. Then, if omitting correction of the coefficient data is selected, the selection unit 217 supplies the coefficient data to the inverse quantization unit 218. If addition of a quantized aliasing value is selected, the selection unit 217 supplies the coefficient data to the calculation unit 214. If subtraction of a quantized aliasing value is selected, the selection unit 217 supplies the coefficient data to the calculation unit 215.

[0160] When the selection unit 217 selects addition of the quantized aliasing value, the calculation unit 214 acquires the coefficient data supplied from the selection unit 217 and adds the quantized aliasing value supplied from the quantization unit 211 to the coefficient data. The calculation unit 214 supplies the correction result (the addition result of the coefficient data and the quantized aliasing value) to the inverse quantization unit 218.

[0161] When the selection unit 217 selects subtraction of the quantized aliasing value, the calculation unit 215 obtains the coefficient data supplied from the selection unit 217 and subtracts the quantized aliasing value supplied from the quantization unit 211 from the coefficient data. The calculation unit 215 supplies the correction result (the subtraction result of the coefficient data and the quantized aliasing value) to the inverse quantization unit 218.

[0162] When the selection unit 217 selects to omit correction of coefficient data, the inverse quantization unit 218 obtains coefficient data supplied from the selection unit 217 and inverse quantizes the coefficient data using the quantization step. When the selection unit 217 selects addition of quantized aliasing values, the inverse quantization unit 218 obtains the addition result of the coefficient data derived by the calculation unit 214 and the quantized aliasing values, and inverse quantizes the addition result using the quantization step. When the selection unit 217 selects subtraction of quantized aliasing values, the inverse quantization unit 218 obtains the subtraction result of the coefficient data derived by the calculation unit 215 and the quantized aliasing values, and inverse quantizes the subtraction result using the quantization step.

[0163] The coefficient data appropriately corrected as described above is a quantized depth value. That is, the inverse quantization unit 218 derives a depth value by inverse quantizing the appropriately corrected coefficient data.

[0164] Note that this quantization step is set for each arbitrary data unit, as in the case of FIG. 13. For example, this quantization step may be set for each block (multiple pixels of a depth image). Furthermore, as in the case of FIG. 13, the inverse quantization unit 218 shares this quantization step with the quantization unit 211. That is, the inverse quantization unit 218 inversely quantizes the coefficient data using the same quantization step as the quantization unit 211. This makes it possible to maintain the correspondence between the folded value and the depth value. For example, the inverse quantization unit 218 derives the depth value by multiplying the appropriately corrected coefficient data by the quantization step.

[0165] The inverse quantization unit 218 outputs the derived depth value as output data to the outside of the decoding device 200.

[0166] By selecting a correction method based on the value of the coefficient data in this way, it is possible to suppress an increase in the difference value. Therefore, it is possible to suppress a decrease in coding efficiency. Also in this case, the value at which the depth value folds over is quantized using the same quantization step as the inverse quantization of the coefficient data, so it is possible to further suppress an increase in the difference value and further suppress a decrease in coding efficiency.

[0167] <Decryption process flow> An example of the flow of the decoding process in this case will be described with reference to the flowchart in FIG.

[0168] In this case, when the decoding process starts, the quantization unit 211 of the decoding device 200 quantizes the foldback value by dividing the foldback value by a predetermined quantization step in step S401.

[0169] In step S402, the decoding unit 212 decodes the coded data that is the input data, and derives the difference data.

[0170] In step S403, the inverse DPCM processing unit 213 performs inverse DPCM processing on the difference data derived in step S402, and derives coefficient data (decoded data) for the current pixel to be processed.

[0171] In step S404, the selection unit 217, under the control of the selection control unit 216, selects a correction method for the coefficient data (decoded data) based on the value of the coefficient data.

[0172] In step S405, the selection control unit 216 determines whether or not to add a quantized aliasing value to the coefficient data. If it is determined that the quantized aliasing value is to be added to the coefficient data, that is, if the selection unit 217 selects adding the quantized aliasing value, the process proceeds to step S406.

[0173] In step S406, the calculation unit 214 adds the quantized folding value derived in step S401 to the coefficient data (decoded data) derived in step S403. When the process of step S406 ends, the process proceeds to step S409.

[0174] Also, if it is determined in step S405 that the quantized folding value is not to be added to the coefficient data, the process proceeds to step S407.

[0175] In step S407, the selection control unit 216 determines whether or not to subtract the quantized aliasing value from the coefficient data. If it is determined that the quantized aliasing value is to be subtracted from the coefficient data, that is, if the selection unit 217 selects subtraction of the quantized aliasing value, the process proceeds to step S408.

[0176] In step S408, the calculation unit 215 subtracts the quantized aliasing value derived in step S401 from the difference data derived in step S403. When the process of step S408 ends, the process proceeds to step S409.

[0177] Also, in step S407, if it is determined that the quantized folding value is not to be subtracted from the coefficient data, that is, if it is determined that correction of the coefficient data is to be omitted (skip), the process proceeds to step S409.

[0178] In step S409, the inverse quantization unit 218 inverse quantizes the coefficient data appropriately corrected as described above (the coefficient data derived in step S403, the addition result derived in step S406, or the subtraction result derived in step S408) using the same quantization step as used for the quantization in step S401, to derive a depth value.

[0179] When the process of step S409 ends, the decoding device 200 performs such a decoding process for each pixel (each depth value) of the depth image.

[0180] By doing so, as described above, it is possible to suppress an increase in the difference value, and therefore to suppress a decrease in coding efficiency. Also in this case, since the value at which the depth value folds over is quantized, it is possible to further suppress an increase in the difference value, and therefore to further suppress a decrease in coding efficiency.

[0181] 6. Fifth Embodiment <Distance measuring device> FIG. 20 is a block diagram showing an example of the main configuration of a distance measuring device. The distance measuring device 500 shown in FIG. 20 performs distance measurement by combining iToFs with multiple modulation frequencies. For example, the distance measuring device 500 performs iToFs with a low modulation frequency to derive a depth value, performs iToFs with a high modulation frequency to derive a depth value, and merges these depth values. In this way, the distance measuring device 500 can improve the accuracy of distance measurement while preventing the measurable range from being narrowed.

[0182] Note that Fig. 20 shows the main processing units, data flows, etc., and is not necessarily all that is shown in Fig. 20. In other words, in this distance measuring device 500, there may be processing units that are not shown as blocks in Fig. 20, and there may be processing or data flows that are not shown as arrows, etc. in Fig. 20.

[0183] 20 , the distance measuring device 500 includes a control unit 501, an optical output unit 511, a detection unit 512, an iToF measurement unit 513, a RAM (Random Access Memory) 514, and a data merge unit 515. The iToF measurement unit 513 includes a depth value derivation unit 521 and an encoding unit 522. The data merge unit 515 includes a decoding unit 531 and a merge processing unit 532.

[0184] The control unit 501 controls each processing unit of the distance measuring device 500. As shown in Fig. 20, the distance measuring device 500 has only one set of an optical output unit 511, a detection unit 512, and an iToF measurement unit 513. The distance measuring device 500 performs both iToF with a high modulation frequency and iToF with a low modulation frequency using this set of the optical output unit 511, the detection unit 512, and the iToF measurement unit 513. Therefore, the control unit 501 time-shares this set of the optical output unit 511, the detection unit 512, and the iToF measurement unit 513 to execute both iToF with a high modulation frequency and iToF with a low modulation frequency.

[0185] The light output unit 511 outputs light with a high modulation frequency (for example, IR laser light) and light with a low modulation frequency (for example, IR laser light) in a time-division manner.

[0186] The detection unit 512 detects reflected light of the light emitted from the light output unit 511. The detection unit 512 detects the reflected light (reflected light of a high modulation frequency) at a timing corresponding to the timing when the light output unit 511 emits light of a high modulation frequency. The detection unit 512 also detects the reflected light (reflected light of a low modulation frequency) at a timing corresponding to the timing when the light output unit 511 emits light of a low modulation frequency. The detection unit 512 supplies data of the detection result (data indicating the amount of light received at each pixel) to the iToF measurement unit 513 (the depth value derivation unit 521 thereof).

[0187] The depth value derivation unit 521 uses the detection result to derive a depth value to the subject by the indirect ToF method. The depth value derivation unit 521 derives depth values ​​by iToF using light with a high modulation frequency and by iToF using light with a low modulation frequency in a time-division manner.

[0188] The merge processing unit 532 of the data merge unit 515 merges these depth values. For this merging, at least the depth values ​​derived by iToF using light with a high modulation frequency are stored in the RAM 514. Therefore, the depth value derivation unit 521 supplies at least the depth values ​​derived by iToF using light with a high modulation frequency to the encoding unit 522. Note that if the depth values ​​derived by iToF using light with a low modulation frequency are not stored in the RAM 514, the depth value derivation unit 521 supplies the depth values ​​to the merge processing unit 532 of the data merge unit 515.

[0189] The encoding unit 522 encodes the depth values ​​to generate encoded data, and supplies the encoded data to the RAM 514 for storage.

[0190] The RAM 514 stores the coded data supplied from the coding unit 522. Furthermore, the RAM 514 supplies the stored coded data to the decoding unit 531 in response to a request from the decoding unit 531.

[0191] The decoding unit 531 of the data merging unit 515 reads and decodes the coded data stored in the RAM 514 to derive depth values. The decoding unit 531 supplies the derived depth values ​​to the merging processing unit 532.

[0192] The merge processing unit 532 acquires depth values ​​derived by iToF using light with a high modulation frequency and depth values ​​derived by iToF using light with a low modulation frequency, merges them, and outputs the merged depth values.

[0193] The present technology can be applied to such a distance measuring device 500. For example, the encoding device 100 described in the first embodiment or the third embodiment may be applied as the encoding unit 522. Furthermore, the decoding device 200 described in the second embodiment or the fourth embodiment may be applied as the decoding unit 531.

[0194] By doing so, it is possible to suppress a decrease in the coding efficiency of the coded data stored in the RAM 514. This makes it possible to suppress an increase in the storage capacity of the RAM 514. As a result, it is possible to suppress an increase in costs. Furthermore, it is possible to suppress an increase in circuit size and power consumption.

[0195] <Distance measurement process flow> An example of the flow of distance measurement processing executed by the distance measuring device 500 will be described with reference to the flowchart of FIG.

[0196] When the distance measurement process is started, the control unit 501 sets the modulation frequency to a high frequency in step S501.

[0197] In step S502, the optical output unit 511 emits light with a high modulation frequency, and the detection unit 512 detects the reflected light.

[0198] In step S503, the depth value derivation unit 521 derives a depth value by iToF using light with a high frequency modulation frequency.

[0199] In step S504, the encoding unit 522 performs an encoding process to encode the depth value.

[0200] In step S505, the RAM 514 stores the coded data derived in step S504.

[0201] In step S506, the control unit 501 sets the modulation frequency to a low frequency.

[0202] In step S507, the optical output unit 511 emits light with a low modulation frequency, and the detection unit 512 detects the reflected light.

[0203] In step S508, the depth value derivation unit 521 derives a depth value by iToF using light with a low modulation frequency.

[0204] In step S509, the decoding unit 531 reads out the coded data stored in step S506. In step S510, the decoding unit 531 executes a decoding process to decode the coded data that has been read out.

[0205] In step S511, the merge processing unit 532 merges the depth value derived by iToF using light with a high modulation frequency and the depth value derived by iToF using light with a low modulation frequency.

[0206] In step S512, the control unit 501 determines whether or not to end the distance measurement process. If it is determined not to end, the process returns to step S501, and the subsequent processes are repeated. Then, if it is determined in step S512 that the distance measurement process is to end, the distance measurement process ends.

[0207] The present technology can be applied to such distance measurement processing. For example, the encoding processing of step S504 may be the encoding processing described with reference to the flowchart of Fig. 12 or the encoding processing described with reference to the flowchart of Fig. 17. Furthermore, the decoding processing of step S510 may be the decoding processing described with reference to the flowchart of Fig. 15 or the decoding processing described with reference to the flowchart of Fig. 19.

[0208] By doing so, it is possible to suppress a decrease in the coding efficiency of the coded data stored in the RAM 514. This makes it possible to suppress an increase in the storage capacity of the RAM 514. As a result, it is possible to suppress an increase in costs. Furthermore, it is possible to suppress an increase in circuit size and power consumption.

[0209] 7. Sixth Embodiment <Parallel configuration> The distance measuring device may have multiple systems for performing iToF. For example, as shown in Fig. 22, the derivation of depth values ​​by iToF using light with a high modulation frequency and the derivation of depth values ​​by iToF using light with a low modulation frequency may be performed using different systems.

[0210] <Distance measuring device> FIG. 23 is a block diagram showing an example of the main configuration of a ranging device in this case. Similar to ranging device 500, ranging device 600 shown in FIG. 23 performs ranging by combining iToFs with multiple modulation frequencies. For example, ranging device 600 performs iToFs with a low modulation frequency to derive a depth value, performs iToFs with a high modulation frequency to derive a depth value, and merges these depth values. In this way, ranging device 600 can improve ranging accuracy while preventing the measurable range from being narrowed.

[0211] 23, the ranging device 600 includes a high-frequency ranging unit 601, a low-frequency ranging unit 602, and a data merging unit 603. The high-frequency ranging unit 601 performs iToF with a high-frequency modulation frequency to derive depth values. The low-frequency ranging unit 602 performs iToF with a low-frequency modulation frequency to derive depth values. The data merging unit 603 acquires the depth values ​​derived by the high-frequency ranging unit 601 and the low-frequency ranging unit 602 and merges them.

[0212] The high frequency ranging unit 601 has an optical output unit 611, a detection unit 612, and an iToF measurement unit 613. The iToF measurement unit 613 has a depth value derivation unit 621 and an encoding unit 622.

[0213] The light output unit 611 emits light (for example, IR laser light) with a high-frequency modulation frequency. The detection unit 612 detects the reflected light and supplies data indicating the detection result to the iToF measurement unit 613 (the depth value derivation unit 621 thereof).

[0214] The depth value derivation unit 621 uses the data supplied from the detection unit 612 and the detection result to derive a depth value to the subject by an indirect ToF method. That is, the depth value derivation unit 621 derives a depth value by iToF using light with a high-frequency modulation frequency. The encoding unit 622 encodes the depth value derived by the depth value derivation unit 621. The encoding unit 622 supplies the generated encoded data (encoded data of the depth value derived by iToF using light with a high-frequency modulation frequency) to the data merge unit 603 (decoding unit 651 thereof).

[0215] The low-frequency ranging unit 602 has an optical output unit 631, a detection unit 632, and an iToF measurement unit 633. The iToF measurement unit 633 has a depth value derivation unit 641 and an encoding unit 642.

[0216] The light output unit 631 emits light (for example, IR laser light) with a low modulation frequency. The detection unit 632 detects the reflected light and supplies data indicating the detection result to the iToF measurement unit 633 (the depth value derivation unit 641 thereof).

[0217] The depth value derivation unit 641 uses the data supplied from the detection unit 632 and the detection result to derive a depth value to the subject by an indirect ToF method. That is, the depth value derivation unit 641 derives a depth value by iToF using light with a low modulation frequency. The encoding unit 642 encodes the depth value derived by the depth value derivation unit 641. The encoding unit 642 supplies the generated encoded data (encoded data of the depth value derived by iToF using light with a low modulation frequency) to the data merge unit 603 (the decoding unit 652 thereof).

[0218] The data merge unit 603 includes a decoding unit 651 , a decoding unit 652 , and a merge processing unit 653 .

[0219] The decoding unit 651 decodes the coded data supplied from the coding unit 622 and generates (restores) depth values ​​derived by iToF using light with a high-frequency modulation frequency. The decoding unit 651 supplies the depth values ​​to the merge processing unit 653.

[0220] The decoding unit 652 decodes the coded data supplied from the coding unit 642 and generates (restores) depth values ​​derived by iToF using light with a low modulation frequency. The decoding unit 652 supplies the depth values ​​to the merge processing unit 653.

[0221] The merge processing unit 653 merges the depth values ​​supplied from the decoding unit 651 and the decoding unit 652. That is, the merge processing unit 653 merges the depth values ​​derived by iToF using light with a high modulation frequency and the depth values ​​derived by iToF using light with a low modulation frequency. The merge processing unit 653 outputs the merged depth values.

[0222] The present technology can be applied to such a distance measuring device 600. For example, the encoding device 100 described in the first embodiment or the third embodiment may be applied as the encoding unit 622. Furthermore, the decoding device 200 described in the second embodiment or the fourth embodiment may be applied as the decoding unit 651.

[0223] This makes it possible to suppress a decrease in the coding efficiency of the coded data transmitted from the high-frequency ranging unit 601 to the data merging unit 603. This makes it possible to suppress an increase in the bandwidth required to transmit depth values ​​from the high-frequency ranging unit 601 to the data merging unit 603, thereby suppressing an increase in costs. It is also possible to suppress an increase in circuit size and power consumption.

[0224] <How to supply wraparound values> The folding back value may be supplied from the encoding side to the decoding side as in the example shown in Fig. 24. For example, the folding back value may be supplied from the encoding unit 622 to the decoding unit 651. Alternatively, the folding back value may be supplied from the encoding unit 642 to the decoding unit 652.

[0225] 25, the folding value may be supplied from a processing unit (e.g., upper system controller 701) separate from the high-frequency ranging unit 601 and the low-frequency ranging unit 602. For example, the folding value may be supplied from the upper system controller 701 to the encoding unit 622 and the decoding unit 651. The folding value may also be supplied from the upper system controller 701 to the encoding unit 642 and the decoding unit 652.

[0226] <Distance measurement process flow> An example of the flow of distance measurement processing executed by the distance measuring device 600 will be described with reference to the flowchart of FIG.

[0227] When the distance measurement process is started, the optical output unit 611 of the high-frequency distance measurement unit 601 emits light with a high-frequency modulation frequency in step S601. The detection unit 612 detects the reflected light.

[0228] In step S602, the depth value derivation unit 621 derives a depth value by iToF using light with a high frequency modulation frequency.

[0229] In step S603, the encoding unit 622 executes an encoding process to encode the depth value. In step S604, the encoding unit 622 transmits the generated encoded data (encoded data of the depth value derived by iToF using light with a high-frequency modulation frequency) to the decoding unit 651 of the data merging unit 603. The decoding unit 651 acquires the encoded data.

[0230] In step S605, the decoding unit 651 executes a decoding process to decode the encoded data and generate (restore) a depth value derived by iToF using light with a high frequency modulation frequency.

[0231] In step S606, the optical output unit 631 of the low-frequency distance measuring unit 602 emits light with a low modulation frequency, and the detection unit 632 detects the reflected light.

[0232] In step S607, the depth value derivation unit 641 derives a depth value by iToF using light with a low modulation frequency.

[0233] In step S608, the encoding unit 642 executes an encoding process to encode the depth value. In step S609, the encoding unit 642 transmits the generated encoded data (encoded data of the depth value derived by iToF using light with a low modulation frequency) to the decoding unit 652 of the data merging unit 603. The decoding unit 652 acquires the encoded data.

[0234] In step S610, the decoding unit 652 executes a decoding process to decode the encoded data and generate (restore) a depth value derived by iToF using light with a low modulation frequency.

[0235] In step S611, the merge processing unit 653 merges the depth value derived by iToF using light with a high modulation frequency generated in step S606 with the depth value derived by iToF using light with a low modulation frequency generated in step S610.

[0236] In step S612, the merge processing unit 653 determines whether or not to end the ranging process. If it is determined not to end, the process returns to step S601, and the subsequent processes are repeated. Then, if it is determined in step S612 that the ranging process is to end, the ranging process ends.

[0237] The present technology can be applied to such distance measurement processing. For example, the encoding processing of step S603 may be the encoding processing described with reference to the flowchart of Fig. 12 or the encoding processing described with reference to the flowchart of Fig. 17. Furthermore, the decoding processing of step S605 may be the decoding processing described with reference to the flowchart of Fig. 15 or the decoding processing described with reference to the flowchart of Fig. 19.

[0238] This makes it possible to suppress a decrease in the coding efficiency of the coded data transmitted from the high-frequency ranging unit 601 to the data merging unit 603. This makes it possible to suppress an increase in the bandwidth required to transmit depth values ​​from the high-frequency ranging unit 601 to the data merging unit 603, thereby suppressing an increase in costs. It is also possible to suppress an increase in circuit size and power consumption.

[0239] <8. Notes> <Use of three or more modulation frequencies> In the above, an example of combining iToFs with two types of modulation frequencies, low frequency and high frequency, has been described as an example of combining iToFs with multiple modulation frequencies, but the combination is arbitrary and is not limited to this example. For example, iToFs with three or more types of modulation frequencies may be combined.

[0240] In this case, it is sufficient to prevent wraparound of the depth values ​​derived by iToF using light with the lowest modulation frequency. In other words, the depth values ​​derived by iToF using light with a higher modulation frequency may wrap around within their respective ranges (fold-over values). Therefore, by applying this technology, it is possible to suppress a decrease in the coding efficiency of these depth values.

[0241] <Prediction method> In the above, an example of DPCM (inverse DPCM) in which the data of the previous pixel to be processed is used as a predicted value has been described, but the prediction method is not limited to this example and may be any method. For example, as shown in FIG. 27, P2 prediction may be used.

[0242] In P2 prediction, as shown in A of FIG. 27, a predicted value of pixel value X of a pixel to be processed, which is shown in gray, is derived using pixel value A of the pixel to the left of the pixel to be processed, pixel value B of the pixel above the pixel to be processed, and pixel value C of the pixel to the upper left of the pixel to be processed. At this time, as shown in B of FIG. 27, a prediction method is set depending on whether pixel value C is greater than value D and value E. As shown in A of FIG. 27, the smaller of pixel value A and pixel value B is set to value D, and the larger of pixel value A and pixel value B is set to value E. Then, if pixel value C is smaller than value D, value E is set as the predicted value of pixel value X. Furthermore, if pixel value C is equal to or greater than value D and smaller than value E, (A+BC) is set as the predicted value of pixel value X. Furthermore, if pixel value C is equal to or greater than value E, value D is set as the predicted value of pixel value X.

[0243] When such P2 prediction is applied, it is possible to suppress a decrease in coding efficiency, as in the case of the above-mentioned DPCM.

[0244] Furthermore, a mechanism may be introduced that uses refinement transmitted in the bitstream to improve the accuracy of data obtained by inverse quantization.

[0245] <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0246] FIG. 28 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0247] In a computer 900 shown in FIG. 28, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0248] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0249] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0250] In a computer configured as above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0251] The program executed by the computer can be applied by recording it on removable media 921 such as package media, for example. In this case, the program can be installed in storage unit 913 via input / output interface 910 by inserting removable media 921 into drive 915.

[0252] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0253] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0254] <Configurations to which this technology can be applied> The present technology can be applied to any configuration, for example, various electronic devices.

[0255] Furthermore, for example, the present technology can also be implemented as a part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0256] Furthermore, for example, the present technology can also be applied to a network system configured by multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network.

[0257] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0258] <Fields and applications where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications thereof are also arbitrary.

[0259] <Other> The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0260] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0261] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0262] Also, for example, each step of a single flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0263] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0264] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0265] The present technology can also be configured as follows. (1) a first quantization unit that quantizes depth values ​​that wrap around a predetermined range using a predetermined quantization step; a difference value derivation unit that derives a difference value between the depth value quantized by the first quantization unit and a predicted value of the depth value; a second quantization unit that quantizes the foldback value of the depth value using the quantization step; a correction unit that appropriately corrects the difference value derived by the difference value derivation unit using the folding value quantized by the second quantization unit; an encoding unit that encodes the difference value appropriately corrected by the correction unit; An information processing device comprising: (2) The correction unit an adder that adds the quantized aliasing value to the difference value; a subtraction unit that subtracts the quantized aliasing value from the difference value; a selection unit that selects the difference value, the addition result of the difference value derived by the addition unit and the quantized aliasing value, and the subtraction result of the difference value derived by the subtraction unit and the quantized aliasing value, which has the smallest absolute value; Equipped with The encoding unit is configured to encode the difference value, the addition result, or the subtraction result selected by the selection unit. The information processing device described in (1). (3) The correction unit a selection unit that selects, based on the difference value and the quantized aliasing value, whether to omit correction of the difference value, add the quantized aliasing value to the difference value, or subtract the quantized aliasing value from the difference value; an adder that adds the quantized aliasing value to the difference value when the selector selects addition of the quantized aliasing value; a subtraction unit that subtracts the quantized aliasing value from the difference value when the selection unit selects subtraction of the quantized aliasing value; Equipped with The encoding unit When the selection unit selects to omit correction of the difference value, the difference value is encoded; When the selection unit selects addition of the quantized folding value, encoding the addition result of the difference value derived by the addition unit and the quantized folding value; When the selection unit selects subtraction of the quantized aliasing value, the subtraction unit encodes the subtraction result of the difference value derived from the subtraction unit and the quantized aliasing value. It is configured as follows: The information processing device described in (1). (4) the first quantization unit quantizes the depth values ​​using the quantization step set for each block of the depth values; The second quantization unit quantizes the foldback value using the quantization step set for each block of the depth value. An information processing device according to any one of (1) to (3). (5) The second quantization unit quantizes the folding value supplied from a system controller external to the information processing device. An information processing device according to any one of (1) to (4). (6) The depth value indicates the distance to the subject, which is derived for each pixel by an indirect ToF (Time-of-Flight) method. An information processing device according to any one of (1) to (5). (7) The depth value indicates the distance to the subject, which is derived for each pixel by irradiating the object with light of a modulation frequency that narrows the range of distance measurements possible compared to the range of values ​​that the distance to the subject can take, and detecting reflected light. (6) An information processing device according to the present invention. (8) The difference value derivation unit uses the depth value of a previous pixel to be processed, which is a pixel processed immediately before, as the predicted value of a current pixel to be processed, which is a pixel currently to be processed, and derives a difference value between the depth value of the current pixel to be processed and the depth value of the previous pixel to be processed. (7) An information processing device according to (7). (9) The encoding unit stores, in a storage unit, encoded data derived by encoding the difference value appropriately corrected by the correction unit. An information processing device according to any one of (1) to (8). (10) quantizing the depth values ​​that wrap around a predetermined range using a predetermined quantization step; deriving a difference between the quantized depth value and a predicted value of the depth value; quantizing the foldback values ​​of the depth values ​​using the quantization step; Using the quantized folding value, appropriately correct the derived difference value; Encoding the appropriately corrected difference value Information processing methods.

[0266] (11) A decoding unit that decodes coded data to derive a difference value between coefficient data relating to a depth value that wraps around within a predetermined range and a predicted value of the coefficient data; a coefficient data derivation unit that derives the coefficient data by adding the predicted value to the difference value derived by the decoding unit; a quantization unit that quantizes the foldback value of the depth value using a predetermined quantization step; a correction unit that appropriately corrects the coefficient data derived by the coefficient data derivation unit using the folding value quantized by the quantization unit; an inverse quantization unit that derives the depth value by inverse quantizing the coefficient data appropriately corrected by the correction unit using the quantization step; An information processing device comprising: (12) The correction unit an adder that adds the quantized folding value to the coefficient data; a subtraction unit that subtracts the quantized aliasing value from the coefficient data; a selection unit that selects, within the range, the coefficient data, the addition result of the coefficient data and the quantized aliasing value derived by the addition unit, and the subtraction result of the coefficient data and the quantized aliasing value derived by the subtraction unit; Equipped with The inverse quantization unit is configured to inverse quantize the coefficient data, the addition result, and the subtraction result selected by the selection unit. (11) An information processing device according to (11). (13) The correction unit a selection unit that selects, based on the coefficient data and the quantized aliasing value, whether to omit correction of the coefficient data, add the quantized aliasing value to the coefficient data, or subtract the quantized aliasing value from the coefficient data; an adder that adds the quantized aliasing value to the coefficient data when the selector selects addition of the quantized aliasing value; a subtraction unit that subtracts the quantized aliasing value from the coefficient data when the selection unit selects subtraction of the quantized aliasing value; Equipped with The inverse quantization unit When the selection unit selects to omit the correction of the coefficient data, the coefficient data is dequantized; When the selection unit selects the addition of the quantized aliasing value, the addition unit dequantizes the result of the addition of the coefficient data and the quantized aliasing value derived by the addition unit; When the selection unit selects subtraction of the quantized aliasing value, the subtraction unit dequantizes the subtraction result of the coefficient data derived from the quantized aliasing value. It is configured as follows: (11) An information processing device according to (11). (14) The quantization unit quantizes the aliasing value using the quantization step set for each block of the depth value, The inverse quantization unit inversely quantizes the coefficient data appropriately corrected by the correction unit, using the quantization step set for each block of the depth values. An information processing device according to any one of (11) to (13). (15) The quantization unit quantizes the folding value supplied from a system controller external to the information processing device or from an encoding unit that generated the encoded data. An information processing device according to any one of (11) to (14). (16) The depth value indicates the distance to the subject, which is derived for each pixel by an indirect ToF (Time-of-Flight) method. An information processing device according to any one of (11) to (15). (17) The depth value indicates the distance to the subject, which is derived for each pixel by irradiating the object with light of a modulation frequency such that the range of distance measurement is narrower than the range of values ​​that the distance to the subject can take, and detecting reflected light. (16) An information processing device according to (16). (18) The coefficient data derivation unit uses the coefficient data of a previous pixel to be processed, which is a pixel processed immediately before, as the predicted value of a current pixel to be processed, which is a pixel currently to be processed, and derives the coefficient data by adding the depth value of the previous pixel to be processed to the depth value of the current pixel to be processed. (17) An information processing device according to (17). (19) The decoding unit reads out the encoded data stored in the storage unit and decodes it. An information processing device according to any one of (11) to (18). (20) Decoding the coded data to derive a difference value between coefficient data relating to a depth value that wraps around within a predetermined range and a predicted value of the coefficient data; deriving the coefficient data by adding the predicted value to the derived difference value; quantizing the foldback depth values ​​using a predetermined quantization step; Using the quantized folding values, appropriately correct the derived coefficient data; deriving the depth values ​​by dequantizing the appropriately corrected coefficient data using the quantization step; Information processing methods. [Explanation of symbols]

[0267] 100 Encoding device, 111 and 112 Quantization unit, 113 DPCM processing unit, 114 and 115 Calculation unit, 116 Selection control unit, 117 Selection unit, 118 Encoding unit, 121 Correction unit, 200 Decoding device, 211 Quantization unit, 212 Decoding unit, 213 Inverse DPCM processing unit, 214 and 215 Calculation unit, 216 Selection control unit, 217 Selection unit, 218 Inverse quantization unit, 221 Correction unit, 500 Distance measuring device, 501 Control unit, 511 Optical output unit, 512 Detection unit, 513 iToF measurement unit, 514 RAM, 515 Data merge unit, 521 Depth value derivation unit, 522 Encoding unit, 531 Decoding unit, 532 Merge processing unit, 600 distance measuring device, 601 high frequency distance measuring unit, 602 low frequency distance measuring unit, 603 data merge unit, 611 optical output unit, 612 detection unit, 613 iToF measurement unit, 621 depth value derivation unit, 622 encoding unit, 631 optical output unit, 632 detection unit, 633 iToF measurement unit, 641 depth value derivation unit, 642 encoding unit, 651 decoding unit, 652 decoding unit, 653 merge processing unit, 701 upper system controller, 900 computer< / itof>

Claims

1. a first quantization unit that quantizes depth values ​​that wrap around a predetermined range using a predetermined quantization step; a difference value derivation unit that derives a difference value between the depth value quantized by the first quantization unit and a predicted value of the depth value; a second quantization unit that quantizes the foldback value of the depth value using the quantization step; a correction unit that appropriately corrects the difference value derived by the difference value derivation unit using the folding value quantized by the second quantization unit; an encoding unit that encodes the difference value appropriately corrected by the correction unit; An information processing device comprising:

2. The correction unit an adder that adds the quantized aliasing value to the difference value; a subtraction unit that subtracts the quantized aliasing value from the difference value; a selection unit that selects the difference value, the addition result of the difference value derived by the addition unit and the quantized aliasing value, and the subtraction result of the difference value derived by the subtraction unit and the quantized aliasing value, which has the smallest absolute value; Equipped with The encoding unit is configured to encode the difference value, the addition result, or the subtraction result selected by the selection unit. The information processing device according to claim 1 .

3. The correction unit a selection unit that selects, based on the difference value and the quantized aliasing value, whether to omit correction of the difference value, add the quantized aliasing value to the difference value, or subtract the quantized aliasing value from the difference value; an adder that adds the quantized aliasing value to the difference value when the selector selects addition of the quantized aliasing value; a subtraction unit that subtracts the quantized aliasing value from the difference value when the selection unit selects subtraction of the quantized aliasing value; Equipped with The encoding unit When the selection unit selects to omit correction of the difference value, the difference value is encoded; When the selection unit selects addition of the quantized folding value, encoding the addition result of the difference value derived by the addition unit and the quantized folding value; When the selection unit selects subtraction of the quantized aliasing value, the subtraction unit encodes the subtraction result of the difference value derived from the subtraction unit and the quantized aliasing value. It is configured as follows: The information processing device according to claim 1 .

4. the first quantization unit quantizes the depth values ​​using the quantization step set for each block of the depth values; The second quantization unit quantizes the foldback value using the quantization step set for each block of the depth value. The information processing device according to claim 1 .

5. The second quantization unit quantizes the folding value supplied from a system controller external to the information processing device. The information processing device according to claim 1 .

6. The depth value indicates the distance to the subject, which is derived for each pixel by an indirect ToF (Time-of-Flight) method. The information processing device according to claim 1 .

7. The depth value indicates the distance to the subject, which is derived for each pixel by irradiating the object with light of a modulation frequency such that the range of distance measurement is narrower than the range of values ​​that the distance to the subject can take, and detecting reflected light. The information processing device according to claim 6 .

8. The difference value derivation unit uses the depth value of a previous pixel to be processed, which is a pixel processed one pixel previously, as the predicted value of a current pixel to be processed, which is a pixel currently to be processed, and derives a difference value between the depth value of the current pixel to be processed and the depth value of the previous pixel to be processed. The information processing device according to claim 7 .

9. The encoding unit stores, in a storage unit, encoded data derived by encoding the difference value appropriately corrected by the correction unit. The information processing device according to claim 1 .

10. quantizing depth values ​​that wrap around a predetermined range using a predetermined quantization step; deriving a difference between the quantized depth value and a predicted value of the depth value; quantizing the foldback values ​​of the depth values ​​using the quantization step; Using the quantized folding value, appropriately correct the derived difference value; Encoding the appropriately corrected difference value Information processing methods.

11. a decoding unit that decodes the coded data to derive a difference value between coefficient data relating to a depth value that wraps around within a predetermined range and a predicted value of the coefficient data; a coefficient data derivation unit that derives the coefficient data by adding the predicted value to the difference value derived by the decoding unit; a quantization unit that quantizes the foldback value of the depth value using a predetermined quantization step; a correction unit that appropriately corrects the coefficient data derived by the coefficient data derivation unit using the folding value quantized by the quantization unit; an inverse quantization unit that derives the depth value by inverse quantizing the coefficient data appropriately corrected by the correction unit using the quantization step; An information processing device comprising:

12. The correction unit an adder that adds the quantized folding value to the coefficient data; a subtraction unit that subtracts the quantized aliasing value from the coefficient data; a selection unit that selects, within the range, the coefficient data, the addition result of the coefficient data and the quantized aliasing value derived by the addition unit, and the subtraction result of the coefficient data and the quantized aliasing value derived by the subtraction unit; Equipped with The inverse quantization unit is configured to inverse quantize the coefficient data, the addition result, and the subtraction result selected by the selection unit. The information processing device according to claim 11.

13. The correction unit a selection unit that selects, based on the coefficient data and the quantized aliasing value, whether to omit correction of the coefficient data, add the quantized aliasing value to the coefficient data, or subtract the quantized aliasing value from the coefficient data; an adder that adds the quantized aliasing value to the coefficient data when the selector selects addition of the quantized aliasing value; a subtraction unit that subtracts the quantized aliasing value from the coefficient data when the selection unit selects subtraction of the quantized aliasing value; Equipped with The inverse quantization unit When the selection unit selects to omit the correction of the coefficient data, the coefficient data is dequantized; When the selection unit selects the addition of the quantized aliasing value, the addition unit dequantizes the result of the addition of the coefficient data and the quantized aliasing value derived by the addition unit; When the selection unit selects subtraction of the quantized aliasing value, the subtraction unit dequantizes the subtraction result of the coefficient data derived from the quantized aliasing value. It is configured as follows: The information processing device according to claim 11.

14. the quantization unit quantizes the aliasing value using the quantization step set for each block of the depth value; The inverse quantization unit inversely quantizes the coefficient data appropriately corrected by the correction unit, using the quantization step set for each block of the depth values. The information processing device according to claim 11.

15. The quantization unit quantizes the folding value supplied from a system controller external to the information processing device or from an encoding unit that generated the encoded data. The information processing device according to claim 11.

16. The depth value indicates the distance to the subject, which is derived for each pixel by an indirect ToF (Time-of-Flight) method. The information processing device according to claim 11.

17. The depth value indicates the distance to the subject, which is derived for each pixel by irradiating the object with light of a modulation frequency such that the range of distance measurement is narrower than the range of values ​​that the distance to the subject can take, and detecting reflected light. The information processing device according to claim 16.

18. The coefficient data derivation unit uses the coefficient data of a previous pixel to be processed, which is a pixel processed one pixel previously, as the predicted value of a current pixel to be processed, which is a pixel currently to be processed, and derives the coefficient data by adding the depth value of the previous pixel to the depth value of the current pixel to be processed. The information processing device according to claim 17.

19. The decoding unit reads out and decodes the encoded data stored in the storage unit. The information processing device according to claim 11.

20. Decoding the coded data to derive a difference value between coefficient data relating to a depth value that wraps around within a predetermined range and a predicted value of the coefficient data; deriving the coefficient data by adding the predicted value to the derived difference value; quantizing the foldback depth values ​​using a predetermined quantization step; Using the quantized folding values, appropriately correct the derived coefficient data; deriving the depth values ​​by dequantizing the appropriately corrected coefficient data using the quantization step; Information processing methods.

Images