Information processing device, solid-state imaging device, display device, and transmission / reception system

Abstract

[Problem] To appropriately add metadata to an image. [Solution] This information processing device is provided with a processing circuit. The processing circuit acquires image data, embeds a reference value in pixels arranged in a first region of a region indicating an image of the image data, embeds encoded metadata in pixels arranged in a second region of the region indicating the image of the image data, the encoded metadata being obtained by converting metadata using the reference value, and outputs the image data in which the reference value and the value of the encoded metadata are embedded.

Description

Information processing device, solid-state imaging device, display device, and transmission / reception system

[0001] The present disclosure relates to an information processing device, a solid-state imaging device, a display device, and a transmission / reception system.

[0002] When transmitting and receiving image and video signals, it is sometimes necessary to transmit additional information other than the image or video data being transmitted and received as metadata, such as frame count, frame ID, VRR (Variable Refresh Rate) delay, register, and timestamp.

[0003] Metadata transmission involves I 2 There are two possible methods for transmitting metadata: one is to use general-purpose control signals such as C (Inter-Integrated Circuit) and SPI (Serial Peripheral Interface), and the other is to embed the metadata in the blanking intervals between data. When using general-purpose control signals, it is difficult to transmit metadata synchronized with the video signal due to the increased bandwidth required for higher resolution and frame rates. Furthermore, when embedding metadata in the blanking intervals, it is often difficult to actually embed the data due to various data formats, etc.

[0004] For these reasons, one possible approach is to embed metadata as pixel values ​​in image data and restore the metadata from the pixel values ​​on the receiving side. However, this may not always be possible due to signal changes during transmission and reception, encoding on the transmitting side, and decoding or filtering on the receiving side.

[0005] Special Publication No. 2021-525382

[0006] Therefore, one non-limiting problem that the embodiments of the present disclosure aim to solve is to appropriately add metadata to images to be transmitted and received. The problem that the embodiments of the present disclosure aim to solve can also be, as some further non-limiting examples, a problem corresponding to the effects described in the embodiments. In other words, a problem that corresponds to at least one of the effects described in the description of the embodiments of the present disclosure can be considered to be a problem that the present disclosure aims to solve.

[0007] According to one embodiment, an information processing device includes a processing circuit that acquires image data, embeds reference values ​​in pixels arranged in a first region of an area representing an image of the image data, embeds encoded metadata obtained by converting metadata using the reference values ​​in pixels arranged in a second region of the area representing the image of the image data, and outputs the image data in which the reference values ​​and values ​​of the encoded metadata have been embedded.

[0008] The reference value may be a value indicating a value in a table that defines conversion between pixel values ​​in the image data and values ​​of the metadata.

[0009] The reference value may be defined as a combination of a pixel value and a value of a predetermined number of bits.

[0010] The processing circuit may generate a bit string by dividing the metadata into the predetermined number of bits, may convert the bit string into a converted pixel value based on the reference value, and may set the pixel value of the pixel placed in the second area to the converted pixel value.

[0011] The processing circuit may assign a different pixel value to each of the binary representations of the predetermined number of bits, and may use the assigned pixel values ​​as the reference values ​​and embed the binary representations in the first region in descending or ascending order.

[0012] The processing circuit may generate a binary representation bit string by dividing the binary representation of the metadata into the predetermined number of bits, and may convert the binary representation bit string using the reference value to generate the encoded metadata.

[0013] The predetermined number of bits may be 4 bits, and the processing circuit may set pixel values ​​corresponding to each of the 4-bit binary values ​​0000b, 0001b, 0010b, 0011b, 0100b, 0101b, 0110b, 0111b, 1000b, 1001b, 1010b, 1011b, 1100b, 1101b, 1110b, and 1111b, and may sequentially set pixel values ​​corresponding to each of the 4-bit values ​​in the first region and embed them as the reference value. Of course, the number of bits may be less than 2 bits, or more than 5 bits, or the like.

[0014] The processing circuit may set the order of magnitude of pixel values ​​of the reference values ​​to be a predetermined order with respect to the binary representation.

[0015] The processing circuitry may define the reference values ​​for the attributes of the metadata in the predetermined order.

[0016] The processing circuit may set pixel values ​​in the first region or the second region to include at least one of pixels in the first region having pixel values ​​that are a predetermined value separate from pixels having pixel values ​​that indicate the reference value, or pixels in the second region having pixel values ​​that are a predetermined value separate from pixels having pixel values ​​that indicate the encoding metadata.

[0017] The processing circuit may set pixel values ​​in the first region or the second region to include at least one of pixels in the first region that have pixel values ​​in the image data separate from pixels having pixel values ​​indicating the reference value, or pixels in the second region that have pixel values ​​in the image data separate from pixels having pixel values ​​indicating the encoding metadata.

[0018] The first region and the second region may be provided in a first predetermined row.

[0019] The first region may be provided in a first predetermined row, and the second region may be provided in a second predetermined row.

[0020] The first region and the second region for a first predetermined row, and the first region and the second region for a second predetermined row may be provided for each row.

[0021] At least one of the reference value and the encoded metadata may be embedded as a value of a pixel group that groups together a plurality of pixels.

[0022] The information processing device may further include a memory unit, and the processing circuit may acquire the image data stored in the memory unit and embed the reference value and the encoded metadata value in the image data.

[0023] According to one embodiment, a solid-state imaging device includes any one of the information processing devices described above and an imaging unit, wherein the information processing device acquires an image captured by the imaging unit as the image data, and embeds the reference value and the value of the encoded metadata in the image data.

[0024] The information processing device and the imaging unit may be provided in the same semiconductor chip.

[0025] According to one embodiment, an information processing device includes a processing circuit that acquires image data, reads reference values ​​from pixels arranged in a first region of an area representing an image of the image data, reads encoded metadata from pixels arranged in a second region of the area representing the image of the image data, and acquires metadata by converting the encoded metadata using the reference values.

[0026] According to one embodiment, a display device includes a receiving unit, the information processing device, and a display unit, wherein the receiving unit receives transmitted data as the image data, the processing circuit acquires the metadata from the received image data, and displays the image data on the display unit.

[0027] The processing circuit may perform signal processing on the image data based on the metadata, and may display the signal-processed image data on the display unit.

[0028] According to one embodiment, a transmission / reception system includes a first information processing device and a second information processing device, wherein the first information processing device acquires image data, embeds reference values ​​in pixels arranged in a first area of ​​an area representing an image of the image data, embeds encoded metadata obtained by converting metadata using the reference values ​​in pixels arranged in a second area of ​​the area representing the image of the image data, and transmits the image data in which the reference values ​​and values ​​of the encoded metadata are embedded, and the second information processing device receives the image data transmitted from the first information processing device, reads the reference values ​​from pixels arranged in the first area of ​​the area representing the image of the image data, reads the encoded metadata from pixels arranged in the second area of ​​the area representing the image of the image data, and acquires metadata obtained by converting the encoded metadata using the reference values.

[0029] According to one embodiment, a program causes a processor to perform at least some of the processes described above, and the program may be stored on a temporary or non-transitory computer-readable medium.

[0030] FIG. 1 is a diagram showing an example of an encoder (information processing device) according to an embodiment. FIG. 1 is a diagram showing an example of a decoder (information processing device) according to an embodiment. FIG. 1 is a diagram showing an example of encoded metadata generation according to an embodiment. FIG. 1 is a diagram showing an example of encoded metadata generation according to an embodiment. FIG. 2 is a diagram showing an example of metadata embedding according to an embodiment. FIG. 3 is a diagram showing an example of metadata embedding according to an embodiment. FIG. 4 is a flowchart showing processing of an information processing device according to an embodiment. FIG. 5 is a flowchart showing processing of an information processing device according to an embodiment. FIG. 6 is a diagram showing an example of metadata embedding according to an embodiment. FIG. 7 is a diagram showing an example of metadata embedding according to an embodiment. FIG. 8 is a diagram showing an example of metadata embedding according to an embodiment. FIG. 9 is a diagram showing an example of metadata embedding according to an embodiment. FIG. 10 is a diagram showing an example of metadata embedding according to an embodiment. FIG. 11 is a diagram showing an example of metadata embedding according to an embodiment. FIG. 12 is a diagram showing an example of metadata embedding according to an embodiment. FIG. 13 is a diagram showing an example of metadata embedding according to an embodiment. FIG. 14 is a diagram showing an example of an electronic device according to an embodiment. FIG. 15 is a diagram showing an example of an electronic device according to an embodiment. FIG. 16 is a diagram showing an example of an electronic device according to an embodiment. FIG. 17 is a diagram showing an example of a system according to an embodiment.

[0031] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The drawings are used for explanation purposes, and the shape, size, and size ratio of each component in an actual device do not necessarily have to be the same as those shown in the drawings. Furthermore, since the drawings are simplified, components necessary for implementation other than those shown in the drawings are also assumed to be appropriately provided.

[0032] In some embodiments, this disclosure describes a device (method) for embedding metadata in image data, a device (method) for reading metadata embedded in image data, and electronic devices and systems using these devices. In this disclosure, an image can be understood as a single piece of data that can be extracted as an image along the time axis in a video. In other words, when simply referred to as image data, it can be understood as part of the video data.

[0033] First, a configuration common to several embodiments of an apparatus for implementing embedding of metadata into image data and reading of metadata from image data according to the present disclosure will be described.

[0034] 1 is a diagram illustrating an example of an information processing device that operates as an encoder for embedding metadata in image data according to an embodiment. The encoder 10 (information processing device) includes an image acquisition circuit 100, a pre-processing circuit 102, a processing circuit 104, a post-processing circuit 106, and an output circuit 108.

[0035] The encoder 10 is an information processing device that acquires image data, converts the image into an appropriate format (and performs appropriate signal and / or image processing, such as compression, as necessary), and embeds metadata in the image. The encoder 10 can also be incorporated into another device and configured to cooperate with the other device as an encoding module or encoding processing unit, or can be configured as an integral part of the other device.

[0036] Although not shown, the encoder 10 may also include other circuits necessary for operation, such as a memory circuit, a control circuit, and a power supply circuit, either internally or externally. All of the circuits may be configured as dedicated circuits, such as circuits including an ASIC (Application Specific Integrated Circuit) or a CMOS sensor (Complementary Metal-Oxide-Semiconductor sensor), or may be configured with a general-purpose processor or the like.

[0037] The image acquisition circuit 100 acquires data relating to an image. The data relating to an image may be the image data itself or raw data before being converted into an image.

[0038] The pre-processing circuit 102 to the post-processing circuit 106 may be implemented as a single processing circuit if necessary. That is, these circuits are shown as separate circuits for convenience, and it is not necessary for each circuit to be configured as a separate circuit in a specific implementation.

[0039] The pre-processing circuit 102 converts the data acquired by the image acquisition circuit 100 into an image data format into which the processing circuit 104 can embed metadata. For example, if the image acquisition circuit 100 acquires raw data about an image, the pre-processing circuit 102 converts this signal into digital image data in a format suitable for the processing circuit 104 to embed data. For example, if the image acquisition circuit 100 acquires digital image data, the pre-processing circuit 102 converts this digital image data into a format suitable for the processing circuit 104 to embed data. The format relates to various image storage formats, such as RGB format, Yuv format, etc.

[0040] Additionally, if the data acquired by the image acquisition circuitry 100 is video data, the pre-processing circuitry 102 may also acquire image data as any part of the video data.

[0041] The processing circuit 104 embeds metadata in the encoder 10. The processing circuit 104 encodes and embeds metadata into the image data acquired by the preprocessing circuit 102. The processing circuit 104 encodes and embeds the metadata in an area indicating the pixel value of a pixel of the image data.

[0042] More specifically, the processing circuit 104 embeds reference values, which are data for encoding metadata, in a first region of the image, and embeds the metadata encoded by the reference values ​​in a second region of the image. The processing circuit 104 embeds the reference values ​​and the encoded metadata as pixel values ​​of pixels that make up the image.

[0043] The post-processing circuit 106 converts the image data in which the processing circuit 104 has embedded metadata into a format suitable for output (transmission). The post-processing circuit 106 can also perform any signal processing on the image data. For example, when the encoder 10 processes a single image for transmission, the post-processing circuit 106 can perform encoding processing such as compression and encryption on the image in which the processing circuit 104 has embedded data. In this case, the post-processing circuit 106 can, as another example, apply additional information such as error detection and error correction to the image data in which the metadata has been embedded to ensure proper decoding.

[0044] This encoding process makes it possible to obtain the encoded metadata as pixel values ​​on the decoding side as well, making it possible to obtain the metadata appropriately.

[0045] The output circuit 108 outputs the image data processed by the post-processing circuit 106 or data including the image data. The output circuit 108 may output the data to, for example, an internal or external storage of the encoder 10, may transmit the data over the Internet via a predetermined output interface, or may transmit the data via any communication means or broadcasting means such as a broadcast.

[0046] 2 is a diagram showing an example of an information processing device that operates as a decoder that reads metadata from image data according to one embodiment. The decoder 20 (information processing device) includes an image acquisition circuit 200, a pre-processing circuit 202, a processing circuit 204, a post-processing circuit 206, and an output circuit 208. The decoder 20 is an information processing device that acquires image data into which metadata has been embedded by the encoder 10 and executes processing to extract the metadata from the image data. The decoder 20 can also be incorporated into another device and configured to cooperate with the other device as a decoding module or decoding processing unit, or to be configured as an integral part of the other device.

[0047] The decoder 20 may also include internal or external circuits necessary for the operation of the memory circuit, control circuit, power supply circuit, etc. All circuits may be configured with dedicated circuits including ASICs, circuits containing light-emitting elements such as LEDs (Light Emitting Diodes), or may be configured with general-purpose processors, etc.

[0048] The image acquisition circuit 200 acquires data including at least the image data output by the output circuit 108. That is, the image acquisition circuit 200 acquires data including at least the image data into which the encoded metadata has been embedded by the encoder 10.

[0049] The pre-processing circuit 202 to the post-processing circuit 206 may be implemented as a single processing circuit if necessary, similar to the pre-processing circuit 102 to the post-processing circuit 106 in the encoder 10. In other words, these circuits are shown as separate circuits for convenience, and in a specific implementation, each circuit does not necessarily have to be configured as a separate circuit.

[0050] The pre-processing circuit 202 converts (decodes) the data acquired by the image acquisition circuit 200 into an image data format from which the processing circuit 204 can read metadata. That is, the pre-processing circuit 202 uses the format of the image data encoded by the encoder 10 to convert the data into a format from which pixel values ​​in a format with embedded metadata can be read, thereby acquiring image data from which the metadata can be extracted. Similar to the processing of the pre-processing circuit 102, the pre-processing circuit 202 can also acquire image data as part of video data.

[0051] The processing circuit 204 extracts the encoded metadata embedded in the image and decodes the encoded metadata to obtain the metadata.

[0052] More specifically, the processing circuit 204 obtains the metadata by reading pixel values ​​that are reference values ​​from a first region in the image, reading pixel values ​​that are encoded metadata from a second region in the image, and decoding the encoded metadata using the reference values.

[0053] The post-processing circuit 206 processes the image data from which the processing circuit 204 extracted metadata (image data decoded by the pre-processing circuit 202) appropriately for output format. For example, the post-processing circuit 206 converts the data from the processing circuit 204 into a signal suitable for controlling display and outputs it via the output circuit 208.

[0054] The post-processing circuit 206 can also process the image to be output using the metadata acquired by the processing circuit 204. That is, the post-processing circuit 206 can use the metadata acquired by the processing circuit 204 to perform signal processing or image processing on the image data decoded by the pre-processing circuit 202. The post-processing circuit 206 can also use the metadata acquired by the processing circuit 204 to generate and output a control signal related to the image data to be output.

[0055] The output circuit 208 outputs data including the image data processed by the post-processing circuit 206. The output circuit 208 can also output the metadata extracted by the processing circuit 204. The output circuit 208 can output the image data as data, or the metadata together with the image data, to, for example, an internal or external storage of the decoder 20.

[0056] 2 shows an example in which image data is output from the output circuit 208. The output circuit 208 can output only image data. Alternatively, as a different example, the output circuit 208 can output metadata in some form.

[0057] For example, the output circuit 208 may output image data in which the post-processing circuit 206 has reflected some parameter of the image based on the metadata extracted by the processing circuit 204. In this case, the output circuit 208 does not need to output the metadata. As another example, if it is desired to obtain metadata as output from the decoder 20, the output circuit 208 may output the metadata together with the image data. Of course, the output circuit 208 may also obtain and output only the metadata from the encoded image data.

[0058] As described above, with the above implementation, even if degradation of the image signal occurs during image generation or transfer, appropriate metadata can be extracted by using the reference value. For example, since the reference value and metadata are stored as a color code, metadata can be reproduced in a way that is robust against signal degradation due to image conversion.

[0059] Next, an example of encoding metadata in the present disclosure will be described.

[0060] 3 is a diagram showing a non-limiting example of encoding metadata generation according to an embodiment. In the following figures and tables, decimal representations are expressed by simply listing the numerical values, and binary representations are expressed by adding the suffix "b" after the numerical values.

[0061] The first table shows an example of the metadata values ​​to be embedded. For example, the conversion will be explained when embedding the values ​​103 as Data 1 and 13 as Data 2 as metadata. These values ​​are expressed in binary as 01100111b and 00001101b, respectively.

[0062] The second table defines the reference values, which means that 00b in the binary representation of the metadata is represented by pixel value 64, 01b by 96, 10b by 128, and 11b by 160.

[0063] In this way, the reference value can be thought of as a value indicating a look-up table (LUT) that converts a binary value of a predetermined number of bits (2 bits in FIG. 3) into a pixel value. The processing circuit 104 embeds values ​​indicating this conversion in a first region in the image.

[0064] The third table shows an example of encoded metadata, which means that the binary representation of the metadata is converted into a pixel value representation according to the LUT indicated by the reference value.

[0065] The binary-represented metadata is divided into a predetermined number of bits, for example, every 2 bits. The divided metadata is rearranged, for example, in little endian order. For example, since the binary representation of data 1 is "01100111b," the divided representation of data 1 is expressed as "11b, 01b, 10b, 01b."

[0066] The coded metadata is expressed as a number obtained by converting the binary value of the divided representation into a pixel value using a reference value. For example, the first data of Data 1 is a value of 160, which is obtained by converting 11b using a reference value. In other words, Data 1 can be expressed as coded metadata as "160, 96, 128, 96." Similarly, Data 2 is divided into "01b, 11b, 00b, 00b" and coded as "96, 128, 64, 64."

[0067] In summary, the processing circuit 104 generates encoded metadata by dividing the binary representation of the metadata into a predetermined number of bits, converting the divided binary values ​​using a reference value, and assigning pixel values ​​to each of the binary values. The processing circuit 104 can embed the metadata into the image data by setting the pixel values ​​of the pixels in the second region to the assigned pixel values ​​as encoded metadata.

[0068] In the above, the metadata embedding was done in little endian format, but big endian format can also be used. The fourth table shows the same conversion as above but in big endian format. You can also use this table to embed the metadata.

[0069] These methods may be determined based on the hardware architecture of either the encoder 10 that embeds the metadata or the decoder 20 that extracts the metadata. In particular, it is considered that a method suitable for combining the divided metadata in binary representation in the decoder 20 is used.

[0070] Of course, these are given as non-limiting examples, and the embodiments of the present disclosure can be applied to other arrangements as well.

[0071] In FIG. 3, an example has been described in which the metadata is divided into two bits as the predetermined number of bits, but the present invention is not limited to this.

[0072] 4 is a diagram showing an example of generation of encoded metadata according to an embodiment. As shown in this diagram, encoded metadata may be generated with the predetermined number of bits set to 3 bits.

[0073] The reference value indicates the pixel value to which the 3-bit binary representation is converted. For example, the binary representation of the metadata is divided into a predetermined number of bits starting from the LSB (Least Significant Bit) to obtain a divided representation. If the number of most significant bits is less than the predetermined number, zeros may be added for padding. The reference value is used to convert this divided representation into a pixel value representation, thereby generating the coded metadata.

[0074] When pixel values ​​are expressed in 8 bits as in Figures 3 and 4, numeric data up to 255 can be supported, but this is not limited to this. Numeric data can be expressed as binary data according to common coding such as little endian, and encoded metadata can be generated. When the number of bits used to describe pixel values ​​is different, the range of numeric data changes accordingly.

[0075] 3 and 4 show the case where the metadata is a numerical value, but the present invention is not limited to this. Character strings can also be converted using a general code. For example, it is possible to define encoded metadata in the same way as above for any character string expressed in a format such as JIS code or Unicode.

[0076] As shown in Figures 3 and 4, the predetermined number of bits into which the metadata is divided may be any number of bits. Also, the reference values ​​may be assigned to have equal tonal differences (every 32) as in Figure 3, or may be assigned to have different tonal differences as in Figure 4.

[0077] By dividing the data into a predetermined number of bits and assigning pixel values ​​to each binary string that serves as a reference for expressing an arbitrary value, the metadata can be appropriately encoded. It is desirable for the reference values ​​to have a certain degree of gradation difference so that they can be appropriately converted into a representation of the predetermined number of bits during decoding. Providing a gradation difference can prevent confusion with other pixel values ​​during decoding.

[0078] Furthermore, although a binary representation is used in the above description, the present invention is not limited to this. For example, the metadata may be converted into a representation other than binary and divided, and the same processing as above may be performed.

[0079] First Embodiment Several embodiments of embedding reference values ​​and metadata in an image will now be described.

[0080] 5 is a diagram showing an example of metadata embedding according to an embodiment. As an example, not limited to this, shown by the diagonal lines in FIG. 5, the encoder 10 can embed metadata in the upper area of ​​the image data. Note that the row to embed the metadata can be set arbitrarily, but since the metadata is embedded in the image data, it is desirable to place it in a position that is difficult for the user to see.

[0081] 6 shows an example of the arrangement of reference values ​​and encoded metadata. In this example, the reference values ​​and encoded metadata are embedded in one line. The processing circuit 104 embeds the metadata in, for example, an arbitrary line (e.g., the top line) in the image data.

[0082] The processing circuit 104 first generates a bit string by dividing the metadata into a predetermined number of bits, as shown in the example of Figure 3, converts this bit string into a pixel value based on a reference value, and uses this pixel value as the pixel value of a pixel located within a predetermined area, thereby encoding the metadata and embedding the encoded metadata in the image.

[0083] In a row where data is to be embedded, the processing circuit 104 first sets a first region where a reference value is to be embedded. The first region can be set at any position within the row. For example, the processing circuit 104 locates a pixel or pixel column having a pixel value of the reference value start flag F0 and embeds a flag indicating the start of the first region. The reference value start flag F0 may be expressed, for example, by a bit having a predetermined value, or by a predetermined value represented by multiple bits.

[0084] The processing circuit 104 embeds a reference value in the first region defined by the reference value start flag F0. First, the processing circuit 104 assigns a pixel value as a reference value to each binary representation. For example, the processing circuit 104 embeds a reference value expressed in 2 bits by assigning the pixels belonging to the first region a pixel value (reference value) corresponding to 00b, a pixel value corresponding to 01b, a pixel value corresponding to 10b, and a pixel value corresponding to 11b, in that order.

[0085] As described above, the reference value may associate a 2-bit value with a pixel value, or may associate a value of 3 or more bits with a pixel value. For example, if the reference value associates a 4-bit value with a pixel value, the binary values ​​are 16 values ​​(or values ​​less than 16 values) of 0000b, 0001b, 0010b, 0011b, 0100b, 0101b, 0110b, 0111b, 1000b, 1001b, 1010b, 1011b, 1100b, 1101b, 1110b, and 1111b, and the reference value is expressed as information linking these values ​​with pixel values.

[0086] In a broad sense, the processing circuit 104 assigns a different pixel value to each binary representation of a predetermined number of bits, and embeds the reference values ​​in ascending order of the binary representation (in the order of 00b, 01b, 10b, 11b) using the assigned pixel values ​​as reference values. Note that the reference values ​​may be embedded in descending order instead of ascending order.

[0087] The processing circuit 104 embeds a data start flag F1 from the pixel next to the last pixel of the first region. The data start flag F1 can be defined and embedded in the same way as the reference value start flag F0. This data start flag F1 defines the start position of the second region.

[0088] The processing circuit 104 embeds the encoded metadata in the second region based on the reference value. The processing circuit 104 can embed the encoded metadata by sequentially changing the pixel values ​​of the pixels belonging to the second region to values ​​represented by the pixel value representation in the third table of Fig. 3. If there are multiple pieces of data to embed, the processing circuit 104 may subsequently embed the next piece of data.

[0089] Fig. 7 is a diagram showing an example of embedding data 1 in Fig. 3. The processing circuit 104 embeds reference values ​​representing the conversion table into the image by using reference values ​​64, 96, 128, and 160, which indicate 00b, 01b, 10b, and 11b, as pixel values ​​in that order from the start position of the first region specified by the reference value start flag.

[0090] For example, the processing circuit 104 can embed the pixel value as a value representing the grayscale of the pixel (RGB color information set to the same value). Without being limited to this, the processing circuit 104 can embed the reference value as any color information, such as individual color information such as R, G, and B, luminance information such as Y, angle information in hue, or information about the pixel in any other format. The format can be appropriately selected depending on the application of the encoder 10 and the decoder 20.

[0091] After embedding the reference value as a pixel value, the processing circuit 104 embeds a data start flag F1, which indicates the end of the first region and the start of the second region. Next, the processing circuit 104 sets the pixel values ​​of the encoded data 1 to 160, 96, 128, and 96 in order from the start position of the second region defined by the data start flag.

[0092] The processing circuit 104 then embeds a flag indicating the end of the data, that is, an end-of-data flag F2 indicating the end of the second region.

[0093] The encoder 10 can thus embed metadata within the image as pixel value information.

[0094] The decoder 20, which extracts metadata from image data, can obtain the metadata by referencing the rows in which the data is embedded, while being able to reference pixel values ​​in the image. The rows in which the data is embedded may be predetermined and shared between the encoder 10 and the decoder 20, or the data in these rows may be embedded in the image in some way, or the decoder 20 may detect the embedded rows by scanning pixel values ​​to detect flags.

[0095] The processing circuit 204 obtains the layout of the first region by detecting the reference value start flag F0.

[0096] The processing circuit 204 sequentially acquires pixel values ​​of the first region to acquire reference value information. For example, by acquiring pixel values ​​of 64, 96, 128, and 160, the processing circuit 204 acquires that the pixel value corresponding to 00b in the binary representation of the divided metadata is 64, the pixel value corresponding to 01b is 96, the pixel value corresponding to 10b is 128, and the pixel value corresponding to 11b is 160.

[0097] In other words, the processing circuit 204 detects that the encoded metadata can be converted to metadata by converting a pixel value of 64 to 00b, 96 to 01b, 128 to 10b, and 160 to 11b.

[0098] The processing circuit 204 uses the data start flag F1 to detect the end of the first region and the start of the second region. In the second region, the processing circuit 204 obtains pixel values ​​of 160, 96, 128, and 96, and by reading the data end flag F2, detects that this sequence of pixel values ​​is the encoded data of the metadata to be restored.

[0099] Based on the information obtained above, the processing circuit 204 extracts that the binary representation of the pixel value sequence of 160, 96, 128, 96 is 11b, 01b, 10b, 01b, and combines these to extract that the binary representation of the metadata is 01100111b. As a result, the processing circuit 204 can obtain that the embedded binary data is the numeric value 103.

[0100] Although the example of indicating an area by a flag has been described, the present invention is not limited to this. For example, it is possible to share predetermined areas as a first area and a second area between the encoder 10 and the decoder 20, and embed or extract reference values ​​and encoding metadata based on this information.

[0101] The flag can also have any length. That is, the user can specify the length of the flag when embedding metadata. An arbitrary length also includes a length of 0, i.e., no flag. In this case, for example, the embedding positions of the reference value and encoded metadata are determined in advance, the encoder 10 embeds data according to the embedding positions, and the decoder 20 reads the data from the pixel value at the embedding position and decodes the metadata.

[0102] The same applies to the following embodiments, and although a case where a flag is provided will be described, the flag can also be arranged arbitrarily or with an arbitrary length in these embodiments.

[0103] In addition, although one first region and one second region are shown in the above description for one pixel line, this is not limiting and multiple combinations of first and second regions may exist in one line.

[0104] Furthermore, a single line may contain multiple first regions and multiple second regions, regardless of their combination. In this case, data may be embedded indicating which first region and which second region correspond to the reference value and encoded metadata, or a first region may be defined as a reference value corresponding to the encoded metadata in the second region embedded next. In other words, a combination of a first region and a second region does not have to be in the same line but may be in different lines, and further, the line may change midway through the first region or second region (i.e., the first region or second region for the same data may span multiple lines).

[0105] FIG. 8 is a flowchart showing the processing of the processing circuit 104 according to one embodiment.

[0106] The processing circuit 104 divides the metadata (S100).

[0107] The processing circuit 104 encodes the divided metadata using the reference value to obtain encoded metadata (S102).

[0108] The processing circuit 104 designates the first region (S104). This designation can be made, for example, by embedding a flag in the image as described above.

[0109] The processing circuit 104 embeds the reference value as a pixel value in the specified first region (S106).

[0110] When the processing circuit 104 has finished writing the reference values, it writes a flag in the image indicating that the first region has ended and the second region has begun (S108).

[0111] The processing circuit 104 embeds the encoded metadata as pixel values ​​in the specified second region (S110).

[0112] The processing circuit 104 determines whether all of the metadata has been encoded and embedded (S112), and if processing has not been completed for all data (S112: NO), repeats the processing from S100.

[0113] If embedding of all metadata has been completed, the processing circuit 104 embeds an end-of-data flag (S114).

[0114] In this manner, the processing circuitry 104 can embed metadata within the image.

[0115] FIG. 9 is a flowchart showing the processing of the processing circuit 204 according to one embodiment.

[0116] The processing circuit 204 detects the first region (S200). The first region can be detected by scanning the image or scanning a specific row in the image and finding a reference value start flag. As described above, the flag is not a required configuration, and if the flag does not exist, the processing circuit 204 acquires the position of the first region using an appropriate method, for example, by specifying a position shared by the encoder 10 and the decoder 20.

[0117] The processing circuit 204 reads the reference value embedded in the first region (S202).

[0118] The processing circuit 204 detects the end of the first region and the beginning of the second region by detecting the data start flag (S204). As with the flag, if the flag is not present, the processing circuit 204 obtains the location of the second region by an appropriate method.

[0119] The processing circuit 204 reads the coded data from the second area (S206).

[0120] The processing circuit 204 decodes the divided data from the read encoded data based on the reference value (S208).

[0121] The processing circuit 204 combines the divided data in a predetermined manner to obtain the metadata (S210).

[0122] The processing circuit 204 repeats the process from S200 until extraction of all metadata is completed (S212: NO), and when extraction of all metadata is completed (S212: YES), the processing circuit 204 ends the metadata extraction process.

[0123] As described above, according to this embodiment, it is possible to embed reference values ​​and metadata in predetermined rows. By embedding data in this manner, it is possible to appropriately encode metadata into image data and decode metadata from image data without impairing the visibility of the image. By embedding reference values ​​as an encoding table, the information processing device can embed and read metadata in a manner that is robust against image degradation, pixel value conversion by predetermined processing, and the like.

[0124] Furthermore, the metadata can be embedded in an area of ​​the image information that is not displayed as an image. In this case, for example, it is possible to display the image data obtained via the decoder 20 with the lines where the metadata is embedded masked.

[0125] Second Embodiment Metadata may be embedded across multiple lines. As a non-limiting example, a combination of a reference value and metadata may be embedded across multiple lines.

[0126] Fig. 10 is a diagram showing an example of embedding metadata according to an embodiment. As shown in Fig. 10, certain metadata and a reference value for encoding may be embedded in one line, and other metadata and a reference value for encoding may be embedded in another line.

[0127] In this figure, two consecutive lines are used, but this is not limiting, and as another example, two separate lines may be used.

[0128] The reference values ​​R00~ and R10~ may be different values, but when using two consecutive rows, it is more desirable to use a different combination from the viewpoint of visibility for the user viewing the image. Even when displaying a metadata embedding area with a mask applied, it is possible to use multiple rows, and similarly, the reference values ​​can be the same combination or different combinations.

[0129] Also, if the reference value is the same across multiple rows, you can set the reference value in the first row and use the same reference value for subsequent rows. In this case, by adding an additional first region along the way, you can use the same reference value for certain rows and different reference values ​​for subsequent rows.

[0130] For example, Data 1 and Data 2 in Figure 3 can be embedded in different rows. Depending on the length of the data, the length of the second region may be different for each row. Similarly, the length of the first region may be different for each row.

[0131] In this way, it is possible to provide a first area and a second area for the first line, and a first area and a second area for the second line, and to embed metadata across multiple lines.

[0132] (Third embodiment) When multiple lines are used, the reference value and the metadata can be embedded in the same line as described above, but the present invention is not limited to this.

[0133] Fig. 11 is a diagram showing another example of embedding data across multiple rows. As shown in Fig. 11, the row in which the reference value is embedded and the row in which the encoded metadata is embedded can be separate rows.

[0134] That is, a first region may be provided in the first row, and a second region may be provided in the second row.

[0135] The reference value is embedded in a first area designated by, for example, a start flag F0 and an end flag F1, and the encoded metadata is embedded in a second area designated by, for example, a start flag F3 and an end flag F4.

[0136] 11, the positions of the flag, the first region, and the second region are aligned in the upper and lower rows, but this is not limiting. For example, the second region may be longer than the first region, in which case the end flags may appear in offset positions in the upper and lower rows.

[0137] In this way, the reference value and the metadata may be embedded in separate rows. In this example, as in the above example, the reference value and the metadata may be embedded in multiple discontinuous rows, rather than in consecutive rows.

[0138] In the above, the reference values ​​may be embedded repeatedly. For example, if the reference values ​​are defined as R0 to Rn, pixel values ​​R0, R1, ..., Rn may be embedded in the first region, and then pixel values ​​R0, R1, ..., Rn may be embedded repeatedly. The pixel values ​​may be embedded twice, or may be embedded more than once. The number of repetitions may also be changed depending on the length of the encoded metadata in the second region.

[0139] When embedding reference values ​​repeatedly, it is not necessary to write all of R0 to Rn in the final repetition. For example, depending on the length of the first and / or second fields, the final repetition may end with embedding reference values ​​from R0 to Rk (k < n).

[0140] One non-limiting method for compressing video information is to compress it based on the pixel values ​​of the pixels immediately above the pixel being compressed. In such compression, if the reference value and the encoded metadata are embedded using similar parameters, such as luminance values, it is desirable that the first and second regions are embedded in rows close to each other. In this embodiment, by repeatedly embedding the reference value as described above, compression errors can be reduced.

[0141] Fourth Embodiment In the above-described embodiments, one value is embedded in one pixel, but the present invention is not limited to this.

[0142] FIG. 12 is a diagram showing an example of embedding according to an embodiment. As shown in this figure, the same value may be embedded in a plurality of consecutive pixels in the horizontal direction (or the vertical direction). For example, the pixels within the dotted line may have the same pixel value. In the example of FIG. 12, every two pixels in the horizontal direction in the image have the same pixel value.

[0143] Fig. 13 is a diagram showing an example of embedding according to an embodiment. As shown in Fig. 13, the same value may be written to multiple pixels in both the horizontal and vertical directions. For example, as shown in Fig. 13, the same pixel value may be written every 2 × 2 pixels, and reference values ​​and encoding metadata may be embedded.

[0144] That is, in an image, at least one of the reference value and the encoded metadata may be embedded as a value of a pixel group that groups together a plurality of pixels.

[0145] By implementing it in this way, it is possible to embed metadata appropriately even in formats in which the same pixel value is applied to multiple pixels, such as video transmission methods such as Yuv422, Yuv411, and Yuv420.

[0146] Furthermore, when data is embedded for each pixel, the pixel values ​​of the embedded data may change due to thinning of Yuv information. According to this embodiment, it is possible to embed metadata using a pixel group whose pixel values ​​do not change even when such Yuv data is thinned, for example, when chromatic difference components are thinned.

[0147] Fifth Embodiment In the above, it has been shown that the same pixel value is used for a plurality of pixels, but it is also possible to provide a dummy pixel value for one pixel value.

[0148] 14 is a diagram showing an example of embedding metadata according to one embodiment. As shown in this diagram, for example, the reference values ​​may be R00, 0, R01, 0, R02, ..., with pixels having a pixel value of 0 sandwiched between each value. Instead of 0, the maximum value may also be used. Similarly, pixels of 0 can also be sandwiched between encoded metadata.

[0149] Also, 0 may be inserted between flags.

[0150] Fig. 15 is a diagram showing an example of embedding metadata according to an embodiment. As shown in this figure, pixels with a pixel value of 0 can be sandwiched vertically instead of horizontally. Also, Fig. 14 and Fig. 15 can be used in combination.

[0151] In other words, the first region may contain pixels with pixel values ​​of a predetermined value in addition to pixels with pixel values ​​indicating reference values, and the second region may contain pixels with pixel values ​​of a predetermined value in addition to pixels with pixel values ​​indicating encoding metadata. As another example, pixel values ​​of a predetermined value may be embedded only in the first region, or pixels of a predetermined value may be embedded only in the second region.

[0152] By thinning out the pixels to be embedded and embedding the data in this way, it is possible to achieve embedding of metadata that is more robust against degradation of pixel values ​​due to compression.

[0153] When inserting pixels having dummy pixel values ​​as shown in Figures 14 and 15, it is also possible to use multiple consecutive pixels as dummy values. For example, information such as R00, 0, 0, R01, etc. can be embedded as reference values. The same applies to encoded metadata.

[0154] Sixth Embodiment Fig. 16 is a diagram showing another example in which pixels having different pixel values ​​are sandwiched between pixels to be embedded. The pixels indicated by diagonal lines in the diagram are pixels having pixel values ​​of the original image. In this way, it is also possible to embed metadata by arranging the original pixel values ​​between the reference values, encoded metadata, and flags.

[0155] In the figure, data is embedded for each pixel, and the pixel value of the original image is shown, but this is not limiting. For example, embedded data for multiple consecutive pixels and image data for multiple consecutive pixels may be arranged alternately.

[0156] In addition, consecutive pixels having the pixel value of the embedded data and consecutive pixels having the pixel value of the image data may be alternately arranged in equal numbers, or in different numbers. For example, a pixel having a pixel value of one pixel of embedded data and a pixel having pixel values ​​of two consecutive pixels of the image may be alternately arranged.

[0157] In other words, the first region may contain pixels having pixel values ​​in the original image data in addition to pixels having pixel values ​​indicating the reference values, and the second region may contain pixels having pixel values ​​in the image data in addition to pixels having pixel values ​​indicating the encoding metadata. As another example, only the first region may contain pixels having pixel values ​​in the original image data, and only the second region may contain pixels having pixel values ​​in the original image data.

[0158] Figure 17 shows an example of a similar implementation across multiple lines. When embedding data across multiple consecutive lines, as shown in this figure, the data can be embedded so that the original pixel values ​​are arranged in a checkerboard pattern. While only the reference value portion is shown in the figure, pixel values ​​may also be embedded in the flag and encoded metadata areas in a similar manner.

[0159] As in the case of FIG. 16, the number of consecutive pixels having the pixel value of the image and the number of consecutive pixels having the pixel value of the embedded data can be determined arbitrarily.

[0160] Embedding metadata in this way can improve the user's visibility of the displayed image.

[0161] Furthermore, compared to masking the area where data is embedded, as given as a non-limiting example in the first embodiment, it is possible to make the area where metadata is embedded less noticeable within the image.

[0162] For example, the post-processing circuit 206 can interpolate the pixel values ​​of the pixels in which data is embedded based on the pixel values ​​of the image, thereby enabling metadata to be embedded without degrading visibility.

[0163] Seventh Embodiment In the above example, all the reference values ​​are arranged in ascending order (for example, in the order of 64, 96, 128, 160), but the present invention is not limited to this.

[0164] 18 is a diagram showing an example of embedding metadata according to an embodiment. As shown in this figure, the magnitude relationship of the reference values ​​may be in any order, not just ascending order. In this case, for example, the conversion in the binary representation of the divided metadata may be defined as follows: 00b is a pixel value of 64, 01b is a pixel value of 128, 10b is a pixel value of 96, and 11b is a pixel value of 160.

[0165] This arbitrary order may be an order that indicates a predetermined magnitude relationship. For example, in FIG. 18, the reference values ​​are arranged in the predetermined order of initial value (00b), large (01b), small (10b), and large (11b). Here, large or small indicates the result of comparison with the previous reference value (grayscale value). For example, the reference values ​​in the first area are, starting from 00b, 64 < 128 > 96 < 160, so the increase or decrease in grayscale from 01b can be indicated by referencing the inequality signs as large (64 "<" 128), small (128 ">" 96), and large (96 "<" 160).

[0166] In this way, the reference values ​​can be embedded in any order having any magnitude relationship.

[0167] This predetermined order can also be assigned a meaning. In other words, it is possible to associate predetermined data with the order of magnitude of the reference values. For example, if the order is ascending, it is metadata indicating coordinate information within the image, if it is large-small-large (FIG. 18), it is metadata indicating a timestamp, and if it is large-large-small, it is metadata indicating a frame ID. In this way, it is possible to assign meaning to the metadata decoded by the reference values.

[0168] As a result, it is possible to implement information that defines the meaning of metadata in each of the above-described embodiments without increasing the amount of image data or changing the length of the metadata data.

[0169] Eighth Embodiment When multiple rows are used as in the third embodiment, the embedding of reference values ​​in the first area and the embedding of data in the second area can be realized for each block spanning multiple rows.

[0170] Fig. 19 is a diagram showing another example of embedding data across multiple rows. As shown in Fig. 19, the rows in which reference values ​​are embedded and the rows in which encoded metadata are embedded can be separate rows, and the reference values ​​and encoded metadata can also be arranged together for each block.

[0171] That is, the first row may have a first area, the second row may have a second area, the combined area of ​​the first and second areas may be divided into blocks, and each block may contain a reference value and metadata.

[0172] The reference value is embedded in a first area designated by, for example, a start flag F0 and an end flag F1, and the metadata is embedded in a second area designated by, for example, a start flag F3 and an end flag F4.

[0173] The reference values ​​R00, R01, ... arranged in the first block can be reference values ​​corresponding to, for example, the metadata D00, D01, .... The same is true for the second block, where D10, D11, ... can be arranged as metadata corresponding to R10, R11, .... In this way, reference values ​​and metadata can be arranged for each block.

[0174] In this way, the row in which the reference value is embedded and the row in which the metadata is embedded may be separate rows, and further, the reference value and the metadata may be arranged for each block.

[0175] The size of a block may be set based on, for example, the compression method of the data to be transmitted and received. For example, when the VDC-M compression algorithm is used for display data, the block size may be 8 x 2. This is a non-limiting example, and the block size may be set based on the compression algorithm or other algorithms related to data placement.

[0176] The flag positions in the first and second lines may be offset, or may be in the same position as shown in the figure. In the above example, a block is formed using two lines, but this is not limited to this, and the block may include any number of lines suitable for an algorithm such as compression. The number of lines may include one line. In addition, if there are three or more lines, the number of lines in which reference values ​​are embedded and the number of lines in which metadata is embedded can also be set arbitrarily.

[0177] As in the case of Figure 11, one non-limiting method for compressing video information is to compress it based on the pixel value immediately above the pixel to be compressed. In such compression, if the reference value and the encoded metadata are embedded using the same type of parameters, for example, by expressing them as luminance values, it is desirable that the first and second regions are embedded in close proximity to each other. In this embodiment, the reference value and the metadata can be embedded in the same block as described above, thereby reducing compression errors.

[0178] Ninth Embodiment Fig. 20 is a diagram showing an example of an electronic device according to an embodiment. The electronic device 1 includes an encoder 10, a storage unit 12, and an output unit 14. The encoder 10 can acquire an image stored in the storage unit 12, embed metadata, and output the image via the output unit 14. In this way, the encoder 10 can be embedded in any electronic device.

[0179] 21 is an example of the electronic device 1 shown in FIG. 20, and the electronic device 1 may be a solid-state imaging device. In this case, the electronic device 1 includes an imaging unit 16. The encoder 10 can acquire image data captured by the imaging unit 16 and embed metadata.

[0180] In the embodiment shown in FIG. 21, the imaging unit 16 and at least a part of the encoder 10 may be provided on the same semiconductor chip.

[0181] A circuit for generating images, such as a GPU (Graphics Processing Unit), may be provided between the memory unit 12 or the imaging unit 16 and the encoder 10. In this case, the GPU may not only be used for generating images, but may also execute part of the processing of the encoder 10 using its highly parallel computing power. In other words, a circuit such as a GPU may be provided as a separate configuration from the encoder 10 of the electronic device 1, or may be provided as specific hardware for implementing the encoder 10.

[0182] 22 is a diagram showing an example of an electronic device according to an embodiment. The electronic device 2 includes a decoder 20, an acquisition unit 22, and an output unit 24. The decoder 20 can acquire image data acquired by the acquisition unit 22, obtain embedded metadata, and output the metadata from the output unit 24.

[0183] FIG. 23 is an example of FIG. 22, and the electronic device 2 may be a display device. In this case, the electronic device 2 includes a display unit 26. The decoder 20 acquires image data acquired by the acquisition unit 22, extracts metadata, performs appropriate image processing, and displays the image on the display unit 26. The decoder 20 may perform image processing using the extracted metadata. The acquisition unit 22 may be a receiving unit, and in this case, the electronic device 2 can operate as a display device that receives data and displays an image.

[0184] 24 is a diagram showing an example of a system that combines the above implementations. System 3 includes, for example, electronic device 1 as a solid-state imaging device and electronic device 2 as a display device.

[0185] The electronic device 1 embeds metadata in captured image information and transmits the image information.

[0186] The electronic device 2 extracts the metadata from the image information received from the electronic device 1 and displays it.

[0187] In this way, the information processing device described in each of the above embodiments can be incorporated as part of the configuration of a solid-state imaging device or a display device, or as part of a system such as a broadcasting or video distribution system.

[0188] Although the above describes an example in which processing is performed by each component, at least a portion of this processing may be executed by a general-purpose processing circuit (processor). In this case, the general-purpose processing circuit can perform the processing by reading a program, executable file, or equivalent program that describes at least a portion of the processing, which is stored in a storage unit such as a memory or storage. In this way, information processing by software may be specifically performed using hardware resources.

[0189] The above-described embodiment may be modified as follows.

[0190] (1) An information processing device comprising a processing circuit, which acquires image data, embeds reference values ​​in pixels arranged in a first region of an area representing an image of the image data, embeds encoded metadata obtained by converting metadata using the reference values ​​in pixels arranged in a second region of the area representing the image of the image data, and outputs the image data in which the reference values ​​and the values ​​of the encoded metadata have been embedded.

[0191] (2) The information processing device according to (1), wherein the reference value indicates a value in a table that defines conversion between pixel values ​​in the image data and values ​​of the metadata.

[0192] (3) The information processing device according to (2), wherein the reference value is defined as a combination of a pixel value and a value having a predetermined number of bits.

[0193] (4) The information processing device according to (3), wherein the processing circuit generates a bit string by dividing the metadata into the predetermined number of bits, converts the bit string into a converted pixel value based on the reference value, and sets the pixel value of the pixel to be placed in the second area as the converted pixel value.

[0194] (5) The information processing device according to (3), wherein the processing circuit assigns a different pixel value to each of the binary expressions of the predetermined number of bits, uses the assigned pixel value as the reference value, and embeds the binary expressions in the first area in descending or ascending order.

[0195] (6) The information processing device described in (5), wherein the processing circuit generates a binary representation bit string by dividing the binary representation of the metadata into the predetermined number of bits, and converts the binary representation bit string using the reference value to generate the encoded metadata.

[0196] (7) The information processing device according to (5), wherein the predetermined number of bits is 4 bits, and the processing circuit sets pixel values ​​corresponding to each of 4-bit binary values ​​0000b, 0001b, 0010b, 0011b, 0100b, 0101b, 0110b, 0111b, 1000b, 1001b, 1010b, 1011b, 1100b, 1101b, 1110b, and 1111b, and sets pixel values ​​corresponding to each of the 4-bit values ​​in order in the first area and embeds them as the reference value.

[0197] (8) The information processing device according to (5), wherein the processing circuit sets the order of magnitude of pixel values ​​of the reference values ​​to be a predetermined order for the binary representation.

[0198] (9) The information processing device according to (8), wherein the processing circuit defines the reference values ​​for the attributes of the metadata in the predetermined order.

[0199] (10) The information processing device described in (5), wherein the processing circuit sets pixel values ​​in the first area or the second area to include at least one of pixels in the first area having pixel values ​​that are a predetermined value other than pixels having pixel values ​​that indicate the reference value, or pixels in the second area having pixel values ​​that are a predetermined value other than pixels having pixel values ​​that indicate the encoding metadata.

[0200] (11) The information processing device described in (5), wherein the processing circuit sets pixel values ​​in the first area or the second area to include at least one of pixels in the first area that have pixel values ​​in the image data other than pixels having pixel values ​​indicating the reference value, or pixels in the second area that have pixel values ​​in the image data other than pixels having pixel values ​​indicating the encoding metadata.

[0201] (12) The information processing device according to any one of (1) to (11), wherein the first area and the second area are provided in a first predetermined row.

[0202] (13) The information processing device according to any one of (1) to (11), wherein the first area is provided in a first predetermined row, and the second area is provided in a second predetermined row.

[0203] (14) The information processing device according to any one of (1) to (11), comprising the first region and the second region for a first predetermined row, and the first region and the second region for a second predetermined row, for each row.

[0204] (15) The information processing device according to any one of (1) to (11), wherein at least one of the reference value and the encoded metadata is embedded as a value of a pixel group that groups together a plurality of pixels.

[0205] (16) An information processing device according to any one of (1) to (15), further comprising a storage unit, wherein the processing circuit acquires the image data stored in the storage unit, and embeds the reference value and the value of the encoded metadata into the image data.

[0206] (17) A solid-state imaging device comprising: an information processing device according to any one of (1) to (15); and an imaging unit, wherein the information processing device acquires an image captured by the imaging unit as the image data, and embeds the reference value and the value of the encoded metadata in the image data.

[0207] (18) The solid-state imaging device according to (17), wherein the information processing device and the imaging unit are provided in the same semiconductor chip.

[0208] (19) An information processing device comprising a processing circuit, acquiring image data, reading reference values ​​from pixels arranged in a first area of ​​an area representing an image of the image data, reading encoded metadata from pixels arranged in a second area of ​​the area representing the image of the image data, and acquiring metadata obtained by converting the encoded metadata using the reference values.

[0209] (20) A display device comprising: a receiving unit; the information processing device according to (19); and a display unit, wherein the receiving unit receives transmitted data as the image data; the processing circuit acquires the metadata from the received image data; and displays the image data on the display unit.

[0210] (21) The display device according to (20), wherein the processing circuit performs signal processing on the image data based on the metadata, and displays the signal-processed image data on the display unit.

[0211] (22) A transmission / reception system comprising a first information processing device and a second information processing device, wherein the first information processing device acquires image data, embeds reference values ​​in pixels arranged in a first area of ​​an area representing an image of the image data, embeds encoded metadata obtained by converting the metadata using the reference values ​​in pixels arranged in a second area of ​​the area representing the image of the image data, and transmits the image data in which the reference values ​​and the values ​​of the encoded metadata are embedded, and the second information processing device receives the image data transmitted from the first information processing device, reads the reference values ​​from pixels arranged in the first area of ​​the area representing the image of the image data, reads the encoded metadata from pixels arranged in the second area of ​​the area representing the image of the image data, and acquires the metadata obtained by converting the encoded metadata using the reference values.

[0212] (23) A program that causes a processor to execute at least a part of the processes described in (1) to (22).

[0213] The aspects of the present disclosure are not limited to the above-described embodiments and include various conceivable modifications, and the effects of the present disclosure are not limited to the above-described contents. The components in each embodiment may be appropriately combined and applied. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and intent of the present disclosure, which is derived from the content defined in the claims and their equivalents.

[0214] 1: Electronic device, 10: Encoder, 100: Image acquisition circuit, 102: Pre-processing circuit, 104: Processing circuit, 106: Post-processing circuit, 108: Output circuit, 12: Memory unit, 14: Output unit, 16: Imaging unit, 2: Electronic device, 20: Decoder, 200: Image acquisition circuit, 202: Pre-processing circuit, 204: Processing circuit, 206: Post-processing circuit, 208: Output circuit, 22: Acquisition unit, 24: Output unit, 26: Display unit, 3: System

Claims

1. An information processing device comprising a processing circuit, which acquires image data, embeds reference values ​​in pixels arranged in a first region of an area representing an image of the image data, embeds encoded metadata obtained by converting metadata using the reference values ​​in pixels arranged in a second region of the area representing the image of the image data, and outputs the image data in which the reference values ​​and the values ​​of the encoded metadata are embedded.

2. The information processing device according to claim 1, wherein the reference value indicates a value in a table that defines conversion between pixel values ​​in the image data and values ​​of the metadata.

3. The information processing device according to claim 2, wherein the reference value is defined for a combination of a pixel value and a value of a predetermined number of bits.

4. The information processing device according to claim 3, wherein the processing circuit generates a bit string by dividing the metadata into the predetermined number of bits, converts the bit string into a converted pixel value based on the reference value, and sets the pixel value of a pixel to be located in the second region to the converted pixel value.

5. The information processing device according to claim 3, wherein the processing circuit assigns a different pixel value to each of the binary expressions of the predetermined number of bits, uses the assigned pixel value as the reference value, and embeds the binary expressions in the first area in descending or ascending order.

6. The information processing device according to claim 5, wherein the processing circuit generates a binary representation bit string by dividing the binary representation of the metadata into the predetermined number of bits, and converts the binary representation bit string using the reference value to generate the encoded metadata.

7. The information processing device according to claim 5, wherein the predetermined number of bits is 4 bits, and wherein the processing circuit sets pixel values ​​corresponding to each of the 4-bit binary values ​​0000b, 0001b, 0010b, 0011b, 0100b, 0101b, 0110b, 0111b, 1000b, 1001b, 1010b, 1011b, 1100b, 1101b, 1110b, and 1111b, and sets pixel values ​​corresponding to each of the 4-bit values ​​in order in the first area and embeds them as the reference value.

8. The information processing device of claim 5, wherein the processing circuit sets pixel values ​​in the first region or the second region to include at least one of pixels in the first region having pixel values ​​that are a predetermined value other than pixels having pixel values ​​that indicate the reference value, or pixels in the second region having pixel values ​​that are a predetermined value other than pixels having pixel values ​​that indicate the encoding metadata.

9. The information processing device of claim 5, wherein the processing circuit sets pixel values ​​in the first region or the second region to include at least one of pixels in the first region having pixel values ​​in the image data other than pixels having pixel values ​​indicating the reference value, or pixels in the second region having pixel values ​​in the image data other than pixels having pixel values ​​indicating the encoding metadata.

10. The information processing device according to claim 1, wherein the first area and the second area are provided in a first predetermined row.

11. The information processing device according to claim 1, wherein the first area is provided on a first predetermined row, and the second area is provided on a second predetermined row.

12. The information processing device according to claim 1, further comprising: the first region and the second region for a first predetermined row; and the first region and the second region for a second predetermined row, for each row.

13. The information processing device according to claim 1, wherein at least one of the reference value and the encoded metadata is embedded as a value of a pixel group that groups together a plurality of pixels.

14. An information processing device as described in claim 1, further comprising a memory unit, wherein the processing circuit acquires the image data stored in the memory unit, and embeds the reference value and the value of the encoded metadata in the image data.

15. A solid-state imaging device comprising: an information processing device according to claim 1; and an imaging unit, wherein the information processing device acquires an image captured by the imaging unit as the image data, and embeds the reference value and the value of the encoded metadata in the image data.

16. The solid-state imaging device according to claim 15, wherein the information processing device and the imaging section are provided within a single semiconductor chip.

17. An information processing device comprising a processing circuit, acquiring image data, reading reference values ​​from pixels arranged in a first region of an area representing an image of the image data, reading encoded metadata from pixels arranged in a second region of the area representing the image of the image data, and acquiring metadata obtained by converting the encoded metadata using the reference values.

18. A display device comprising: a receiving unit; an information processing device according to claim 17; and a display unit, wherein the receiving unit receives transmitted data as the image data, and the processing circuit acquires the metadata from the received image data, and displays the image data on the display unit.

19. The display device according to claim 18, wherein the processing circuit performs signal processing on the image data based on the metadata, and displays the signal-processed image data on the display unit.

20. A transmission / reception system comprising a first information processing device and a second information processing device, wherein the first information processing device acquires image data, embeds reference values ​​in pixels arranged in a first area of ​​an area indicating an image of the image data, embeds encoded metadata obtained by converting the metadata using the reference values ​​in pixels arranged in a second area of ​​the area indicating the image of the image data, and transmits the image data in which the reference values ​​and the values ​​of the encoded metadata are embedded, and the second information processing device receives the image data transmitted from the first information processing device, reads the reference values ​​from pixels arranged in the first area of ​​the area indicating the image of the image data, reads the encoded metadata from pixels arranged in the second area of ​​the area indicating the image of the image data, and acquires metadata obtained by converting the encoded metadata using the reference values.

Images