Information processing device, solid-state imaging device, display device, and transmission / reception system

Abstract

[Problem] To appropriately add metadata to an image. [Solution] This information processing device is provided with a processing circuit. The processing circuit acquires image data, embeds reference values into pixels arranged in a first region indicating a region of an image of the image data, embeds encoded metadata into pixels arranged in a second region indicating the region of the image of the image data, and embeds the encoded metadata into pixels arranged in a third region indicating the region of the image of the image data. The processing circuit acquires encoded metadata by converting the metadata using the reference value, and outputs image data in which the reference value and the value of the encoded metadata are embedded.

Description

Technical Field

[0001] This disclosure relates to information processing devices, solid-state imaging devices, display devices, and transceiver systems. Background Technology

[0002] In the transmission and reception of image and video signals, it is sometimes desirable to send additional information as metadata, in addition to the image and video data to be transmitted and received. Metadata includes information such as frame counts, frame IDs, delay amounts in variable refresh rate (VRR), registers, and timestamps.

[0003] For the transmission of metadata, consider using methods such as internal integrated circuits (I) 2 C) Methods that use general control signals from a serial peripheral interface (SPI) to perform transmission, and methods that embed data during blanking between multiple data streams. When using general control signals, it is difficult to send metadata synchronized with the video signal due to increased bandwidth caused by higher resolution and higher frame rates. Furthermore, when embedding data during blanking, it is often difficult to achieve actual data embedding considering various data formats.

[0004] For these reasons, it is possible to consider embedding metadata as pixel values ​​into the image data and recovering the metadata from the pixel values ​​at the receiving end. However, recovery cannot be performed consistently and appropriately by changing the signals in transmission and reception, the encoding on the transmitting side, the decoding or filtering on the receiving side, etc.

[0005] Reference List

[0006] Patent documents

[0007] Patent Document 1: Japanese translation of PCT International Application Publication No. 2021-525382 Summary of the Invention

[0008] The problem to be solved by the present invention

[0009] Therefore, one non-limiting problem that embodiments of this disclosure aim to solve is the appropriate addition of metadata to images to be sent and received. The problem solved by embodiments of this disclosure may also be a problem corresponding to the effects described in the embodiments, as some further non-limiting examples. That is, a problem corresponding to at least one optional effect described in the description of embodiments of this disclosure may be a problem to be solved by this disclosure.

[0010] Solution to the problem

[0011] According to one embodiment, the information processing apparatus includes processing circuitry.

[0012] The processing circuit acquires image data, embeds reference values ​​into pixels in a first region of the image area indicating the image data, embeds encoded metadata obtained by converting metadata using reference values ​​into pixels in a second region of the image area indicating the image data, and outputs image data with values ​​embedded with reference values ​​and encoded metadata.

[0013] Reference values ​​can be values ​​from a table that indicates the conversion between pixel values ​​in image data and values ​​in metadata.

[0014] Reference values ​​can be defined for a combination of pixel values ​​and predetermined bit values.

[0015] The processing circuit can generate a bit string obtained by dividing the metadata into bits according to a predetermined number of bits, convert the bit string into a converted pixel value based on a reference value, and set the pixel value of the pixel arranged in the second region to the converted pixel value.

[0016] The processing circuit can assign different pixel values ​​to a binary representation of a predetermined number of bits, and use the assigned pixel values ​​as reference values ​​to embed them into the first region in descending or ascending order of the binary representation.

[0017] The processing circuitry can generate a binary representation bit string obtained by dividing the binary representation of metadata into bits according to a predetermined number of bits, and convert the binary representation bit string using a reference value to generate encoded metadata.

[0018] The predetermined bit depth can be 4 bits, and the processing circuit can set pixel values ​​corresponding to the 4-bit binary values ​​0000b, 0001b, 0010b, 0011b, 0100b, 0101b, 0110b, 0111b, 1000b, 1001b, 1010b, 1011b, 1100b, 1101b, 1110b, and 1111b. The pixel values ​​corresponding to the 4-bit values ​​are sequentially set in the first region, and these pixel values ​​are embedded as reference values. Of course, the predetermined bit depth can be less than 2 bits or greater than 5 bits.

[0019] The processing circuit can set the pixel values ​​of the reference values ​​to a predetermined order in binary representation.

[0020] The processing circuitry can define reference values ​​for metadata attributes in a predetermined order.

[0021] The processing circuit can set pixel values ​​in a first region or a second region to include at least one of the following pixels: a pixel in the first region that has a predetermined pixel value, which is different from a pixel having a pixel value indicating a reference value, and a pixel in the second region that has a pixel value indicating encoded metadata, which has a predetermined pixel value.

[0022] The processing circuit can set pixel values ​​in a first region or a second region to include at least one of the following pixels: pixels in the first region that have pixel values ​​in image data that are separate from pixel values ​​having reference values, and pixels in the second region that have pixel values ​​in image data that are different from pixel values ​​having pixel values ​​having encoded metadata.

[0023] The first and second regions can be set in the first predetermined row.

[0024] The first region can be set in the first predetermined line, and the second region can be set in the second predetermined line.

[0025] A first region and a second region can be set for a first predetermined line, and a first region and a second region can be set for a second predetermined line.

[0026] At least one of the reference value and the encoded metadata can be embedded as the value of a pixel group comprising multiple pixels.

[0027] The information processing apparatus may also include a storage unit, and the processing circuitry may acquire image data stored in the storage unit and embed reference values ​​and encoded metadata values ​​into the image data.

[0028] According to one embodiment, the solid-state imaging apparatus includes an information processing device and an imaging unit as described in any of the above descriptions.

[0029] The information processing device acquires the image captured by the imaging unit as image data and embeds the values ​​of reference values ​​and encoded metadata into the image data.

[0030] Information processing devices and imaging units can be housed in the same semiconductor chip.

[0031] According to one embodiment, the information processing apparatus includes a processing circuit.

[0032] The processing circuit acquires image data, reads reference values ​​from pixels in a first region of the image area indicating the image data, reads encoded metadata from pixels in a second region of the image area indicating the image data, and acquires metadata obtained by converting the encoded metadata using the reference values.

[0033] According to one embodiment, the display device includes a receiving unit, an information processing unit, and a display unit.

[0034] The receiving unit receives the transmitted data as image data, and the processing circuit obtains metadata from the received image data and displays the image data on the display unit.

[0035] The processing circuitry can perform signal processing on the image data based on metadata and display the processed image data on the display unit.

[0036] According to one embodiment, the transceiver system includes a first information processing device and a second information processing device.

[0037] The first information processing device acquires image data, embeds reference values ​​into pixels in a first region of an image region indicating the image data, embeds encoded metadata obtained by converting metadata using the reference values ​​into pixels in a second region of the image region indicating the image data, and transmits image data containing the values ​​embedded with the reference values ​​and the encoded metadata. The second information processing device receives the image data transmitted from the first information processing device, reads reference values ​​from pixels in the first region of the image region indicating the image data, reads encoded metadata from pixels in the second region of the image region indicating the image data, and acquires metadata obtained by converting the encoded metadata using the reference values.

[0038] According to one embodiment, a program causes a processor to perform at least a portion of the above-described processes. The program may be stored in a temporary or non-temporary computer-readable medium. Attached Figure Description

[0039] Figure 1 This is a diagram illustrating an example of an encoder (information processing device) according to one embodiment.

[0040] Figure 2 This is a diagram illustrating an example of a decoder (information processing device) according to one embodiment.

[0041] Figure 3 This is a diagram illustrating an example of generated encoded metadata according to one implementation method.

[0042] Figure 4 This is a diagram illustrating an example of generated encoded metadata according to one implementation method.

[0043] Figure 5 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0044] Figure 6 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0045] Figure 7 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0046] Figure 8This is a flowchart illustrating the processing of an information processing apparatus according to one embodiment.

[0047] Figure 9 This is a flowchart illustrating the processing of an information processing apparatus according to one embodiment.

[0048] Figure 10 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0049] Figure 11 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0050] Figure 12 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0051] Figure 13 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0052] Figure 14 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0053] Figure 15 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0054] Figure 16 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0055] Figure 17 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0056] Figure 18 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0057] Figure 19 This is a diagram illustrating an example of metadata embedding according to one implementation.

[0058] Figure 20 This is a diagram illustrating an example of an electronic device according to one embodiment.

[0059] Figure 21 This is a diagram illustrating an example of an electronic device according to one embodiment.

[0060] Figure 22 This is a diagram illustrating an example of an electronic device according to one embodiment.

[0061] Figure 23 This is a diagram illustrating an example of an electronic device according to one embodiment.

[0062] Figure 24 This is a diagram illustrating an example of a system according to one implementation. Detailed Implementation

[0063] In the following description, embodiments of the present disclosure will be illustrated with reference to the accompanying drawings. The drawings are for illustrative purposes, and the shape and size of each unit in the actual device, its size ratio to another unit, etc., need not necessarily be as shown in the drawings. Furthermore, because the drawings are shown in a simplified manner, configurations suitable for implementation are provided in addition to those shown in the drawings.

[0064] In some embodiments of this disclosure, apparatus (methods) for embedding metadata in image data, apparatus (methods) for reading metadata embedded in image data, and electronic devices and systems using these apparatuses will be described. An image in this disclosure can be understood as data that can be extracted as an image relative to the timeline in a video. That is, where the image is merely image data, the image can be understood as part of the video data.

[0065] First, regarding the apparatus for embedding metadata into image data and reading metadata from image data in this disclosure, a configuration common to some implementations will be described.

[0066] Figure 1 This is an illustration of an example of an information processing apparatus used as an encoder according to one embodiment, which embeds metadata into image data. The encoder 10 (information processing apparatus) includes an image acquisition circuit 100, a preprocessing circuit 102, a processing circuit 104, a post-processing circuit 106, and an output circuit 108.

[0067] Encoder 10 is an information processing device that acquires data about an image, converts the image into an appropriate format (and performs appropriate signal processing, such as compression and / or image processing, as needed), and performs processing to embed metadata into the image. Furthermore, encoder 10 can be combined with other devices as an encoding module and encoding processing unit, and can cooperate with or be integrated with other devices.

[0068] Note that, although not shown, encoder 10 may appropriately include, internally or externally, the circuitry required for its operation, such as storage circuitry, control circuitry, power supply circuitry, etc. All circuitry may be dedicated circuitry, such as circuitry including application-specific integrated circuits (ASICs) and complementary metal-oxide-semiconductor sensors, or may include general-purpose processors, etc.

[0069] The image acquisition circuit 100 acquires data about the image. The data about the image can be the image data itself or the raw data before it was converted into an image.

[0070] The preprocessing circuit 102 to the postprocessing circuit 106 can be implemented as a single processing circuit as needed. That is, for convenience, these circuits are shown as independent circuits, and each configuration is not necessarily an independent circuit in any particular implementation.

[0071] The preprocessing circuit 102 converts the data acquired by the image acquisition circuit 100 into image data format in which metadata is embedded by the processing circuit 104. For example, when the image acquisition circuit 100 acquires raw image data, the preprocessing circuit 102 converts the signal into digital image data by the processing circuit 104, making it suitable for embedding data. For example, when the image acquisition circuit 100 acquires digital image data, the preprocessing circuit 102 converts the digital image data into a format suitable for embedding data by the processing circuit 104. For example, the format involves various image storage formats, such as RGB format and Yuv format.

[0072] Furthermore, when the data acquired by the image acquisition circuit 100 is video data, the preprocessing circuit 102 can also acquire image data as any part of the video data.

[0073] Processing circuit 104 performs metadata embedding in encoder 10. Processing circuit 104 encodes and embeds metadata in the image data acquired by preprocessing circuit 102. Processing circuit 104 encodes and embeds metadata in regions indicating the pixel values ​​of pixels in the image data.

[0074] More specifically, the processing circuit 104 embeds reference values ​​as metadata for encoding in a first region of the image, and embeds metadata encoded by the reference values ​​in a second region of the image. The processing circuit 104 embeds the reference values ​​and encoded metadata as pixel values ​​of the pixels constituting the image.

[0075] The post-processing circuit 106 converts the image data with embedded metadata from the processing circuit 104 into a format suitable for output (transmission). Furthermore, the post-processing circuit 106 can also perform any signal processing on the image data. For example, in the case where the encoder 10 performs the process of transmitting an image, the post-processing circuit 106 can perform encoding processing such as compression or encryption on the image with data embedded by the processing circuit 104. In this case, as another example, the post-processing circuit 106 can apply additional information, such as error detection and error correction for appropriate decoding, to the image data with embedded metadata.

[0076] Through this encoding process, the encoded metadata can also be obtained as pixel values ​​on the decoding side, and metadata can be obtained appropriately.

[0077] The output circuit 108 outputs image data processed by the post-processing circuit 106 or data containing the image data. For example, the output circuit 108 can output data to the internal or external memory of the encoder 10, transmit data to the Internet through a predetermined output interface, or transmit data through any communication method such as broadcasting or broadcasting equipment.

[0078] Figure 2 This diagram illustrates an example of an information processing apparatus used as a decoder according to one embodiment, which reads metadata from image data. The decoder 20 (information processing apparatus) includes an image acquisition circuit 200, a preprocessing circuit 202, a processing circuit 204, a post-processing circuit 206, and an output circuit 208. The decoder 20 is an information processing apparatus that acquires image data with metadata embedded by the encoder 10 and performs processing to extract metadata from the image data. The decoder 20 can also be combined with other devices as a decoding module and decoding processing unit, and can cooperate with or be integrated with other devices.

[0079] Furthermore, the decoder 20 may appropriately include, internally or externally, the circuitry required for its operation, such as storage circuitry, control circuitry, and power supply circuitry. All circuitry may be dedicated circuitry, referred to as circuitry including light-emitting elements such as ASICs or light-emitting diodes (LEDs), or it may be a general-purpose processor, etc.

[0080] The image acquisition circuit 200 acquires data that includes at least the image data output from the output circuit 108. That is, the image acquisition circuit 200 acquires data that includes at least the image data in which encoded metadata is embedded by the encoder 10.

[0081] The preprocessing circuits 202 to 206 can be implemented as a single processing circuit, similar to the preprocessing circuits 102 to 106 in the encoder 10. That is, for convenience, these circuits are shown as independent circuits, and as a specific implementation, each configuration need not be an independent circuit.

[0082] The preprocessing circuit 202 converts (decodes) the data acquired by the image acquisition circuit 200 into an image data format from which the processing circuit 204 can read metadata. That is, due to the use of the image data format encoded by the encoder 10, the data is converted into a format of pixel values ​​that can be read with embedded metadata, and image data from which metadata can be extracted is acquired. Similar to the processing of the preprocessing circuit 102, the preprocessing circuit 202 can also acquire image data as part of video data.

[0083] The processing circuit 204 obtains metadata by extracting encoded metadata embedded in the image and decoding the encoded metadata.

[0084] More specifically, the processing circuit 204 obtains metadata by reading pixel values ​​as reference values ​​from a first region of the image, reading pixel values ​​as encoded metadata from a second region of the image, and decoding the encoded metadata using the reference values.

[0085] The post-processing circuit 206 appropriately processes the image data (image data decoded by the pre-processing circuit 202) from which the metadata has been extracted by the processing circuit 204 into the format to be output. For example, the post-processing circuit 206 converts the data from the processing circuit 204 into a signal suitable for controlling the display, and outputs the signal through the output circuit 208.

[0086] Furthermore, the post-processing circuit 206 can perform image-related processing using the metadata obtained by the processing circuit 204. That is, the post-processing circuit 206 can perform signal processing or image processing on the image data decoded by the pre-processing circuit 202 using the metadata obtained by the processing circuit 204. Additionally, the post-processing circuit 206 can also generate and output control signals related to the image data to be output using the metadata obtained by the processing circuit 204.

[0087] The output circuit 208 outputs data including the image data processed by the post-processing circuit 206. Furthermore, the output circuit 208 can also output metadata extracted by the processing circuit 204. For example, the output circuit 208 can output image data or metadata along with the image data as data to internal or external memory of the decoder 20.

[0088] Notice, Figure 2 An example of outputting image data from output circuit 208 is shown. Output circuit 208 can only output image data. Furthermore, as a different example, output circuit 208 can also output metadata in some form.

[0089] For example, output circuit 208 can output image data reflected by post-processing circuit 206 on a certain parameter of the image based on metadata extracted by processing circuit 204. In this case, output circuit 208 does not need to output metadata. As another example, if it is desired to obtain metadata from decoder 20 as output, output circuit 208 can output metadata along with image data. Of course, output circuit 208 can also obtain and output only metadata from encoded image data.

[0090] As described above, according to the implementation method, even if the image signal deteriorates during image generation or transmission, the metadata can be appropriately extracted using reference values. For example, since the reference values ​​and metadata are stored as color codes, metadata that is resistant to signal degradation caused by image conversion, etc., can be reproduced.

[0091] Next, examples of the encoding of metadata in this disclosure will be described.

[0092] Figure 3 This is a diagram illustrating a non-limiting example of generated encoded metadata according to one embodiment. In the following, in the figures or tables, decimal representation is shown by describing the numerical value as is, while binary representation is shown by adding the suffix 'b' after the numerical value.

[0093] The first table presents examples of the values ​​of the metadata to be embedded. For example, it describes the transformations when the value 103 is embedded as data 1 and the value 13 as data 2 as metadata. These values ​​are represented in binary as 01100111b and 00001101b, respectively.

[0094] The second table presents the definitions of the reference values. This table means that the binary representation of the metadata is 00b represented by 64 pixels, 01b by 96 pixels, 10b by 128 pixels, and 11b by 160 pixels.

[0095] As mentioned above, the reference value can be regarded as an indication of the number of pre-ordered digits (in Figure 3 In the image, a 2-bit binary value is converted into a lookup table (LUT) value for the pixel value. The processing circuit 104 then embeds the converted value into the first region of the image.

[0096] The third table is a diagram illustrating an example of encoded metadata obtained by encoding the metadata. Metadata conversion means that the binary representation of the metadata is converted into a pixel value representation based on a LUT indicated by a reference value.

[0097] The binary representation of metadata is divided into predetermined numbers, for example, every two bits. For example, in a little-endian system, the divided metadata is rearranged. For example, since the binary representation of data 1 is "01100111b", the partitioned representation of data 1 is represented as "11b, 01b, 10b, 01b".

[0098] The encoded metadata is represented by numerical values ​​obtained by converting the segmented binary values ​​into pixel values ​​using reference values. For example, the first data of data 1 is the value 160 obtained by converting 11b to the reference value. That is, data 1 can be represented as "160, 96, 128, 96" as encoded metadata. Similarly, data 2 is segmented into "01b, 11b, 00b, 00b" and encoded as "96, 128, 64, 64".

[0099] In summary, the processing circuit 104 generates encoded metadata by dividing the value obtained from the binary representation of the metadata into predetermined bits, converting the divided binary values ​​using reference values, and assigning pixel values ​​to the binary values. The processing circuit 104 can embed the metadata into the image data by setting the pixel values ​​of pixels in the second region to the assigned pixel values ​​as the encoded metadata.

[0100] In the above description, metadata is embedded using a little-endian system, but a big-endian system can also be used. The fourth table represents a transformation similar to the one described in the big-endian system. Metadata embedding can be performed using this table.

[0101] These systems can be determined based on the hardware architecture of the encoder 10 that embeds metadata or the decoder 20 that extracts metadata. Specifically, a system suitable for combining segmented metadata strips in binary representation in the decoder 20 can be envisioned.

[0102] Of course, these systems are given as non-limiting examples, and the patterns in this disclosure can be applied to other arrangement sequences.

[0103] exist Figure 3 An example of dividing metadata into 2 bits (as a predetermined number of bits for dividing metadata) has been described in the literature, but the present invention is not limited thereto.

[0104] Figure 4 This is a diagram illustrating an example of generating encoded metadata according to one implementation. As shown in the diagram, encoded metadata can be generated by setting a predetermined number of bits to 3.

[0105] The reference value indicates what pixel value the 3-bit binary representation will be converted into. In the encoded metadata, for example, the binary representation of the metadata is segmented from the least significant bit (LSB) in predetermined number of bits, and a segmented representation is obtained. If the most significant bit depth is less than the predetermined number of bits, it can be padded with 0s. Using the reference value, this segmented representation is converted into a pixel value representation, thus generating the encoded metadata.

[0106] In such Figure 3 and Figure 4 When the pixel values ​​shown are represented by 8 bits, for example, a maximum of 255 numerical data can be supported, but the invention is not limited to this. Numerical data can be represented in binary form to generate encoded metadata, according to encoding methods such as common little-endian. The range of numerical data changes with the bit depth used to describe the pixel values.

[0107] In addition, Figure 3 and Figure 4 The examples have shown cases where the metadata is numerical, but the invention is not limited to this. Strings can also be converted using generic codes. For example, similar to the description above, encoded metadata can be defined for any string represented in a format such as JIS code or Unicode.

[0108] like Figure 3 and Figure 4 As shown, the predetermined number of bits used to segment the metadata can be of any bit depth. Furthermore, the reference value can be as follows: Figure 3 They are assigned with equal grayscale differences (every 32), or they can be arranged as follows: Figure 4 They are then assigned different grayscale differences.

[0109] Pixel values ​​are assigned to each binary string used to segment the metadata into a predetermined number of bits and represent any value, thus allowing the metadata to be properly encoded. During decoding, it is desirable for the reference values ​​to have a certain degree of grayscale difference so that they can be properly converted into a predetermined number of bits. By setting this grayscale difference, it is possible to suppress confusion with other pixel values ​​during decoding.

[0110] Furthermore, a binary representation was used, but the invention is not limited thereto. For example, metadata can be converted to other representations instead of binary and segmented, and similar processing as described above can be performed.

[0111] (First Implementation)

[0112] Some implementations will describe the embedding of reference values ​​and metadata in the image.

[0113] Figure 5 This is a diagram illustrating an example of metadata embedding according to one implementation method. For example... Figure 5 As shown by the slash in the non-restrictive example, encoder 10 can embed metadata into the upper region of the image data. Note that the rows for embedding metadata can be set arbitrarily, but since the metadata is embedded in the image data, it is desirable to place it in a location that is difficult for the user to visually recognize.

[0114] Figure 6This is a diagram illustrating an example of the arrangement of reference values ​​and encoded metadata. This example demonstrates embedding reference values ​​and encoded metadata within a single row. For instance, processing circuitry 104 embeds metadata within any row of image data (e.g., the topmost row).

[0115] Processing circuit 104 first according to Figure 3 Examples of this method include generating bit strings by segmenting metadata into bits at predetermined intervals, converting the bit strings into pixel values ​​based on reference values, encoding the metadata using the pixel values ​​as pixel values ​​of pixels arranged in a predetermined area, and embedding the encoded metadata into the image.

[0116] In the row of embedded data, the processing circuit 104 first sets a first region for embedding reference values. The first region can be set at any position in the row. The processing circuit 104 arranges pixels or a pixel array with pixel values, for example, having a reference value start flag F0, and embeds a flag to indicate the start of the first region. The reference value start flag F0 can be represented by bits with predetermined values, or it can be represented by predetermined values ​​using multiple bits.

[0117] The processing circuit 104 embeds reference values ​​in a first region defined by the reference value start flag F0. First, the processing circuit 104 assigns pixel values ​​as reference values ​​to each binary representation. For example, the processing circuit 104 embeds reference values ​​represented by 2 bits by sequentially setting the pixels belonging to the first region to pixel values ​​corresponding to 00b (reference values), pixel values ​​corresponding to 01b, pixel values ​​corresponding to 10b, and pixel values ​​corresponding to 11b.

[0118] Note that, as mentioned above, the reference value can be obtained by mapping a 2-bit value to a pixel value, or by mapping a value with a depth of 3 bits or higher to a pixel value. For example, in the case of obtaining the reference value by mapping a 4-bit value to a pixel value, the binary value is 16 values ​​(or fewer) from 0000b, 0001b, 0010b, 0011b, 0100b, 0101b, 0110b, 0111b, 1000b, 1001b, 1010b, 1011b, 1100b, 1101b, 1110b, and 1111b, and the reference value is represented as information obtained by associating these values ​​with the pixel value.

[0119] In a broader sense, the processing circuit 104 assigns different pixel values ​​to a predetermined number of bits in a binary representation and embeds the reference values ​​in ascending order (00b, 01b, 10b, and 11b) of the binary representation using the assigned pixel values ​​as reference values. Note that embedding can be performed in descending order instead of ascending order.

[0120] The processing circuit 104 embeds a data start flag F1 from the pixel following the last pixel in the first region. Similar to the reference value start flag F0, the data start flag F1 can be defined and embedded. The starting position of the second region is defined by the data start flag F1.

[0121] Processing circuit 104 embeds metadata encoded based on reference values ​​into the second region. Processing circuit 104 can do this by sequentially setting the pixel values ​​of pixels belonging to the second region to, for example, [the values ​​are determined by the reference values]. Figure 3 The pixel values ​​in the third table indicate the values ​​used to embed the encoded metadata. In the case of multiple data entries to be embedded, the processing circuit 104 can sequentially embed the next data entry.

[0122] Figure 7 It shows Figure 3 The diagram illustrates an example of how data 1 is embedded. Processing circuit 104 embeds reference values ​​representing a transformation table into the image, starting from the beginning position of a first region specified by a reference value start flag, using 64, 96, 128, and 160 as pixel values, where 64, 96, 128, and 160 are reference values ​​representing 00b, 01b, 10b, and 11b, respectively.

[0123] For example, processing circuit 104 can embed pixel values ​​as values ​​representing the grayscale of the pixel (RGB color information set with the same value). The invention is not limited to this, and processing circuit 104 can embed reference values ​​as pixel values ​​with information related to any color, such as individual color information like R, G, or B, brightness information like Y, angular information in hue, or pixel-related information in other formats. This format can be appropriately selected depending on the use of encoder 10 and decoder 20.

[0124] After embedding the reference value as the pixel value, the processing circuit 104 embeds a data start flag F1 indicating the end of the first region and the beginning of the second region. Subsequently, the processing circuit 104 sequentially sets the pixel values ​​of data 1 encoded as 160, 96, 128, and 96, starting from the starting position of the second region defined by the data start flag.

[0125] Subsequently, the processing circuit 104 embeds a data end flag indicating the end of the data, namely, a data end flag F2 indicating the end of the second region.

[0126] As described above, encoder 10 is able to embed metadata into an image as information about pixel values.

[0127] Decoder 20, which extracts metadata from image data, can acquire metadata while referencing rows containing embedded data, provided that pixel values ​​in the image can be referenced. The rows containing embedded data can be predetermined and shared by encoder 10 and decoder 20. The data in these rows can be embedded into the image in some way, or detected in decoder 20 by scanning pixel values ​​to identify markers.

[0128] The processing circuit 204 obtains the configuration of the first region by detecting the reference value start flag F0.

[0129] The processing circuit 204 sequentially acquires the pixel values ​​of the first region and obtains reference value information. The processing circuit 204 acquires, for example, pixel values ​​64, 96, 128, and 160, to obtain the pixel value corresponding to 00b in the binary representation of the segmented metadata as 64, the pixel value corresponding to 01b as 96, the pixel value corresponding to 10b as 128, and the pixel value corresponding to 11b as 160.

[0130] In other words, the processing circuit 204 detects that the encoded metadata is converted to metadata by converting the pixel value to 00b when the pixel value is 64; converting the pixel value to 01b when the pixel value is 96; converting the pixel value to 10b when the pixel value is 128; or converting the pixel value to 11b when the pixel value is 160.

[0131] The processing circuit 204 detects the end of the first region and the start of the second region using the data start flag F1. The processing circuit 204 detects the encoded data of the metadata to be recovered by acquiring the pixel values ​​of 160, 96, 128, and 96 in the second region and reading the data end flag F2.

[0132] Based on the information obtained above, the processing circuit 204 extracts the binary representations 11b, 01b, 10b, and 01b represented by the pixel value arrays 160, 96, 128, and 96, combines these values, and extracts the binary representation of the metadata 01100111b. Thus, the processing circuit 204 is able to obtain the embedded binary data as the value 103.

[0133] Note that while an example of a region indicated by a marker has been described, the invention is not limited thereto. For example, encoder 10 and decoder 20 may share predetermined regions as a first region and a second region, and may embed or extract reference values ​​and encoded metadata based on this information.

[0134] Furthermore, the flag can have any length. That is, the user can specify the length of the flag and embed metadata. Any length includes a length of 0, i.e., no flag is set. In this case, for example, the embedding position of the reference value and the encoded metadata are predetermined, the encoder 10 embeds the data according to the embedding position, and the decoder 20 can read the data from the pixel value at the embedding position and decode the metadata.

[0135] This also applies to the following embodiments, and the provision of signs will be described. However, in these embodiments, the signs can also be arranged arbitrarily or have any length.

[0136] Furthermore, in the above description, a first region and a second region are shown in a single pixel row, but the invention is not limited thereto. Combinations of multiple first regions and multiple second regions can exist in a single row.

[0137] Furthermore, multiple first regions and multiple second regions can exist on a single line, regardless of their combination. In this case, data indicating which first region and which second region is the corresponding reference value and encoded metadata can be embedded, or the first region can be defined as a reference value corresponding to the encoded metadata in the second region to be embedded next. That is, the combination of first and second regions does not need to exist on the same line and can exist on different lines, or can be in a pattern of line breaks in the middle of the first or second region (the same first or second region of data spans multiple lines).

[0138] Figure 8 This is a flowchart illustrating the processing of the processing circuit 104 according to one embodiment.

[0139] Processing circuit 104 segments metadata (S100).

[0140] The processing circuit 104 obtains the encoded metadata by encoding multiple segmented metadata using reference values ​​(S102).

[0141] The processing circuit 104 designates a first region (S104). As described above, this designation can be done, for example, by embedding a mark into an image.

[0142] The processing circuit 104 embeds the reference value as a pixel value into the designated first region (S106).

[0143] When the reference value is written, the processing circuit 104 writes a flag in the image indicating the end of the first region and the beginning of the second region (S108).

[0144] The processing circuit 104 embeds the encoded metadata as pixel values ​​into the designated second region (S110).

[0145] Processing circuit 104 determines whether all metadata has been encoded and embedded (S112), and repeats the processing that started from S100 if not all data processing is completed (S112: no).

[0146] After all data entries have been embedded, the processing circuit 104 embeds the data end marker (S114).

[0147] As described above, the processing circuit 104 can embed metadata into the image.

[0148] Figure 9 This is a flowchart illustrating the processing of the processing circuit 204 according to one embodiment.

[0149] Processing circuit 204 detects a first region (S200). The first region can be detected by scanning the interior of the image or by scanning a predetermined row in the image and finding a reference value start marker. As mentioned above, this marker is not a necessary configuration, and in the absence of the marker, processing circuit 204 obtains the location of the first region by an appropriate method, such as by specifying a location shared by encoder 10 and decoder 20.

[0150] Processing circuit 204 reads the reference value embedded in the first region (S202).

[0151] The processing circuit 204 detects the end of the first region and the beginning of the second region by detecting a data start flag (S204). This flag is similar to that described above, and in the absence of the flag, the processing circuit 204 obtains the position of the second region by an appropriate method.

[0152] The processing circuit 204 reads the encoded data from the second region (S206).

[0153] The processing circuit 204 decodes multiple data segments based on reference values ​​from the read encoded data (S208).

[0154] The processing circuit 204 obtains metadata by combining the segmented data using a predetermined method (S210).

[0155] The processing circuit 204 repeats the processing that started from S200 until all metadata has been extracted (S212: No), and when all metadata has been extracted (S212: Yes), the metadata extraction process ends.

[0156] As described above, according to this embodiment, reference values ​​and metadata can be embedded in predetermined rows. By embedding data as described above, it is thus possible to appropriately encode metadata into image data and decode metadata from image data without compromising image visibility. Reference values ​​are embedded in an encoding table, and therefore the information processing apparatus can achieve robust embedding and reading of metadata, including those robust to image degradation and pixel value conversions caused by predetermined processing.

[0157] Furthermore, areas in the image information that are not displayed as images can be used as embedding locations for metadata. In this case, for example, image data acquired by decoder 20 can be displayed while masking rows containing embedded metadata.

[0158] (Second Implementation)

[0159] Metadata embedding can span multiple lines. As a non-restrictive example, a combination of reference values ​​and metadata can be embedded in each of multiple lines.

[0160] Figure 10 This is a diagram illustrating an example of metadata embedding according to one implementation method. For example... Figure 10 As explained, some metadata and reference values ​​used for encoding can be embedded in one line, and other metadata and reference values ​​used for encoding can be embedded in other lines.

[0161] In this illustration, two consecutive rows are used, but the invention is not limited thereto. Two separate rows can be used as another example.

[0162] Reference values ​​R00 and R10 can be different, but from the perspective of visual appeal for the user viewing the image, it is preferable to use different combinations when using two consecutive lines. Multiple lines can still be used even when the area containing embedded metadata is masked and displayed, and similarly, the reference values ​​can be the same combination or different combinations.

[0163] Furthermore, when the reference value is the same across multiple rows, the reference value can be set in the first row, and the same reference value can be used in subsequent rows. In this case, a first region is further set in the middle, thus enabling the same reference value to be used in multiple rows, and different reference values ​​to be used in subsequent rows.

[0164] For example, Figure 3 Data 1 and data 2 can be embedded in different rows. The length of the second region can be different for each row, depending on the length of the data. Similarly, the length of the first region can be different for each row.

[0165] As described above, the first row has a first region and a second region, the second row has a first region and a second region, and metadata can be embedded in multiple rows.

[0166] (Third implementation method)

[0167] When using multiple lines, reference values ​​and metadata can be embedded in the same line as described above, but the invention is not limited thereto.

[0168] Figure 11 This is a diagram illustrating another example of embedding data across multiple rows. For example... Figure 11 As shown, the rows containing embedded reference values ​​and the rows containing embedded encoded metadata can be separate rows.

[0169] That is, the first area can be set in the first row, and the second area can be set in the second row.

[0170] Reference values ​​are embedded in a first region, for example, specified by a start flag F0 and an end flag F1. For example, encoded metadata is embedded in a second region, specified by a start flag F3 and an end flag F4.

[0171] Note that in Figure 11 In this invention, the positions of the markers, the first region, and the second region are aligned in the up and down directions, but the invention is not limited thereto. For example, the second region may be longer than the first region, and in this case, the end marker may appear at an offset position in the up and down directions.

[0172] As mentioned above, the rows containing embedded reference values ​​and the rows containing embedded metadata can be separate rows. Similar to the description above, in this example, multiple non-contiguous rows can also be embedded instead of contiguous rows.

[0173] In the above, reference values ​​can be repeatedly embedded. For example, when reference values ​​are defined as R0 to Rn, the pixel values ​​of R0, R1, ..., and Rn can be embedded in the first region, and then the pixel values ​​of R0, R1, ..., and Rn can be further repeatedly embedded. The repetition can be two embeddings of pixel values, or it can be more than one embedding. Furthermore, the number of repetitions can be varied depending on the length of the encoded metadata in the second region.

[0174] Note that in the case of repeated embedded reference values, it is not necessary to describe all R0 to Rn in the last repetition. For example, depending on the length of the first region and / or the second region, the last repetition can be ended by embedding reference values ​​R0 to Rk (k < n).

[0175] As a non-limiting method for compressing video information, there exists a situation where compression is performed based on the pixel values ​​(of multiple pixels) directly above the pixel to be compressed. In such compression, for example, when embedding is performed using parameters of the same type (such as reference values ​​and encoded metadata represented by luminance values), it is desirable for the rows of the first and second regions to be embedded close to each other. In this embodiment, as described above, compression errors can be suppressed by repeatedly embedding reference values.

[0176] (Fourth Implementation)

[0177] In each of the above embodiments, a value is embedded in a pixel, but the present invention is not limited thereto.

[0178] Figure 12 This is an illustration showing an example of embedding according to one embodiment. As shown in the illustration, for example, the same value can be embedded in multiple consecutive pixels in the horizontal (or vertical) direction. For example, pixels in the dashed lines can have the same pixel value. Figure 12 In the example, the same pixel value is set for every two pixels in the image in the horizontal direction.

[0179] Figure 13 This is a diagram illustrating an example of an embedding according to one implementation. For example... Figure 13 As shown, the same value can be written to multiple pixels in both the horizontal and vertical directions. For example, as... Figure 13 As shown, the same pixel value can be written for every 2×2 pixels, and reference values ​​and encoded metadata can be embedded.

[0180] That is, in an image, at least one of the reference value and the encoded metadata can be embedded as the value of a pixel group comprising multiple pixels.

[0181] The implementation is performed as described above, and therefore metadata can be appropriately embedded even in formats that apply the same pixel value across multiple pixels, such as video transmission systems like Yuv422, Yuv411, and Yuv420.

[0182] Furthermore, data is embedded for each pixel, and therefore, due to the thinning of information in Yuv, the pixel values ​​for which data is to be embedded change. According to this embodiment, metadata can be embedded using pixel groups whose pixel values ​​do not change even after such data thinning (e.g., chroma component thinning) in Yuv.

[0183] (Fifth implementation method)

[0184] In the above description, the same pixel value is set for multiple pixels, but a dummy pixel value can be provided for one pixel value.

[0185] Figure 14 This is a diagram illustrating an example of metadata embedding according to one implementation. As shown in the diagram, for example, reference values ​​may include pixels with a pixel value of 0, such as R00, 0, R01, 0, R02, ... etc. The reference value may be a maximum value instead of 0. Similarly, pixel 0 may be inserted between multiple coded metadata entries.

[0186] In addition, 0 can be inserted between the signs.

[0187] Figure 15 This is an illustration of an example of metadata embedding according to one implementation. As shown in the figure, pixels with a pixel value of 0 can be sandwiched in the vertical direction instead of the horizontal direction. Furthermore, Figure 14 and Figure 15 They can be used in combination.

[0188] In other words, the first region may include pixels with predetermined values ​​in addition to pixels with pixel values ​​indicating reference values, and the second region may include pixels with predetermined values ​​in addition to pixels with pixel values ​​indicating encoded metadata. As another example, pixels with predetermined values ​​may be embedded only in the first region, or pixels with predetermined values ​​may be embedded only in the second region.

[0189] As described above, the pixels to be embedded are sparsified and the data is embedded. This enables the embedding of metadata that is more robust to pixel value degradation caused by compression.

[0190] Note that, as Figure 14 and Figure 15 As shown, when inserting pixels with dummy pixel values, multiple consecutive pixels can be used as dummy values. For example, information such as R00, 0, 0, R01, ... can be embedded as reference values. A similar situation applies to encoded metadata.

[0191] (Sixth Implementation Method)

[0192] Figure 16 This is an illustration showing another example of a pixel with another pixel value sandwiched between pixels to be embedded. The pixels indicated by diagonal lines in the accompanying figure are pixels with the pixel values ​​of the original image. As mentioned above, metadata can also be embedded by arranging reference values, encoded metadata, and flags to sandwich the original pixel values ​​between them.

[0193] The accompanying drawings illustrate a pattern for embedding data for each pixel and providing pixel values ​​for the original image, but the invention is not limited thereto. For example, embedded data of multiple consecutive pixels and image data of multiple consecutive pixels may be arranged alternately.

[0194] Furthermore, consecutive pixels with multiple embedded data values ​​and consecutive pixels with multiple image data values ​​can be arranged in the same number alternately, or in different numbers alternately. For example, pixels with one pixel of embedded data and pixels with two consecutive image data values ​​can be arranged alternately.

[0195] In other words, the first region may include pixels with pixel values ​​indicating reference values, as well as pixels with pixel values ​​in the original image data; or the second region may include pixels with pixel values ​​indicating encoded metadata, as well as pixels with pixel values ​​in the image data. As another example, the first region may include only pixels with pixel values ​​in the original image data, or the second region may include only pixels with pixel values ​​in the original image data.

[0196] Figure 17 This diagram illustrates an example of a similar implementation across multiple rows. As shown, when multiple data entries are embedded consecutively across multiple rows, the data entries can be embedded in a checkerboard pattern, arranging the original pixel values. Although only the reference value portion is shown in the diagram, pixel values ​​can, of course, be similarly incorporated into areas containing flags and multiple coded metadata.

[0197] Similar to Figure 16 In this case, the number of pixels of consecutive pixels with pixel values ​​of the image and the number of pixels of consecutive pixels with pixel values ​​of embedded data bars can be arbitrarily determined.

[0198] As mentioned above, embedding multiple metadata entries can improve the visibility of the displayed images for the user.

[0199] Furthermore, compared to the case of regions with embedded data, as described as a non-limiting example of the first embodiment, regions with embedded metadata can be made less conspicuous in the image.

[0200] For example, post-processing circuit 206 can perform interpolation on the pixel values ​​of pixels that have embedded data, based on the pixel values ​​of the pixels having the image's pixel values. By doing so, metadata embedding can be achieved without further degrading visibility.

[0201] (Seventh Implementation)

[0202] In the examples above, all reference values ​​are arranged in ascending order (e.g., the order of 64, 96, 128, and 160), but the invention is not limited thereto.

[0203] Figure 18 This is a diagram illustrating an example of metadata embedding according to one implementation. As shown in the diagram, the order of reference values ​​can be arbitrary rather than ascending. In this case, for example, in the conversion of the binary representation of segmented metadata bars, 00b can be defined as pixel value 64, 01b can be defined as pixel value 128, 10b can be defined as pixel value 96, and 11b can be defined as pixel value 160.

[0204] Any order can be an order that indicates a predetermined size relationship. For example, in Figure 18 In this context, the reference values ​​are arranged in a predetermined order: initial value (00b), larger (01b), smaller (10b), and larger (11b). Here, larger represents the comparison result with the previous reference value (grayscale value). For example, since the reference values ​​in the first region are in the order 64 < 128 > 96 < 160 starting from 00b, the inequality signs of larger (64 "<" 128), smaller (128 ">" 96), and larger (96 "<" 160) can be interpreted as an increase or decrease in grayscale starting from 01b.

[0205] As mentioned above, reference values ​​can be embedded into an order that has any size relationship.

[0206] The predetermined order can also be given meaning. In other words, predetermined data can be associated with the order of magnitude of reference values. For example, predetermined data is metadata indicating coordinate information in an image in ascending order, and in large, small, and large orders (…). Figure 18 Metadata indicating timestamps, and metadata indicating frame IDs in the cases of large, large, and small, etc., can give meaning to the metadata that is to be decoded using reference values.

[0207] Therefore, it is possible to provide information that assigns meaning to metadata in the above-described embodiments without increasing the amount of image data or changing the length of metadata.

[0208] (Eighth Implementation)

[0209] When using multiple rows in the same way as in the third embodiment, it is possible to embed the reference value in the first region and the data in the second region for each block on the multiple rows.

[0210] Figure 19 This is a diagram illustrating another example of embedding data across multiple rows. For example... Figure 19As shown, the rows for embedding reference values ​​and the rows for embedding encoded metadata can be set as separate rows, and the reference values ​​and encoded metadata can be arranged together for each block.

[0211] That is, the first region can be set in the first row, the second region can be set in the second row, the region where the first and second regions are combined can be divided for each block, and the reference values ​​and metadata can be arranged centrally for each block.

[0212] Reference values ​​are embedded in a first region, for example, specified by a start flag F0 and an end flag F1. Metadata is embedded in a second region, for example, specified by a start flag F3 and an end flag F4.

[0213] R00, R01, ... (which serve as reference values ​​arranged in the first block) can be, for example, reference values ​​corresponding to D00, D01, ... (which serve as multiple metadata entries). The same applies to the second block, and D10, D11, ... can be arranged as multiple metadata entries corresponding to R10, R11, ... As described above, reference values ​​and metadata can be arranged for each block.

[0214] As mentioned above, the rows containing embedded reference values ​​and the rows containing embedded metadata can be separate rows, and the reference values ​​and metadata can be arranged for each block.

[0215] The block size can be set based on, for example, the method used to compress the data to be sent and received. For instance, in the case where the VDC-M compression algorithm is used to display data, the block size can be set to 8×2. This is described as a non-limiting example, and the block size can be set by the compression algorithm or other algorithms related to data arrangement.

[0216] In the first and second rows, the positions of the marker portions can be staggered, or they can be in the same position as shown in the figure. Furthermore, while the above description uses two rows to form a block, the invention is not limited to this, and the block can include any number of rows suitable for algorithms such as compression. An arbitrary number of rows can include a single row. Moreover, when the number of rows is three or more, the number of rows embedding reference values ​​and the number of rows embedding metadata can also be arbitrarily set.

[0217] Similar to Figure 11 In this case, as a non-limiting method for compressing video information, compression can be performed based on the pixel value directly above the pixel to be compressed. In such compression, for example, when embedding is performed using parameters of the same type (such as reference values ​​represented by luminance values ​​and encoded metadata), it is desirable that the rows embedded in the first and second regions be close to each other. In this embodiment, as described above, because the reference values ​​and metadata can be embedded in the same block, compression errors can be suppressed.

[0218] (Ninth Implementation)

[0219] Figure 20 This is an illustration of an example of an electronic device according to one embodiment. The electronic device 1 includes an encoder 10, a storage unit 12, and an output unit 14. The encoder 10 can acquire an image stored in the storage unit 12, embed metadata, and output the metadata through the output unit 14. As described above, the encoder 10 can be embedded in any electronic device.

[0220] Figure 21 yes Figure 20 For example, electronic device 1 can be a solid-state imaging device. In this case, electronic device 1 includes imaging unit 16. Encoder 10 can acquire image data captured by imaging unit 16 and embed metadata.

[0221] Note that in Figure 21 In this mode, at least a portion of the imaging unit 16 and the encoder 10 may be disposed within the same semiconductor chip.

[0222] Note that circuitry for generating images, such as a graphics processing unit (GPU), can be disposed between the storage unit 12 or the imaging unit 16 and the encoder 10. In this case, the GPU can perform part of the encoder 10's processing by using high parallel computing power, not just for generating images. That is, the circuitry, such as the GPU, can be configured differently from the encoder 10 of the electronic device 1, or it can be configured as dedicated hardware for mounting the encoder 10.

[0223] Figure 22 This is an illustration of an example of an electronic device according to one embodiment. The electronic device 2 includes a decoder 20, an acquisition unit 22, and an output unit 24. The decoder 20 can acquire image data acquired by the acquisition unit 22, acquire embedded metadata, and output the embedded metadata from the output unit 24.

[0224] Figure 23 yes Figure 22 For example, electronic device 2 can be a display device. In this case, electronic device 2 includes display unit 26. Decoder 20 can acquire image data acquired by acquisition unit 22, extract metadata, perform image processing appropriately, and display the image data on display unit 26. Decoder 20 can perform image processing by using the extracted metadata. Acquisition unit 22 can be a receiving unit, and in this case, electronic device 2 can operate as a display device that receives data and displays images.

[0225] Figure 24This is a diagram illustrating an example of a system combining the above embodiments. System 3 includes, for example, an electronic device 1 as a solid-state imaging device and an electronic device 2 as a display device.

[0226] Electronic device 1 embeds metadata into the captured image information and sends the image information.

[0227] Electronic device 2 extracts metadata from the image information received by electronic device 1 and displays the metadata.

[0228] As described above, the information processing apparatus described in each of the above embodiments may be incorporated as part of the construction of a solid-state imaging apparatus or display apparatus, or as part of a system such as broadcasting or motion picture distribution.

[0229] Note that examples of processing implemented in each component have been described above, but at least a portion of the processing can be executed by a general-purpose processing circuit (processor). In this case, the general-purpose processing circuit can implement the processing by reading a program describing at least a portion of the processing, an executable file, or an equivalent of a program stored in a storage unit such as a memory or storage device. As mentioned above, information processing via software can be specifically implemented using hardware resources.

[0230] The above example can be implemented in the following ways.

[0231] (1) An information processing device, comprising: Processing circuit, The processing circuit acquires image data, embeds reference values ​​into pixels in a first region of the image area indicating the image data, embeds encoded metadata obtained by converting metadata using reference values ​​into pixels in a second region of the image area indicating the image data, and outputs image data containing the values ​​embedded with reference values ​​and encoded metadata.

[0232] (2) The information processing device according to (1), The reference value indicates a value in a table that defines the conversion between pixel values ​​in the image data and values ​​in the metadata.

[0233] (3) The information processing device according to (2), The reference value is defined for a combination of pixel value and a predetermined number of bits.

[0234] (4) The information processing device according to (3), The processing circuit generates a bit string obtained by dividing the metadata into bits according to a predetermined number of bits, converts the bit string into a converted pixel value based on a reference value, and sets the pixel value of the pixel arranged in the second region to the converted pixel value.

[0235] (5) The information processing device according to (3), The processing circuit assigns different pixel values ​​to binary representations of a predetermined number of bits, and embeds the assigned pixel values ​​as reference values ​​into the first region in descending or ascending order of binary representation.

[0236] (6) The information processing device according to (5), The processing circuit generates a binary representation bit string obtained by dividing the binary representation of metadata into bits according to a predetermined number of bits, and generates encoded metadata by converting the binary representation bit string using a reference value.

[0237] (7) The information processing device according to (5), The predetermined number of digits is 4. Furthermore, the processing circuit sets pixel values ​​corresponding to the 4-bit binary values ​​0000b, 0001b, 0010b, 0011b, 0100b, 0101b, 0110b, 0111b, 1000b, 1001b, 1010b, 1011b, 1100b, 1101b, 1110b, and 1111b, sequentially sets the pixel values ​​corresponding to the 4-bit values ​​in the first region, and embeds the pixel values ​​as reference values.

[0238] (8) The information processing device according to (5), The processing circuit sets the pixel values ​​of the reference values ​​to a predetermined order in binary representation.

[0239] (9) The information processing device according to (8), The processing circuit defines reference values ​​for the attributes of the metadata in a predetermined order.

[0240] (10) The information processing device according to (5), The processing circuit is configured in the pixel values ​​of the first region or the second region to include at least one of the following pixels: a pixel in the first region that has a predetermined pixel value, which is different from a pixel value having an indicative reference value, and a pixel in the second region that has a predetermined pixel value, which is different from a pixel value having an indicative encoded metadata value.

[0241] (11) The information processing device according to (5), The processing circuit is configured in the pixel values ​​of the first region or the second region to include at least one of the following pixels: a pixel in the first region that has a pixel value in the image data that is different from a pixel value having an indicative reference value, and a pixel in the second region that has a pixel value in the image data that is different from a pixel value having a pixel value having an indicative encoded metadata.

[0242] (12) An information processing apparatus according to any one of (1) to (11), The first region and the second region are set in the first predetermined row.

[0243] (13) An information processing apparatus according to any one of (1) to (11), The first region is set in the first predetermined row, and The second area is set in the second predetermined line.

[0244] (14) An information processing apparatus according to any one of (1) to (11), Specifically, a first region and a second region are set for the first predetermined line, and Set up a first area and a second area for the second pre-defined line.

[0245] (15) An information processing apparatus according to any one of (1) to (11), In this context, at least one of the reference value and the encoded metadata is embedded as the value of a pixel group comprising multiple pixels.

[0246] (16) The information processing apparatus according to any one of (1) to (15) further includes: Storage unit, The processing circuit acquires the image data stored in the storage unit and embeds the reference value and the value of the encoded metadata into the image data.

[0247] (17) A solid-state imaging device, comprising: An information processing device according to any one of (1) to (15); and Imaging unit, The information processing device acquires the image captured by the imaging unit as image data and embeds the values ​​of reference values ​​and encoded metadata into the image data.

[0248] (18) Based on the solid-state imaging device of (17), The information processing device and the imaging unit are housed in the same semiconductor chip.

[0249] (19) An information processing apparatus, comprising: Processing circuit, The processing circuit acquires image data by reading reference values ​​from pixels in a first region of the image area indicating the image data; reads encoded metadata from pixels in a second region of the image area indicating the image data; and acquires metadata obtained by converting the encoded metadata using the reference values.

[0250] (20) A display device, comprising: Receiving unit; According to the information processing device of (19); and Display unit, The receiving unit receives the transmitted data as image data, and The processing circuit obtains metadata from the received image data, and Display image data on the display unit.

[0251] (21) Based on the display device of (20), The processing circuit performs signal processing on the image data based on metadata, and The processed image data is displayed on the display unit.

[0252] (22) A transceiver system, comprising: First information processing device and second information processing device The first information processing device acquires image data, embeds reference values ​​into pixels in a first region of the image area indicating the image data, embeds encoded metadata obtained by converting metadata using the reference values ​​into pixels in a second region of the image area indicating the image data, and transmits image data containing the embedded reference values ​​and encoded metadata. The second information processing device receives image data sent from the first information processing device, reads reference values ​​from pixels in a first region arranged in the area of ​​the image indicating the image data, reads encoded metadata from pixels in a second region arranged in the area of ​​the image indicating the image data, and obtains metadata obtained by converting the encoded metadata using the reference values.

[0253] (23) A program for causing a processor to perform at least one of the processes described in (1) to (22).

[0254] This disclosure is not limited to the embodiments described above, but includes various conceivable modifications, and the effects of this disclosure are not limited to the above content. Components in each embodiment can be appropriately combined and applied. That is, various additions, changes, and partial deletions can be made without departing from the conceptual idea and spirit of this disclosure derived from the content defined in the claims and their equivalents.

[0255] Reference Symbol List

[0256] 1. Electronic device

[0257] 10 Encoders

[0258] 100 Image Acquisition Circuit

[0259] 102 Preprocessing Circuit

[0260] 104 Processing Circuit

[0261] 106 Post-processing circuit

[0262] 108 Output Circuit

[0263] 12 storage units

[0264] 14 Output Unit

[0265] 16 imaging units

[0266] 2 Electronic devices

[0267] 20 decoders

[0268] 200 Image Acquisition Circuit

[0269] 202 Preprocessing Circuit

[0270] 204 Processing Circuit

[0271] 206 Post-processing circuit

[0272] 208 Output Circuit

[0273] 22 Acquisition Unit

[0274] 24 Output Units

[0275] 26 display units

[0276] 3. System.

Claims

1. An information processing apparatus, comprising: Processing circuit, The processing circuit acquires image data, embeds reference values ​​into pixels in a first region of an image region indicating the image data, embeds encoded metadata obtained by converting metadata using the reference values ​​into pixels in a second region of the image region indicating the image data, and outputs the image data containing the values ​​embedded with the reference values ​​and the encoded metadata.

2. The information processing device according to claim 1, in, The reference value indicates a value in a table that defines the conversion between pixel values ​​in the image data and values ​​in the metadata.

3. The information processing device according to claim 2, in, The reference value is defined for a combination of pixel value and a predetermined number of bits.

4. The information processing device according to claim 3, in, The processing circuit generates a bit string by dividing the metadata into segments of predetermined bits. Based on the reference value, the bit string is converted into a converted pixel value, and The pixel values ​​of the pixels arranged in the second region are set to the pixel values ​​of the transformation.

5. The information processing device according to claim 3, in, The processing circuit assigns different pixel values ​​to the binary representation of the predetermined number of bits, and uses the assigned pixel values ​​as reference values ​​to embed them into the first region in descending or ascending order of the binary representation.

6. The information processing apparatus according to claim 5, in, The processing circuit generates a binary representation bit string obtained by dividing the binary representation of the metadata into bits according to a predetermined number of bits, and converts the binary representation bit string using the reference value to generate the encoded metadata.

7. The information processing apparatus according to claim 5, in, The predetermined number of bits is 4. Furthermore, the processing circuit sets pixel values ​​corresponding to the 4-bit binary values ​​0000b, 0001b, 0010b, 0011b, 0100b, 0101b, 0110b, 0111b, 1000b, 1001b, 1010b, 1011b, 1100b, 1101b, 1110b, and 1111b. In the first region, pixel values ​​corresponding to 4-bit values ​​are sequentially set, and the pixel values ​​are embedded as the reference values.

8. The information processing apparatus according to claim 5, in, The processing circuit is configured in the first region or the second region to include at least one of the following pixels: a pixel in the first region that has a predetermined pixel value, which is different from a pixel having a pixel value indicating the reference value, and a pixel in the second region that has a predetermined pixel value, which is different from a pixel having a pixel value indicating the encoded metadata.

9. The information processing apparatus according to claim 5, in, The processing circuit is configured in the first region or the second region to include at least one of the following pixels: a pixel in the first region that has a pixel value in the image data that is different from a pixel value that indicates the reference value, and a pixel in the second region that has a pixel value in the image data that is different from a pixel value that indicates the encoded metadata.

10. The information processing apparatus according to claim 1, in, The first region and the second region are set in the first predetermined row.

11. The information processing apparatus according to claim 1, in, The first region is set in the first predetermined row, and The second region is set in the second predetermined row.

12. The information processing apparatus according to claim 1, in, The first region and the second region are set for the first predetermined line, and Set the first region and the second region for the second predetermined line.

13. The information processing apparatus according to claim 1, in, At least one of the reference value and the encoded metadata is embedded as a value of a pixel group comprising multiple pixels.

14. The information processing apparatus according to claim 1, further comprising: Storage unit, The processing circuit acquires the image data stored in the storage unit and embeds the reference value and the value of the encoded metadata into the image data.

15. A solid-state imaging device, comprising: The information processing apparatus according to claim 1; as well as Imaging unit, The information processing device acquires the image captured by the imaging unit as the image data, and embeds the reference value and the value of the encoded metadata into the image data.

16. The solid-state imaging device according to claim 15, in, The information processing device and the imaging unit are housed in the same semiconductor chip.

17. An information processing apparatus, comprising: Processing circuit, The processing circuit acquires image data, reads reference values ​​from pixels in a first region of an image region indicating the image data, reads encoded metadata from pixels in a second region of the image region indicating the image data, and acquires metadata obtained by converting the encoded metadata using the reference values.

18. A display device, comprising: Receiving unit; The information processing apparatus according to claim 17; as well as Display unit, The receiving unit receives the transmitted data as image data, and The processing circuit obtains the metadata from the received image data and displays the image data on the display unit.

19. The display device according to claim 18, in, The processing circuit performs signal processing on the image data based on the metadata, and displays the signal-processed image data on the display unit.

20. A transceiver system, comprising: First information processing device and second information processing device The first information processing device acquires image data, embeds reference values ​​into pixels in a first region of an image representing the image data, embeds encoded metadata obtained by converting metadata using the reference values ​​into pixels in a second region of the image representing the image data, and transmits the image data containing the embedded reference values ​​and the encoded metadata. The second information processing device receives the image data sent from the first information processing device, reads the reference value from pixels in the first region arranged in the area of ​​the image indicating the image data, reads the encoded metadata from pixels in the second region arranged in the area of ​​the image indicating the image data, and obtains metadata obtained by converting the encoded metadata using the reference value.

Images