Image compression device

Abstract

This image compression device (1) comprises: an acquisition unit (10) that acquires MIPI image data in which respective images of a plurality of virtual channels are superimposed in units of lines constituting the images; an identification unit (20) that identifies, on the basis of identifiers added to each of a plurality of lines included in the MIPI image data, whether each of the plurality of lines is a line of an image of one of the plurality of virtual channels, and simultaneously outputs two or more identified lines of different virtual channel images; a buffer unit (30) that has a plurality of N-line buffers corresponding to the plurality of virtual channels, receives the two or more lines, and stores each of the two or more lines in an N-line buffer of the corresponding virtual channel on the basis of the identifiers added to each of the two or more lines; and a compression unit (50) that reads the data stored in the plurality of N-line buffers and compresses the respective images of the plurality of virtual channels.

Description

Image Compression Device

[0001] The present disclosure relates to an image compression device.

[0002] Japanese Patent Application Laid-Open No. 2003-144222 discloses a technique relating to an image processing device that can improve the efficiency of image processing.

[0003] JP 2010-263475 A

[0004] In recent years, a technology has been proposed that uses MIPI (Mobile Industry Processor Interface)-compliant MIPI image data to enable simultaneous processing of multiple images from multiple cameras by transmitting them through a single interface, thereby improving image processing efficiency. However, when this technology is used, a separate interface for transmitting and receiving data is required for each of up to four virtual channels, which increases the circuit size.

[0005] Therefore, the present disclosure provides an image compression device that can reduce the circuit scale.

[0006] The image compression device according to the present disclosure includes an acquisition unit that acquires MIPI image data that conforms to the MIPI standard, in which images of a plurality of virtual channels are superimposed on a line-by-line basis to form the image; an identification unit that identifies, based on an identifier added to each of the plurality of lines included in the MIPI image data, which of the plurality of virtual channels each corresponds to an image line of, and outputs two or more of the identified lines of the images of the different virtual channels simultaneously; a buffer unit that has a plurality of N-line buffers (N is an integer greater than or equal to 1) corresponding to the plurality of virtual channels, that receives the two or more lines and stores each of the two or more lines in the N-line buffer of the corresponding virtual channel based on the identifier added to each of the two or more lines; and a compression unit that reads out the data stored in the plurality of N-line buffers and compresses the images of the plurality of virtual channels.

[0007] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0008] According to an image compression device according to an aspect of the present disclosure, the circuit scale can be reduced.

[0009] It is a block diagram showing an example of an image compression device according to an embodiment. It is a diagram for explaining MIPI image data. It is a diagram for explaining the operation of a discrimination unit and a buffer unit according to an embodiment. It is a diagram for explaining the operation of an arbitration unit according to an embodiment. It is a diagram for explaining the effect of a two-bank configuration.

[0010] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0011] The embodiments described below are all comprehensive or specific examples, and the numerical values, shapes, materials, components, arrangement and connection of the components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure.

[0012] (Embodiment) An image compression device according to an embodiment will be described below.

[0013] 1 is a block diagram showing an example of an image compression device 1 according to an embodiment. In addition to the image compression device 1, cameras 100a to 100d and a frame memory 200 are also shown in FIG.

[0014] The image compression device 1 is a device that compresses MIPI image data that conforms to the MIPI standard. MIPI image data is image data in which images from multiple virtual channels are superimposed on a line-by-line basis. Here, MIPI image data will be described in comparison with normal image data using FIG. 2 .

[0015] Fig. 2 is a diagram for explaining MIPI image data, in which the structure of one frame of normal image data is shown in the upper part of Fig. 2, and the structure of one frame of MIPI image data is shown in the lower part of Fig. 2.

[0016] As shown in Figure 2, in normal image data, one frame of data for one system (e.g., one frame of data from one camera) is transmitted, whereas in MIPI image data, one frame of data for up to four systems (in other words, four virtual channels) (e.g., four one-frame data from four cameras) is superimposed and transmitted line by line. Note that a line is a group of pixels in one row in an image. For example, when the resolution is 1280 x 720, a line is a group of pixels consisting of 1280 pixels extending horizontally.

[0017] In the time it takes to transmit one frame of data for one system of normal image data, up to four systems of MIPI image data can be transmitted using high-speed differential signals. In MIPI image data, a virtual channel (VC) exists for each system, and here the four virtual channels are referred to as VC0, VC1, VC2, and VC3, respectively.

[0018] Since video is made up of a plurality of consecutive images, the image compression device 1 can also compress video by compressing a plurality of consecutive images. Therefore, the image compression device 1 is also a video compression device that compresses MIPI video data.

[0019] Returning to the explanation of FIG. 1 , the image compression device 1 includes an acquisition unit 10, an identification unit 20, a buffer unit 30, an arbitration unit 40, and a compression unit 50. The image compression device 1 is a computer including a processor (microprocessor) and a memory. The memory may be a read-only memory (ROM) or a random access memory (RAM), and can store a program executed by the processor. The identification unit 20, the buffer unit 30, the arbitration unit 40, and the compression unit 50 are realized by the processor executing a program stored in the memory.

[0020] The acquisition unit 10 acquires MIPI image data. The acquisition unit 10 is, for example, a D-PHY (physical layer). For example, the acquisition unit 10 acquires MIPI image data from four cameras 100a, 100b, 100c, and 100d, in which images from four virtual channels are superimposed on a line-by-line basis. The acquisition unit 10 may also acquire MIPI image data from a memory such as a RAM or a hard disk that can temporarily store and read data. For example, the acquisition unit 10 converts serial data into byte-by-byte data. The following describes an example in which the multiple virtual channels are four virtual channels. Note that this is just an example, and the multiple virtual channels are not limited to four virtual channels.

[0021] The identification unit 20 identifies which of the four virtual channels each of the multiple lines belongs to, based on an identifier added to each of the multiple lines included in the MIPI image data, and simultaneously outputs two or more of the identified lines of the images of the different virtual channels.

[0022] The buffer unit 30 has a plurality of N-line buffers (N is an integer equal to or greater than 1) corresponding to four virtual channels. For example, the buffer unit 30 has an 8-line buffer 32a corresponding to VC0, an 8-line buffer 32b corresponding to VC1, an 8-line buffer 32c corresponding to VC2, and an 8-line buffer 32d corresponding to VC3. Each 8-line buffer is capable of storing eight lines. In the following, an example will be described in which the plurality of N-line buffers are four 8-line buffers 32a, 32b, 32c, and 32d. Note that this is just an example, and the N-line buffer is not limited to an 8-line buffer, and the plurality of N-line buffers is not limited to four N-line buffers.

[0023] The buffer unit 30 has a memory interface that receives two or more lines output from the identification unit 20 .

[0024] Here, the operations of the identification unit 20 and the buffer unit 30 will be described with reference to FIG.

[0025] FIG. 3 is a diagram for explaining the operations of the identifying unit 20 and the buffer unit 30 according to the embodiment.

[0026] 3, the identification unit 20 reads the VC-IDs added to the communication packet headers of each of the multiple lines included in the MIPI image data and separates the multiple lines into virtual channels. The identification unit 20 then simultaneously outputs to the memory interface two or more combinations of VC data (line data) and their VC-IDs, each with a different VC-ID. For example, the identification unit 20 simultaneously outputs a combination of VC0 data and a VC-ID indicating VC0, and a combination of VC1 data and a VC-ID indicating VC1, and also simultaneously outputs a combination of VC2 data and a VC-ID indicating VC2, and a combination of VC3 data and a VC-ID indicating VC3.

[0027] The buffer unit 30 receives two or more lines from the identification unit 20. For example, the buffer unit 30 has memory interfaces 31a and 31b. The buffer unit 30 receives the VC0 and VC1 lines output simultaneously from the identification unit 20 at the memory interface 31a, and receives the VC2 and VC3 lines output simultaneously from the identification unit 20 at the memory interface 31b.

[0028] The buffer unit 30 stores each of the two or more lines in the 8-line buffer of the corresponding virtual channel based on the identifier attached to each of the two or more lines. For example, the buffer unit 30 stores the VC0 line in the corresponding 8-line buffer 32a based on the VC-ID attached to the VC0 line, and stores the VC1 line in the corresponding 8-line buffer 32b based on the VC-ID attached to the VC1 line. Also, for example, the buffer unit 30 stores the VC2 line in the corresponding 8-line buffer 32c based on the VC-ID attached to the VC2 line, and stores the VC3 line in the corresponding 8-line buffer 32d based on the VC-ID attached to the VC3 line.

[0029] Returning to the explanation of FIG. 1 , the buffer unit 30 includes block generators 33a, 33b, 33c, and 33d, each for one of four 8-line buffers 32a, 32b, 32c, and 32d, that generate block data from the eight lines stored in the corresponding 8-line buffer, each having a predetermined number of horizontal pixels and eight vertical pixels. For example, the predetermined number is 16. The block generator 33a generates 16×8 block data from eight lines of VC0, the block generator 33b generates 16×8 block data from eight lines of VC1, the block generator 33c generates 16×8 block data from eight lines of VC2, and the block generator 33d generates 16×8 block data from eight lines of VC3. For example, if one line has 1280 pixels, 1280 / 16 = 80 block data are generated from the eight lines. This allows compression of block data of a certain size. It should be noted that the predetermined number is not limited to 16, and as mentioned above, the N-line buffer is not limited to an 8-line buffer, so the size of the block data is not limited to 16×8.

[0030] The arbitration unit 40 receives block data from each of the four 8-line buffers 32a, 32b, 32c, and 32d and arbitrates the block data received from each of the four 8-line buffers 32a, 32b, 32c, and 32d. For example, if a compressor is provided for each 8-line buffer, the block data from each 8-line buffer can be directly compressed by the corresponding compressor. However, in this case, the number of compressors increases as the number of virtual channels (systems) increases, resulting in a larger circuit size. Therefore, the arbitration unit 40 arbitrates the block data for each virtual channel. For example, the arbitration unit 40 notifies the compressor 50 of which virtual channel the block data received from each line buffer belongs to and which pixel group in the image the block data belongs to when sequentially transmitting the block data. This allows one compressor 50 to compress the block data for the four virtual channels. In this way, the block data for the four virtual channels can be compressed by a smaller number of compressors (e.g., one compressor 50) than the number of virtual channels, thereby reducing the circuit size.

[0031] The arbitration unit 40 may count the number of received block data independently for each of the four 8-line buffers 32 a, 32 b, 32 c, and 32 d. For example, the arbitration unit 40 may have a counter corresponding to each of the four 8-line buffers 32 a, 32 b, 32 c, and 32 d. This will be described with reference to FIG. 4.

[0032] FIG. 4 is a diagram for explaining the operation of the arbitration unit 40 according to the embodiment.

[0033] For example, there may be cases where frames of multiple virtual channels are asynchronous and the transfer order of block data is irregular. Figure 4 shows an example where VC0 and VC1 frames are asynchronous and the transfer order of VC0 and VC1 block data is irregular. As shown in Figure 4, the arbitration unit 40 counts the number of VC0 block data received from the 8-line buffer 32a from the start timing of the VC0 frame, and independently counts the number of VC1 block data received from the 8-line buffer 32b from the start timing of the VC1 frame. The start timing of a frame can be recognized by reading the Frame Start Packet.

[0034] This allows block data to be processed for each virtual channel, so that even if the frames of the four virtual channels are asynchronous and the transfer order of the block data is irregular, images of the four virtual channels can be processed.

[0035] Returning to the explanation of FIG. 1 , the compression unit 50 reads the data stored in the four 8-line buffers 32 a, 32 b, 32 c, and 32 d and compresses the images of the four virtual channels. Specifically, the compression unit 50 compresses the images of the four virtual channels on a block-by-block basis. The compression unit 50 may also perform encoding. The compression unit 50 then stores the compressed data for each of the four virtual channels in the frame memory 200.

[0036] At least one of the four 8-line buffers 32a, 32b, 32c, and 32d may have a multi-bank configuration that allows parallel reading and writing. For example, the effect of the buffer unit 30 having a two-bank 8-line buffer will be described with reference to FIG. 5.

[0037] FIG. 5 is a diagram for explaining the effect of the two-bank configuration.

[0038] 5, if the buffer unit 30 does not have a two-bank 8-line buffer, the buffer unit 30 writes 8 lines of data to the 8-line buffer, generates block data from the 8 lines of data written to the 8-line buffer, and the compression unit 50 compresses the generated block data. Until all 8 lines of data written to the 8-line buffer have been converted into block data and compressed, the buffer unit 30 cannot write the next 8 lines of data to the 8-line buffer, resulting in a wait for processing. The higher the image resolution, the longer the wait for processing, which could prevent high-resolution images from being compressed and stored in the frame memory.

[0039] In contrast, as shown in the lower part of FIG. 5 , if the buffer unit 30 has an 8-line buffer with a two-bank configuration, for example, the buffer unit 30 writes the first half of a line to one of the two banks (Bank 1) and the second half of the line to the other bank (Bank 2). For example, half of a line corresponds to 640 pixels when the horizontal resolution is 1280 pixels. The buffer unit 30 generates block data from the 8 lines of data written to Bank 1, and the compression unit 50 compresses the generated block data and writes it to Bank 2 in parallel. In this way, since the generation and compression of block data for the data written to Bank 1 and the writing to Bank 2 can be performed in parallel, processing waits are less likely to occur. This makes it possible to handle even high-resolution images and speed up processing.

[0040] In addition, the buffer unit 30 may use at least two of the four 8-line buffers 32a, 32b, 32c, and 32d as banks that can be read from and written to in parallel. When a virtual channel is not used, the at least two 8-line buffers used as banks can be used for only one channel, making it possible to handle even high-resolution images and speed up processing.

[0041] As described above, two or more image lines of different virtual channels included in MIPI image data are output together at the same time, eliminating the need for an interface for data transmission and reception for each of the four virtual channels. For example, by outputting image lines of two of the four virtual channel image lines together at the same time, the number of data transmission and reception interfaces can be reduced from four to two. This allows for a reduction in circuit size.

[0042] (Other Embodiments) As described above, the embodiments have been described as examples of the technology according to the present disclosure. However, the technology according to the present disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made as appropriate. For example, the following modifications are also included in one embodiment of the present disclosure.

[0043] For example, in the above embodiment, the buffer unit 30 generates block data. However, the buffer unit 30 does not have to generate block data (in other words, the buffer unit 30 does not have to include the block generators 33a, 33b, 33c, and 33d). That is, compression does not have to be performed for each block of data, but may be performed for each line. In this case, the arbitration unit 40 may receive lines from each of the four 8-line buffers 32a, 32b, 32c, and 32d and arbitrate the lines received from each of the four 8-line buffers 32a, 32b, 32c, and 32d. This allows the lines of the four virtual channels to be compressed by a smaller number of compressors than the number of virtual channels, thereby reducing the circuit size. In addition, arbitration unit 40 may count the number of lines received from each of the four 8-line buffers 32a, 32b, 32c, and 32d independently, thereby allowing processing of images from the four virtual channels even when the frames of the four virtual channels are asynchronous and the line transfer order is irregular.

[0044] For example, in the above embodiment, an example has been described in which the image compression device 1 includes the arbitration unit 40, but the image compression device 1 does not necessarily have to include the arbitration unit 40. In this case, a compression unit may be provided in a one-to-one correspondence with the 8-line buffer.

[0045] In the above embodiment, each component included in the image compression device 1 may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0046] Some or all of the functions of the image compression device 1 according to the above embodiment are typically realized as an LSI, which is an integrated circuit. These may be individually implemented on a single chip, or some or all of them may be integrated on a single chip. Furthermore, the integrated circuit is not limited to an LSI, and may be realized using a dedicated circuit or a general-purpose processor. It is also possible to use an FPGA (Field Programmable Gate Array), which can be programmed after LSI manufacture, or a reconfigurable processor, which allows the connections and settings of circuit cells within an LSI to be reconfigured.

[0047] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that each component included in the image compression device 1 may be integrated into an integrated circuit using that technology.

[0048] In addition, this disclosure also includes forms obtained by making various modifications to the embodiments that a person skilled in the art would think of, and forms realized by arbitrarily combining the components and functions in each embodiment within the scope of the present disclosure.

[0049] (Additional Notes) The above description of the embodiments discloses the following techniques.

[0050] (Technology 1) An image compression device comprising: an acquisition unit that acquires MIPI image data that conforms to the MIPI standard, in which images of multiple virtual channels are superimposed on a line-by-line basis that constitutes the image; an identification unit that identifies, based on an identifier added to each of the multiple lines included in the MIPI image data, which of the multiple virtual channels each corresponds to an image line of, and outputs two or more of the identified lines of the images of the different virtual channels together at the same time; a buffer unit that has multiple N-line buffers (N is an integer greater than or equal to 1) corresponding to the multiple virtual channels, that receives the two or more lines and stores each of the two or more lines in the N-line buffer of the corresponding virtual channel based on the identifier added to each of the two or more lines; and a compression unit that reads out data stored in the multiple N-line buffers and compresses the images of the multiple virtual channels.

[0051] According to this, two or more lines of images of different virtual channels included in MIPI image data are output together at the same time, eliminating the need for an interface for data transmission and reception for each of the multiple virtual channels. For example, by outputting lines of images of two of four virtual channels at the same time, the number of interfaces for data transmission and reception can be reduced from four to two. This allows for a reduction in circuit size.

[0052] (Technology 2) The image compression device according to Technology 1, wherein the buffer unit generates, for each of the plurality of N-line buffers, block data having a predetermined number of pixels in the horizontal direction and N pixels in the vertical direction from the N lines stored in the N-line buffer, and the compression unit compresses each image of the plurality of virtual channels for each block of data.

[0053] In this way, compression may be performed for each block of data of a certain size.

[0054] (Technology 3) The image compression device according to Technology 2, further comprising an arbitration unit, which receives block data from each of the plurality of N-line buffers and performs arbitration regarding the block data received from each of the plurality of N-line buffers.

[0055] This allows block data of a plurality of virtual channels to be compressed by a number of compression units that is less than the number of the plurality of virtual channels, thereby reducing the circuit scale.

[0056] (Technology 4) The image compression device according to Technology 3, wherein the arbitration unit counts the number of received block data independently for each of the plurality of N-line buffers.

[0057] This makes it possible to process images on a plurality of virtual channels even if the frames on the plurality of virtual channels are asynchronous and the transfer order of block data is irregular.

[0058] (Technology 5) The image compression device according to Technology 1, further comprising an arbitration unit that receives lines from each of the plurality of N-line buffers and performs arbitration regarding the lines received from each of the plurality of N-line buffers.

[0059] This allows the lines of a plurality of virtual channels to be compressed by a number of compression units that is less than the number of the virtual channels, thereby reducing the circuit scale.

[0060] (Technology 6) The image compression device according to Technology 5, wherein the arbitration unit counts the number of received lines independently for each of the plurality of N-line buffers.

[0061] This makes it possible to process images from a plurality of virtual channels even if the frames of the virtual channels are asynchronous and the order of line transfer is irregular.

[0062] (Technology 7) An image compression device according to any one of Technologies 1 to 6, wherein at least one of the plurality of N-line buffers has a multi-bank configuration that allows parallel reading and writing.

[0063] This allows processing of one bank and processing of another bank to be performed in parallel, making it possible to handle even images with high resolution and speed up processing.

[0064] (Technology 8) An image compression device according to any one of techniques 1 to 6, wherein the buffer unit uses at least two of the plurality of N-line buffers as banks that can be read and written in parallel.

[0065] According to this, when the virtual channel is not in use, at least two N-line buffers used as a bank can be used for only one channel, so that even images with high resolution can be handled and processing can be speeded up.

[0066] The present disclosure can be applied to image processing systems that handle MIPI image data.

[0067] REFERENCE SIGNS LIST 1 Image compression device 10 Acquisition unit 20 Identification unit 30 Buffer unit 31a, 31b Memory interface 32a, 32b, 32c, 32d 8-line buffer 33a, 33b, 33c, 33d Block generation unit 40 Arbitration unit 50 Compression unit 100a, 100b, 100c, 100d Camera 200 Frame memory

Claims

1. An image compression device comprising: an acquisition unit that acquires MIPI image data that conforms to the MIPI (Mobile Industry Processor Interface) standard, in which images of multiple virtual channels are superimposed on a line-by-line basis to form the image; an identification unit that identifies which of the multiple virtual channels each of the multiple lines included in the MIPI image data belongs to, based on an identifier added to each of the multiple lines, and outputs two or more lines of the images of the identified different virtual channels together, a buffer unit having multiple N-line buffers (N is an integer greater than or equal to 1) corresponding to the multiple virtual channels, that receives the two or more lines and stores each of the two or more lines in the N-line buffer of the corresponding virtual channel based on the identifier added to each of the two or more lines; and a compression unit that reads out the data stored in the multiple N-line buffers and compresses the images of the multiple virtual channels.

2. The image compression device according to claim 1, wherein the buffer unit generates, for each of the plurality of N-line buffers, block data having a predetermined number of pixels in the horizontal direction and N pixels in the vertical direction from the N lines stored in the N-line buffer, and the compression unit compresses the images of each of the plurality of virtual channels for each block of data.

3. The image compression device according to claim 2, further comprising an arbitration unit that receives block data from each of the plurality of N-line buffers and performs arbitration regarding the block data received from each of the plurality of N-line buffers.

4. The image compression device according to claim 3, wherein the arbitration unit counts the number of block data received independently for each of the plurality of N-line buffers.

5. The image compression device according to claim 1, further comprising an arbitration unit that receives lines from each of the plurality of N-line buffers and performs arbitration regarding the lines received from each of the plurality of N-line buffers.

6. The image compression device according to claim 5, wherein the arbitration unit counts the number of lines received from each of the plurality of N-line buffers independently.

7. The image compression device according to any one of claims 1 to 6, wherein at least one of said plurality of N-line buffers has a multi-bank configuration that allows parallel reading and writing.

8. The image compression device according to any one of claims 1 to 6, wherein the buffer section uses at least two of the plurality of N-line buffers as banks capable of parallel reading and writing.

Images