Image capturing system

Abstract

To provide an image capturing system that can improve performance, such as improving power efficiency and reducing communication costs, in digital image capturing accompanied by image processing.SOLUTION: The present invention is an image capturing system including an image capturing device and an image processing device. The image capturing device captures an image of a subject to be captured to obtain raw captured data, applies image processing to the raw captured data to create a pre-processed image, and transmits the pre-processed image to the image processing device. The image processing device receives the pre-processed image and applies image processing including at least de-mosaic processing to the pre-processed image to create an output image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image capture system. [Background technology]

[0002] Patent document 1 discloses a broadcasting system that performs image processing on an image captured by a camera to generate video data (or image data), transmits the generated video data to a receiving device using a transmitting device, and displays the video data on the viewer's display device, allowing the viewer to watch the video content.

[0003] In the above system, the camera comprises an image sensor and an image processing device.

[0004] The image sensor generates unprocessed raw data, which is then processed by the image processing device (such as noise reduction and interpolation) to generate video data. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6700921 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the system of Patent Document 1 requires the camera to perform high-load image processing, which increases the power consumption of the camera.

[0007] Furthermore, the video data (image data) sent from the camera contains three channels of data (RGB) per pixel, whereas the RAW data output from the image sensor generally contains one channel of data per pixel.

[0008] This increases the amount of data transmitted from the camera, which leads to problems such as increased power consumption and increased communication costs.

[0009] Therefore, there is a demand for an image capturing system that can improve performance, such as by improving power efficiency and reducing communication costs, in digital image capturing that involves image processing. [Means for solving the problem]

[0010] The first invention is an image capturing system comprising an image capturing device and an image processing device, wherein the image capturing device captures an image of an object to be captured to obtain raw captured data, applies image processing to the raw captured data to create a pre-processed image, and transmits the pre-processed image to the image processing device, and the image processing device receives the pre-processed image and applies image processing including at least demosaic processing to the pre-processed image to create an output image.

[0011] The second aspect of the present invention is an image capturing system comprising an image capturing device and an image processing device, wherein the image capturing device captures an image of a subject to be captured to obtain raw captured data, applies image processing to the raw captured data to create a pre-processed image, compresses the pre-processed image to create a compressed pre-processed image, and transmits the compressed pre-processed image, and the image processing device receives the compressed pre-processed image, decompresses the compressed pre-processed image to create a decompressed pre-processed image, and applies image processing including at least demosaic processing to the decompressed pre-processed image to create an output image.

[0012] The third aspect of the present invention is an image capturing system comprising an image capturing device and an image processing device, wherein the image capturing device captures an object to be captured to obtain raw captured data, applies image processing to the raw captured data to create a pre-processed image, changes the arrangement of image data of the pre-processed image to create an aligned pre-processed image, compresses the aligned pre-processed image to create a compressed aligned pre-processed image, and transmits the compressed aligned pre-processed image, and the image processing device receives the compressed aligned pre-processed image, decompresses the compressed aligned pre-processed image to create an decompressed aligned pre-processed image, changes the arrangement of image data of the decompressed aligned pre-processed image to an arrangement equivalent to that of the pre-processed image to create an decompressed pre-processed image, and applies image processing including at least demosaic processing to the decompressed pre-processed image to create an output image. [Effects of the Invention]

[0013] According to the present invention, it is possible to improve performance such as improving power efficiency and reducing communication costs in digital image capture accompanied by image processing. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram showing a configuration of an image capturing system according to a first embodiment. [Figure 2] 4 is a flowchart showing the overall operation of the image capturing system (image capturing device and image processing device) according to the first embodiment. [Figure 3] 5 is a flowchart showing the operation (calculation of an RGB image) of an image signal pre-processing unit and a pre-processed image input unit in the image capturing system according to the first embodiment. [Figure 4] 2A to 2C are explanatory diagrams showing examples of a Bayer image and an RGB image according to the first embodiment. [Figure 5] FIG. 10 is a block diagram showing the configuration of an image capturing system according to a second embodiment. [Figure 6] 10 is a flowchart showing the overall operation of an image capturing system (image capturing device and image processing device) according to a second embodiment. [Figure 7] FIG. 10 is a block diagram showing the configuration of an image capturing system according to a third embodiment. [Figure 8] 10 is a flowchart showing the overall operation of an image capturing system (image capturing device and image processing device) according to a third embodiment. [Figure 9] FIG. 1 is a schematic diagram (part 1) showing an example of a method for changing a pixel array. [Figure 10] FIG. 2 is a schematic diagram (part 2) showing an example of a method for changing a pixel array. DETAILED DESCRIPTION OF THE INVENTION

[0015] (A) First embodiment A first embodiment of an image capturing system according to the present invention will be described in detail below with reference to the drawings.

[0016] (A-1) Configuration of the First Embodiment <Overall structure> FIG. 1 is a block diagram showing the configuration of an image capturing system according to the first embodiment.

[0017] In FIG. 1, an image capturing system 1 includes an image capturing device 10 and an image processing device 20.

[0018] The image capturing device 10 is a device that receives input from a person or a program, captures an image of a subject, performs simple pre-image processing on the captured data, and outputs a pre-processed image.

[0019] The image processing device 20 is a device that performs main image processing on the pre-processed image output by the image capturing device 10 and outputs a final output image.

[0020] The image capturing device 10 and the image processing device 20 are physically separate devices, and are connected to each other by any communication method, whether wired or wireless.

[0021] <Image capture device> As shown in FIG. 1, the image capturing device 10 includes an imaging unit 11, an image signal pre-processing unit 12, and a pre-processed image output unit 13.

[0022] The image capturing device 10 can be realized by a circuit device having a CPU, ROM, RAM, EEPROM, etc. The ROM stores an image processing program, etc. that can be read by the CPU. The image processing program causes the CPU (computer) to function as the image signal pre-processing unit 12. The functions of the image capturing device 10 are realized by the CPU executing the image processing program, etc. stored in the ROM.

[0023] The imaging unit 11 is configured, for example, by an imaging element, a lens, an aperture, a shutter, a photographing operation input device, and the like.

[0024] In the first embodiment, the imaging element is provided with a Bayer array color filter as an example of its function, and the output imaging data is also Bayer array data. As a modified example, the imaging element (image sensor) may be provided with a color filter other than the Bayer array, and the format of the output data is not particularly limited. The photography operation input device controls the components of the imaging unit 11, such as the imaging element, lens, aperture, and shutter, by manual operation input or commands from a program, etc. Other commonly available apertures and shutters may be used.

[0025] The image signal pre-processing unit 12 performs simple calculations on the captured image data output from the image sensor of the imaging unit 11 to calculate a pre-processed image. The processing performed by the image signal pre-processing unit 12 may use commonly known means, except for demosaic processing (Bayer to RGB conversion processing). The image data format to which the data is converted is a commonly known uncompressed image format (for example, monochrome bitmap, RAW, etc.).

[0026] The pre-processed image output unit 13 outputs the pre-processed image output from the image signal pre-processing unit 12 to the outside (the output destination is the image processing device 20). For example, the pre-processed image output unit 13 may be a communication unit that forms a packet including data of the pre-processed image according to a communication protocol of a network (not shown) and transmits the packet to the image processing device 20.

[0027] <Image processing device> As shown in FIG. 1, the image processing device 20 includes a pre-processed image input unit 21, an image signal main processing unit 22, and an image output unit .

[0028] The image processing device 20 can be realized by a circuit device having a CPU, ROM, RAM, EEPROM, etc. The ROM stores an image signal processing program that can be read by the CPU. The image processing program causes the CPU (computer) to function as the image signal main processing unit 22. The CPU executes the image processing program, etc. stored in the ROM, thereby realizing the functions of the image capturing device 10.

[0029] The pre-processed image input unit 21 inputs the pre-processed image output from the image capturing device 10 to the image processing device 20. For example, the pre-processed image input unit 21 may be a communication unit that receives a packet including data of the pre-processed image according to a communication protocol of a network (not shown) and inputs the packet to the image processing device 20.

[0030] The image signal main processing unit 22 performs calculations on the input pre-processed image to calculate an output image. The processing performed by the image signal main processing unit 22 may use generally known means, but it is assumed that this processing includes demosaic processing.

[0031] The image output unit 23 outputs the output image output from the image signal main processing unit 22 to the outside of the image processing device 20. The image output unit 23 may output the image data to an SD card or the like, or to a display device such as a liquid crystal display, for example.

[0032] (A-2) Operation of the First Embodiment Next, the operation of the image capturing system 1 according to the first embodiment having the above-described configuration will be described.

[0033] (A-2-1) Overall movement FIG. 2 is a flowchart showing the overall operation of the image capturing system (image capturing device and image processing device) according to the first embodiment.

[0034] <Step S11> In the image capturing device 10, the image capturing unit 11 receives operational inputs and capture execution commands from a person or a program, captures an image of a subject, and outputs raw image data. The raw image data is raw data output by an image sensor, unlike commonly used file formats such as bitmap or JPEG. In the first embodiment, the pixel array of the raw image data is a Bayer array.

[0035] <Step S12> The image signal pre-processing unit 12 performs image processing on the raw image data input from the imaging unit 11 to generate a pre-processed image. The image processing performed in step S12 will be described in detail later.

[0036] <Step S13> The pre-processed image output unit 13 outputs the pre-processed image to the outside of the image capturing device 10. The output destination is the image processing device 20.

[0037] <Step S14> The pre-processed image input unit 21 stores the pre-processed image input from the image capturing device 10 inside the image processing device 20 (storage means such as a memory, not shown).

[0038] <Step S15> The image signal main processing unit 22 performs image processing on the pre-processed image to generate an output image. The image processing performed in step S15 will be described in detail later.

[0039] <Step S16> The image output unit 23 outputs the output image to the outside of the image processing device 20. The output destination may be any device, such as a storage device such as a HDD, or a display device such as a liquid crystal display.

[0040] (A-2-2) Details of the processing in steps S12 and S14 FIG. 3 is a flowchart showing the operation (calculation of an RGB image) of the image signal pre-processing unit and the pre-processed image input unit in the image capturing system according to the first embodiment.

[0041] The processes shown in steps S102, S103, S104, and S105 below are examples of processes that are generally performed when converting a Bayer image to an RGB image. Processes other than those shown here may be incorporated as needed, or the processes shown here may be removed or their order may be changed, but the Bayer → RGB conversion process in step S104 is always performed.

[0042] Furthermore, the processes shown in steps S102, S103, and S105 may be executed by either the image signal pre-processing unit 12 or the image signal main processing unit 22, but the process shown in step S104 is executed only by the image signal main processing unit 22. That is, step S104 is executed after data (Bayer image) is transmitted from the image capturing device 10 to the image processing device 20.

[0043] <Step S101> The image signal pre-processing unit 12 or the image signal main processing unit 22 acquires Bayer image data to be processed (data input from the imaging unit 11 or data input from the pre-processed image input unit 21).

[0044] <Step S102> The image signal pre-processing unit 12 or the image signal main processing unit 22 performs white balance processing on the Bayer image data. Bayer image data generally includes data for each color channel of RGB, which are the three primary colors of light. In the case of raw data output from the imaging unit 11 (image sensor), the sensitivity of each color channel often differs. White balance processing is performed to match the sensitivity of each color. Any commonly used method may be used for white balance processing.

[0045] <Step S103> The image signal pre-processing unit 12 or the image signal main processing unit 22 performs defective pixel correction processing. An image sensor is composed of many pixels, and some pixels may output large or small values ​​regardless of the amount of light they receive. Pixels in this state are called defective pixels. If defective pixel correction is not performed, pixels that are significantly brighter (white dots) or darker (black dots) than the surrounding pixels will appear in the image, leading to degradation of image quality. To prevent this degradation of image quality, defective pixel correction processing is performed. Any commonly used method may be used for defective pixel correction processing.

[0046] <Step S104> The image signal main processing unit 22 converts the Bayer data into RGB data using any commonly known method.

[0047] <Step S105> The image signal pre-processing unit 12 or the image signal main processing unit 22 performs color matrix correction processing. The response spectrum of each RGB color channel of the image sensor does not necessarily match the RGB response spectrum of the human eye. This processing is performed to mix the three RGB colors to create a color that is closer to what humans perceive. The specific processing content performed in this processing may be any commonly known method.

[0048] <Step S106> The image signal main processing unit 22 outputs RGB image data.

[0049] <Supplementary explanation of step S104> The reason why step S104 is executed by the image signal main processing unit 22, that is, the image processing device 20, will be explained below.

[0050] FIG. 4 is an explanatory diagram showing an example of a Bayer image and an RGB image according to the first embodiment.

[0051] 4, in the case of a Bayer image (FIG. 4(a)), the raw image data output from the imaging unit 11 has a format different from that of normal RGB image data, and contains data of only one color channel of RGB per pixel. The process shown in step S104 is a process of expanding one-channel data to three-channel data, and therefore, in principle, is a process that increases the amount of data.

[0052] When the processing shown in step S104 is executed by the image signal pre-processing unit 12, i.e., the image capturing device 10, the content of the data transmitted from the image capturing device 10 to the image processing device 20 is an RGB image. On the other hand, when the processing shown in step S104 is executed by the image signal main processing unit 22, i.e., the image processing device 20, the content of the data transmitted from the image capturing device 10 to the image processing device 20 is a Bayer image.

[0053] Generally, when comparing an RGB image and a Bayer image of the same resolution, the Bayer image has a smaller amount of data. In order to reduce the amount of data during transmission, the process shown in S104 is executed by the image signal main processing unit 22.

[0054] (A-3) Effects of the First Embodiment According to the first embodiment, the following effects are achieved.

[0055] It is possible to reduce the amount of data transmitted between the image capturing device 10 and the image processing device 20. This is expected to reduce the communication cost between the image capturing device 10 and the image processing device 20. It is also expected to reduce the amount of power consumed when transmitting data from the image capturing device 10 to the image processing device 20.

[0056] Although the Bayer to RGB image conversion process (the process of step S104 described above) is a process that requires a relatively large amount of calculation, by performing this process in the image processing device 20, it is expected that the amount of power consumed by the image capturing device 10 will be reduced.

[0057] (B) Second embodiment A second embodiment of an image capturing system according to the present invention will be described in detail below with reference to the drawings.

[0058] (B-1) Configuration of the second embodiment FIG. 5 is a block diagram showing the configuration of an image capturing system according to the second embodiment.

[0059] 5, an image capturing system 1A includes an image capturing device 10A and an image processing device 20A. The second embodiment will be described below, focusing on the differences from the first embodiment.

[0060] The image capturing device 10A has a pre-processed image encoding unit 14 in addition to the configuration of the image capturing device 10 of the first embodiment, and has a compressed pre-processed image output unit 15 instead of the pre-processed image output unit 13.

[0061] The pre-processed image encoding unit 14 compresses the pre-processed image processed by the image signal pre-processing unit 12 to generate a compressed pre-processed image.

[0062] The compressed pre-processed image output unit 15 is similar to the above-described pre-processed image output unit 13, except that it outputs a compressed pre-processed image instead of a pre-processed image.

[0063] The image processing device 20A includes a pre-processed image decoding unit 25 in addition to the configuration of the image processing device 20 of the first embodiment, and a compressed pre-processed image input unit 24 instead of the pre-processed image input unit 21.

[0064] The compressed pre-processed image input unit 24 is similar to the above-described pre-processed image input unit 21 except that it inputs a compressed pre-processed image instead of a pre-processed image.

[0065] The preprocessed image decoding unit 25 decompresses the compressed preprocessed image input to the preprocessed image input unit 21 to generate a decompressed preprocessed image.

[0066] (B-2) Operation of the Second Embodiment Next, the operation of the image capturing system 1A of the second embodiment having the above-described configuration will be described.

[0067] Fig. 6 is a flowchart showing the overall operation of the image capturing system (image capturing device and image processing device) according to the second embodiment. Note that the processes of steps S11, S12, S15, and S16 in Fig. 6 are the same as those described above in Fig. 2, and therefore detailed description thereof will be omitted. The following description will focus on the processes of steps S21 to S24, which are unique to the second embodiment.

[0068] <Step S21> The pre-processed image encoding unit 14 compresses the pre-processed image to generate a compressed pre-processed image. A commonly known compression method may be used, and either lossless or lossy compression may be used. When using JPEG as the compression method, it is desirable to use an appropriate quantization table to compress the Bayer data.

[0069] <Step S22> The compressed pre-processed image output unit 15 outputs the compressed pre-processed image to the outside of the image capturing device 10A, to the image processing device 20A.

[0070] <Step S23> The compressed pre-processed image input unit 24 holds the compressed pre-processed image input from the image capturing device 10A in the image processing device 20A.

[0071] <Step S24> The preprocessed image decoding unit 25 decompresses the compressed preprocessed image to generate a decompressed preprocessed image. If a lossless compression method is used as the compression method in step S21, the decompressed preprocessed image and the preprocessed image will have the same data content. On the other hand, if a lossy compression method is used, the decompressed preprocessed image and the preprocessed image will not necessarily have the same data content.

[0072] Thereafter, in step S15, the image signal main processing unit 22 performs image processing on the decompressed pre-processed image to generate an output image, and in step S16, the image output unit 23 outputs the output image that has been subjected to image processing.

[0073] (B-3) Effects of the Second Embodiment According to the second embodiment, in addition to the effects of the first embodiment, the following effects are achieved.

[0074] Since the size of data transmitted between the image capturing device 10A and the image processing device 20A is reduced by data compression, the amount of data transmitted can be reduced compared to the first embodiment, which is expected to further reduce the communication cost between the image capturing device 10A and the image processing device 20A.

[0075] Furthermore, it is expected that the amount of power consumed when transmitting data from the image capturing device 10A to the image processing device 20A will be further reduced.

[0076] (C) Third embodiment Hereinafter, a third embodiment of an image capturing system according to the present invention will be described in detail with reference to the drawings.

[0077] (C-1) Configuration of the third embodiment FIG. 7 is a block diagram showing the configuration of an image capturing system according to the third embodiment.

[0078] 7, an image capturing system 1B includes an image capturing device 10B and an image processing device 20B. The following description of the third embodiment will focus on the differences from the first and second embodiments.

[0079] The image capturing device 10B includes a pre-processed image alignment unit 16 in addition to the configuration of the image capturing device 10A of the second embodiment.

[0080] The pre-processed image alignment unit 16 changes the pixel arrangement of the pre-processed image to generate an aligned pre-processed image.

[0081] The image processing device 20B includes a pre-processed image inverse sorting unit 26 in addition to the configuration of the image processing device 20B of the second embodiment.

[0082] The preprocessed image inverse alignment unit 26 restores the pixel array of the aligned preprocessed image to the same pixel array as the original preprocessed image, thereby generating a decompressed preprocessed image.

[0083] (C-2) Operation of the Third Embodiment Next, the operation of the image capturing system 1B of the third embodiment having the above-described configuration will be described.

[0084] Fig. 8 is a flowchart showing the overall operation of an image capturing system (image capturing device and image processing device) according to the third embodiment. Note that the processes of steps S11, S12, S15, S16, and S21 to S24 in Fig. 8 are similar to the processes described above with reference to Figs. 2 and 6, and therefore detailed description thereof will be omitted. The following description will focus on the processes of steps S31 and S32, which are unique to the third embodiment.

[0085] <Step S31> The pre-processed image sorting unit 16 changes the pixel arrangement of the pre-processed image to generate an aligned pre-processed image. The pixel arrangement is changed to an arrangement that increases the compression rate or enables high-speed compression in the image compression process in the subsequent pre-processed image encoding unit 14. An example of a pixel arrangement change method will be described later.

[0086] <Step S21> The operation of each part in step S21 in the third embodiment differs from the operation of each part in the second embodiment only in that a compressed aligned pre-processed image is generated from an aligned pre-processed image; the operation of each other in the other parts is the same as the operation of each part in the second embodiment.

[0087] <Step S22> The operation of each part in step S22 in the third embodiment differs from the operation of each part in the second embodiment only in that the data handled is a compressed, aligned, pre-processed image; the operation of each other in the other parts is the same as the operation of each part in the second embodiment.

[0088] <Step S23> The operation of each part in step S23 in the third embodiment differs from the operation of each part in the second embodiment only in that the data handled is a compressed, aligned, pre-processed image; the operation of each other in the other parts is the same as the operation of each part in the second embodiment.

[0089] <Step S24> The operation of each part in step S24 in the third embodiment differs from the operation of each part in the second embodiment only in that a decompressed aligned pre-processed image is generated from a compressed aligned pre-processed image; the operation of each other in the other parts is the same as the operation of each part in the second embodiment.

[0090] <Step S32> The preprocessed image inverse alignment unit 26 restores the pixel array of the decompressed aligned preprocessed image to the same pixel array as the original preprocessed image, thereby generating a decompressed preprocessed image.

[0091] Thereafter, in step S15, the image signal main processing unit 22 performs image processing on the decompressed pre-processed image to generate an output image, and in step S16, the image output unit 23 outputs the output image that has been subjected to image processing.

[0092] <Supplementary explanation of step S31> The change in pixel arrangement in step S31 will be described below.

[0093] The reason for changing the pixel arrangement is that the compression rate of image data in the Bayer array is lower than that of normal images. In the Bayer array, pixels of different color channels are arranged adjacently and alternately. This results in high-frequency variations in shading on the image plane, which makes it difficult to achieve a high compression rate from the perspective of general image compression. For this reason, the pixel arrangement is changed before compression.

[0094] An example of a method for changing the pixel arrangement will be described below. Note that the method described below is merely an example, and the effects of the present invention can be achieved even if a pixel arrangement change method different from the method described here is used.

[0095] FIG. 9 is a schematic diagram (part 1) showing an example of a method for changing the pixel array.

[0096] Figure 9(a) is a pre-processed image, which is image data in the Bayer array before the pixel arrangement is changed. By extracting and combining the data from Figure 9(a) so that the pixel positional relationships for each color channel are not changed, it is possible to obtain three images, each consisting of only the red channel (hereinafter referred to as the R channel), green channel (hereinafter referred to as the G channel), and blue channel (hereinafter referred to as the B channel), as shown in Figure 9(b1). Furthermore, by combining the three image data from Figure 9(b1), it is possible to obtain a single image in which the pixels are grouped together for each color channel, as shown in Figure 9(b2).

[0097] In both the states of FIG. 9(b1) and FIG. 9(b2), the pixels of different color channels are not arranged alternately adjacent to each other, so it is thought that the compression ratio during compression will be relatively high.

[0098] FIG. 10 is a schematic diagram (part 2) showing an example of a method for changing the pixel array.

[0099] The basic concept of the method in FIG. 10 is the same as that shown in FIG. 9, but in this method, the G channel pixel is r G channel pixels and G pixels included in the row where the B channel exists b The pixels are divided into two types: channel and pixel.

[0100] In contrast to Fig. 10(a), when extracting and combining pixels so that the positional relationship of the pixels for each color channel does not change, R channel pixels, B channel pixels, G r channel pixels, and G b By classifying the pixel data into four types, it is possible to obtain four images of each color channel, as shown in Figure 10(b1). Furthermore, by combining the four image data in Figure 10(b1), it is possible to obtain a single image with pixels grouped by color channel, as shown in Figure 10(b2).

[0101] In both the states of FIG. 10(b1) and FIG. 10(b2), pixels of different color channels are not arranged alternately adjacent to each other, so it is thought that the compression ratio during compression will be relatively high.

[0102] (C-3) Effects of the Third Embodiment According to the third embodiment, in addition to the effects of the second embodiment, the following effects are achieved.

[0103] Since the size of data transmitted between the image capturing device 10B and the image processing device 20B is reduced, the amount of data transmitted can be further reduced compared to the third embodiment. This can be expected to further reduce the communication cost between the image capturing device 10B and the image processing device 20B. In addition, it can be expected to further reduce the amount of power consumed when transmitting data from the image capturing device 10B to the image processing device 20B.

[0104] (D) Although various modified embodiments have been mentioned in the above-described embodiment, the following modified embodiments can also be applied.

[0105] (D-1) In the above-described embodiment, an example was described in which the image capturing device 10 (10A, 10B) and the image processing device 20 (20A, 20B) were physically separate devices, but they may also be physically integrated devices.

[0106] (D-2) In the above-described embodiment, an example was described in which the color filter type provided in the image sensor was a Bayer array, but an image sensor provided with a color filter type other than the Bayer array may also be used. [Explanation of symbols]

[0107] 1, 1A, 1B...Image capture system, 10, 10A, 10B...Image capture device, 11...Image capture unit, 12...Image signal pre-processing unit, 13...Pre-processed image output unit, 14...Pre-processed image encoding unit, 15...Compressed pre-processed image output unit, 16...Pre-processed image alignment unit, 20, 20A, 20B...Image processing device, 21...Pre-processed image input unit, 22...Image signal main processing unit, 23...Image output unit, 24...Compressed pre-processed image input unit, 25...Pre-processed image decoding unit, 26...Pre-processed image reverse alignment unit.

Claims

1. An image capturing system including an image capturing device and an image processing device, the image capturing device captures an image of a target object to acquire raw image data, applies image processing to the raw image data to create a pre-processed image, and transmits the pre-processed image to the image processing device; The image processing device receives the pre-processed image and applies image processing, including at least demosaicing, to the pre-processed image to create an output image. An image capturing system characterized by:

2. The image capturing device is an imaging unit that captures an image of the object to be photographed and generates the raw photographed data; an image signal pre-processing unit that applies image processing to the captured raw data to generate the pre-processed image; a pre-processed image output unit that transmits the pre-processed image to the image processing device; 2. The image capturing system according to claim 1, further comprising:

3. The image processing device includes: a pre-processed image input for receiving the pre-processed image; a main image signal processing unit that applies image processing to the pre-processed image to generate an output image; 2. The image capturing system according to claim 1, further comprising:

4. 4. The image capturing system according to claim 3, wherein the image signal main processing unit performs the demosaic processing on the pre-processed image.

5. 5. The image capturing system according to claim 1, wherein the pre-processed image is uncompressed image data.

6. An image capturing system including an image capturing device and an image processing device, the image capturing device captures an image of a target object to acquire raw image data, applies image processing to the raw image data to create a pre-processed image, compresses the pre-processed image to create a compressed pre-processed image, and transmits the compressed pre-processed image; The image processing device receives the compressed preprocessed image, decompresses the compressed preprocessed image to create a decompressed preprocessed image, and applies image processing, including at least demosaicing, to the decompressed preprocessed image to create an output image. An image capturing system characterized by:

7. The image capturing device is an imaging unit that captures an image of the object to be photographed and generates the raw photographed data; an image signal pre-processing unit that applies image processing to the captured raw data to generate the pre-processed image; a pre-processed image encoding unit that applies a data compression process to the pre-processed image to generate the compressed pre-processed image; a compressed pre-processed image output unit that transmits the compressed pre-processed image to the image processing device; 7. The image capturing system according to claim 6, further comprising:

8. The image processing device includes: a compressed pre-processed image input for receiving the compressed pre-processed image; a pre-processed image decoder that applies a data decompression process to the compressed pre-processed image to generate a decompressed pre-processed image; an image signal main processing unit that applies image processing to the decompressed pre-processed image to generate an output image; 7. The image capturing system according to claim 6, further comprising:

9. 9. The imaging system according to claim 8, wherein the image signal main processing unit performs the demosaic processing on the decompressed pre-processed image.

10. 10. The image capturing system according to claim 6, wherein the compressed pre-processed image is image data obtained by performing a compression process on the pre-processed image.

11. An image capturing system including an image capturing device and an image processing device, the image capturing device captures an image of a target object to obtain raw image data, applies image processing to the raw image data to create a pre-processed image, changes the arrangement of image data of the pre-processed image to create an aligned pre-processed image, compresses the aligned pre-processed image to create a compressed aligned pre-processed image, and transmits the compressed aligned pre-processed image; The image processing device receives the compressed, aligned preprocessed image, decompresses the compressed, aligned preprocessed image to create a decompressed, aligned preprocessed image, changes the arrangement of image data of the decompressed, aligned preprocessed image to an arrangement equivalent to that of the preprocessed image to create a decompressed preprocessed image, and applies image processing, including at least demosaicing, to the decompressed preprocessed image to create an output image. An image capturing system characterized by:

12. The image capturing device is an imaging unit that captures an image of the object to be photographed and generates the raw photographed data; an image signal pre-processing unit that applies image processing to the captured raw data to generate the pre-processed image; a pre-processed image alignment unit that changes a pixel array of the pre-processed image to generate the aligned pre-processed image; a pre-processed image encoding unit that applies a data compression process to the aligned pre-processed image to generate the compressed aligned pre-processed image; a compressed pre-processed image output unit for transmitting the compressed, aligned pre-processed image to the image processing device; 12. The image capturing system according to claim 11, further comprising:

13. The image processing device includes: a compressed preprocessed image input for receiving the compressed, aligned preprocessed image; a pre-processed image decoder that applies a data decompression process to the compressed aligned pre-processed image to generate the decompressed aligned pre-processed image; a pre-processed image de-aligner that rearranges pixels of the decompressed aligned pre-processed image to generate the decompressed aligned pre-processed image; an image signal main processing unit that applies image processing to the decompressed pre-processed image to generate an output image; 13. The imaging system according to claim 12, further comprising:

14. 14. The imaging system according to claim 13, wherein the image signal main processing unit performs the demosaic processing on the decompressed pre-processed image.

15. The image capturing system according to claim 14 , wherein the compressed, aligned, pre-processed image is image data obtained by performing a compression process on the aligned pre-processed image.

16. The image capturing system according to claim 15, wherein the aligned pre-processed image is image data in which the pixel arrangement of the pre-processed image is changed so as to increase the compression rate in the compression process in the pre-processed image encoding unit.

17. The image capturing system according to claim 15, wherein the aligned pre-processed image is image data in which the pixel arrangement of the pre-processed image is changed so that the compression process in the pre-processed image encoding unit is completed quickly.

18. The image capturing system of claim 15, wherein the image data alignment process in the pre-processing image alignment unit is a process of extracting and combining pixels for each color channel so that the positional relationship between pixels does not change, and generating an image for each color channel.

19. 19. The image capturing system according to claim 18, wherein the image data alignment process in the pre-processing image alignment unit is a process of combining images for each color channel to generate a single image.

20. 20. The image capturing system according to claim 19, wherein the color channels used in the image data alignment process in the pre-processing image alignment unit are three channels: a red channel, a green channel, and a blue channel.

21. 20. The image capturing system of claim 19, wherein the color channels in the image data alignment process in the pre-processing image alignment unit are four channels in the Bayer array data: a red channel, a blue channel, a green channel in a row in which a pixel of the red channel exists, and a green channel in a row in which a pixel of the blue channel exists.

Images