Image generating device

Abstract

Provided is an image generation technology capable of making it possible to read written ink characters that have disappeared due to weathering of ancient wooden tablets, and to restore, to some extent, the faded green color of verdigris pigments in ancient paintings so as to be visible.　This image generating device receives visible light from a subject to generate R, G, B images 30, 31, 32, and receives infrared light from the subject to generate an infrared light image 34. Then, a first color image 33 having the R, G and B images as component images is generated. In addition, a second color image 36 equivalent to one obtained by replacing the pixel values of the R component image 31 of the first color image with the pixel values of the infrared light image 34 is generated. The first color image 33 is equivalent to seeing the subject with the naked eye, but in the second color image 36, the weathered ink writing is clearer, and the faded green color of the verdigris pigment is restored to some extent. An ortho-image scanner employing a telecentric objective optical system in which a visible light image and an infrared light image are obtained with the same magnification is used to image the subject.

Description

Image Generation Device

[0001] The present invention relates to an image generating device, such as an image scanner, that generates an image of a subject using visible light and non-visible light (for example, infrared light or ultraviolet light) from the subject.

[0002] Image processing techniques using color images based on visible light and infrared light images are disclosed in the following Patent Documents 1 to 4.

[0003] Japanese Patent No. 4949806 Japanese Patent No. 5171004 Japanese Patent Laid-Open No. 2022-113614 Japanese Patent No. 4030002

[0004] For example, in fields such as archaeology, history, and art, color images based on visible light from the subject are commonly used when observing ancient wooden tablets, paintings, and the like unearthed during excavations. However, due to weathering and deterioration of the surfaces of the wooden tablets and paintings, the ink characters written on the wooden tablets are often unreadable, and the colors of the paintings have become dull. For example, in the case of paintings, the green pigments used in verdigris often deteriorate and turn the paintings into a dull yellow-gray color. Therefore, there is a demand for making the characters more readable or for restoring the dull green parts to a more vivid green. The technologies disclosed in the above-mentioned Patent Documents 1 to 4 do not address such demands.

[0005] It is therefore an object of the present invention to provide an image generation technique that is useful for observing weathered or deteriorated artworks in archaeology, history, engineering, and the like, for example, by making ink inscriptions on ancient artifacts legible or by restoring the color of deteriorated patina pigments to some degree.

[0006] Other objects may become apparent in the following disclosure.

[0007] An image generating device according to one embodiment includes an imaging device that receives visible light from a subject to generate an R, G, B image, and receives invisible light from the subject to generate an invisible light image; and a control device that receives the R, G, B image and the invisible light image from the imaging device, generates a first color image having the R, G, B images as component images, and generates and outputs a second color image equivalent to the first color image in which pixel values ​​of one of the R, G, B component images of the first color image are replaced with pixel values ​​of the invisible light image.

[0008] An image generating device according to one embodiment generates an infrared light image using infrared light as invisible light, and generates a second color image equivalent to the first color image in which the pixel values ​​of the R component image are replaced with the pixel values ​​of the infrared light image.

[0009] In one embodiment of the image generating device, the imaging device has a telecentric optical system that forms an orthoimage of a subject image with a constant magnification even when the working distance between the imaging device and the subject changes, and an image sensor unit that receives the subject image formed by the telecentric optical system and outputs pixel values ​​of each of an R image, a G image, a B image, and a non-visible light image.

[0010] In one embodiment, the image generating device sets the working distance to a first distance for visible light so that a visible light image of the subject is formed on the light receiving surface of the image sensor unit, then drives the imaging device to generate the R, G, and B images, and also sets the working distance to a second distance for invisible light so that a non-visible light image of the subject is formed on the light receiving surface of the image sensor unit, then drives the imaging device to generate a non-visible light image.

[0011] In an image generating device according to one embodiment, the imaging device has spectral sensitivity characteristics in which the sensitivity curve for visible light and the sensitivity curve for invisible light do not substantially overlap.

[0012] An image generating device according to one embodiment generates an ultraviolet light image using ultraviolet light as invisible light, and generates a second color image equivalent to the first color image in which the pixel values ​​of the R component image are replaced with the pixel values ​​of the ultraviolet light image.

[0013] It is a diagram showing the overall configuration of an image generating device according to one embodiment of the present invention. It is a diagram showing the configuration of the optical system of the imaging device. It is a diagram showing the spectral sensitivity characteristics of the imaging device. It is a flowchart showing the control flow of the control device. It is a diagram explaining image processing by the control device. It is a diagram showing a comparison arrangement of a faithful color image, an IR image, and a modified color image of a wooden tablet.

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the embodiments described below do not limit the scope of the invention as claimed, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solution of the invention.

[0015] FIG. 1 is a diagram showing the overall configuration of an image generating device 1 according to an embodiment of the present invention.

[0016] As will be understood from the following description, the image generating device 1 according to this embodiment is an orthoimage scanner using a telecentric objective optical system. As shown in FIG. 1 , a table 4 is placed on a floor 2 for placing a subject 3 thereon. In this embodiment, the subject 3 is, for example, an ancient wooden tablet or painting unearthed during an excavation. In the following description, for the convenience of explaining three-dimensional positional relationships, an orthogonal three-dimensional coordinate system having X, Y, and Z axes as shown in FIG. 1 is defined. Here, the Z axis is the vertical direction (height direction), and the X and Y axes are horizontal directions.

[0017] A columnar X track 5 is laid above the table 4 in parallel to the X axis. A columnar Y track 6 is attached on top of this X track 5 in parallel to the Y axis. A columnar Z track 7 is attached on top of this Y track 6 in parallel to the Z axis. A columnar carriage 8 is attached to this Z track 7 in parallel to the Z axis.

[0018] An electric self-propelled device (not shown) is attached to the X track 5, and a Y track 6 is attached to this electric self-propelled device, which can move back and forth in a direction parallel to the X axis (lateral direction) within the length of the X track 5. An electric self-propelled device (not shown) is attached to the Y track 6, and a Z track 7 is attached to this electric self-propelled device, which can move back and forth in a direction parallel to the Y axis (depth direction) within the length of the Y track 6 by an electric automatic device. A carriage 8 having an electric self-propelled device (not shown) is attached to the Z track 7, and the carriage 8 can move back and forth in a direction parallel to the Z axis (height direction) within the length of the Z track 7. As a result of the X track 5, Y track 6, Z track 7, and carriage 8 being connected in series in this manner, the carriage 8 can move three-dimensionally along the X, Y, and Z axes.

[0019] An imaging device 9 is fixed on the carriage 8. The carriage 8 moves the imaging device 9 in three dimensions relative to the table 4. The imaging device 9 has an image sensor unit 10 and a telecentric imaging unit 11. Furthermore, at least two separate light source units 12 and 13 are fixed to the carriage 8 (or the imaging device 9) (fixing structures are not shown). These two light source units 12 and 13 are, for example, formed of elongated LED lamps extending along the Y axis, have a columnar or linear shape, and are arranged on either side of the optical axis 11A of the telecentric imaging unit 11 so as to minimize the occurrence of shadows at the imaging location of the subject 3. The light source units 12 and 13 irradiate the subject 3 with visible light and invisible light (infrared light, particularly near-infrared light, in this embodiment).

[0020] The image sensor unit 10 includes a lens barrel and a color separation imaging system housed within the lens barrel. This color separation imaging system includes a color separation prism that separates input light into red (R), green (G), blue (B), and infrared (IR) color (wavelength) components, and, for example, four linear image sensors. Three of the linear image sensors can capture visible light images (RGB color images), and the other linear image sensor can capture infrared light images. Note that a conventionally known two-dimensional image sensor (area sensor) can be used instead of the linear image sensors.

[0021] Telecentric imaging unit 11 has a lens barrel and a telecentric optical system housed within the lens barrel, and is disposed so that its optical axis 11A is parallel to the Z axis and faces in the positive direction of the Z axis (the shooting direction, i.e., the direction toward subject 3). Telecentric imaging unit 11 forms an image of a straight (i.e., linear) area of ​​a predetermined length parallel to the Y axis, located at a position a predetermined working distance from its front end in the shooting direction along optical axis 11A, on a linear image sensor within image sensor unit 10.

[0022] 1, the image generating device 1 further includes a control device 14 incorporating a computer, a power supply circuit, and other electric and electronic circuits. The control device 14 is electrically connected to the electrically driven self-propelled devices of the X track 5, the Y track 6, the Z track 7, and the carriage 8, the linear image sensor in the image sensor unit 10, and the light source units 12 and 13 via signal cables and power cables (not shown). The control device 14 has the function of controlling the three-dimensional movement and position of the imaging device 9 by driving and controlling the electrically driven self-propelled devices of the X track 5, the Y track 6, the Z track 7, and the carriage 8. The control device 14 also has the function of inputting image data sequentially output from the linear image sensor in the image sensor unit 10 while the subject 3 is being scanned by the image sensor unit 10, and processing the image data.

[0023] 1 as a simple box, it may actually be composed of a device drive circuit unit that drives the above-mentioned electric self-propelled device, image sensor unit 10, and light source units 12 and 13, etc., and receives signals from these devices, and a computer terminal that is connected to the device drive circuit unit via wired or wireless communication and that has a computer program that performs the above-mentioned control and the image processing described below. This computer terminal may be, for example, a general-purpose personal computer, and may include input means such as a keyboard and mouse that are operated to perform the image processing described below, output means such as a display for displaying data, and an external output terminal for transmitting and receiving control data and image data to and from the above-mentioned device drive circuit unit.

[0024] FIG. 2 shows the configuration of the optical system of the imaging device 9.

[0025] As shown in FIG. 2 , the telecentric optical system 11B in the telecentric imaging unit 11 is composed of an objective lens 20, an aperture 21, and an imaging lens 22, and forms an orthogonal projection image of the subject 3 on the light-receiving surfaces of four linear image sensors 25R, 25G, 25B, and 25IR for R, G, B, and IR in the image sensor unit 10. Note that a conventionally known, commercially available telecentric optical system (an object-side telecentric lens system or a double-telecentric lens system) can be used as the telecentric optical system. As shown in FIG. 2 , the position of the subject 3 at which the telecentric optical system 11B focuses, i.e., the working distance (D1) which is the distance from the subject 3 to the objective lens 20, differs between the case where the R, G, and B images of the subject 3 are formed on the light-receiving surfaces of the R, G, and B linear image sensors 25R, 25G, and 25IR and the case where the infrared image of the subject 3 is formed on the light-receiving surface of the IR linear image sensor 25IR, as shown in FIG. Therefore, the image generating device 1 is configured to capture RGB color images and infrared images at different times, and automatically controls the working distance to the distance D1 or D2 at which the image is in focus at each time.

[0026] As shown in FIG. 2 , the color separation imaging system 10B within the image sensor unit 10 is disposed on the exit side of the telecentric imaging unit 11 and includes a color separation prism 23 that separates the incident light beam from the telecentric imaging unit 11 into red light (R light), green light (G light), blue light (B light), and infrared light (IR light). At the exit of each color light from the color separation prism 23, a trimming filter that selectively passes the light of each color is disposed. Also, image sensors 25R, 25G, 25B, and 25IR are provided that receive the images of each color that have passed through the trimming filters and convert them into image data representing a large number of pixel values. Note that FIG. 2 representatively depicts the path of the G light beam that passes through the color separation prism 23 to form a G image on the G image sensor 25G. Of the paths of the other colors of light, only the path of the chief ray emitted from the center position of the subject 3 is shown.

[0027] FIG. 3 is a diagram showing the spectral sensitivity characteristics of the imaging device 9. As shown in FIG.

[0028] As shown in FIG. 3 , the imaging device 9 has spectral sensitivity characteristics in which the sensitivity curves for visible light (R, G, B) and infrared light (IR) do not overlap with each other across a wavelength band of just over 700 nm. Also, as shown in FIG. 3 , there is little overlap between the sensitivity curves for the R, G, and B color components, and in particular, the sensitivity curve for the R component does not substantially overlap with the sensitivity curves for the other color components (G and B). Therefore, the RGB color image generated by the imaging device 9 is a substantially pure visible light image that is substantially free of infrared components, and the infrared light image is a substantially pure infrared light image that is substantially free of visible light components. Furthermore, the R image within the visible light image is a substantially pure R image that is almost free of other color components. Such spectral sensitivity characteristics offer the advantage that, when image processing is performed in the control described below, in which pixel values ​​of the infrared light image are replaced with pixel values ​​of the R image, the image processing does not substantially affect the G image and B image (i.e., does not excessively destroy the original visible light color image).

[0029] Fig. 4 is a flowchart showing the flow of control by the control device 14. Fig. 5 is a diagram for explaining image processing by the control device 14.

[0030] First, the control device 14 sets the working distance to the first distance D1 for visible light so that a visible light image of the subject 3 is formed on the light receiving surfaces of the R, G, and B image sensors 25R, 25G, and 25B of the image sensor unit 10 (step 1).

[0031] Next, while scanning the subject 3 with the imaging device 9, the control device 14 receives a large number of R, G, and B pixel values ​​that make up the R, G, and B image from the R, G, and B image sensors 25R, 25G, and 25B of the imaging device 9 (step 2).

[0032] Next, the control device 14 integrates the R, G, and B pixel values ​​for each color to create an R image 30, a G image 31, and a B image 32 of the subject 3, as shown in FIG. 5 (step 3).

[0033] Next, the control device 14 integrates the R image 30, the G image 31, and the B image 32 as component images to create a color image 33 of the subject 3 (step 4), as shown in Figure 5. In this specification, this color image 33 is referred to as a "faithful color image," meaning that it is an image that faithfully reproduces the colors seen by the naked eye.

[0034] Next, the control device 14 sets the working distance to the second distance D2 for infrared light so that an infrared light image of the subject 3 is formed on the light receiving surface of the IR image sensor 25IR of the image sensor unit 10 (step 5).

[0035] Next, the control device 14 receives a large number of IR pixel values ​​that constitute an IR image from the IR image sensor 25IR of the imaging device 9 while scanning the subject 3 with the imaging device 9 (step 6).

[0036] Controller 14 then integrates the IR pixel values ​​to create an IR (infrared) image 34 of subject 3, as shown in FIG. 5 (step 7).

[0037] Next, the control device 14 copies the IR image 34 to create a copy image 35 having the same pixel values ​​as the IR image 34 (step 8), as shown in Figure 5. This copy image 35 will be used as a pseudo or substitute R image in the next step 9, and so will be referred to as a "modified R image" in this specification to distinguish it from the original R image 30.

[0038] Next, the control device 14 integrates the transformed R image 35, G image 31, and B image 32 as component images to create a transformed color image 36 of the subject 3 (step 9), as shown in FIG. 5 . This transformed color image 36 is equivalent to the faithful color image 33 in which the pixel values ​​of the R image 30 are replaced with the pixel values ​​of the IR image 34. Therefore, the R component (transformed R image 35) of the transformed color image 36 exhibits the shading characteristics of the IR image 34. Note that the process for generating the transformed color image 36 described above is merely an example, and the transformed color image 36 may be generated by other processes. For example, the transformed color image 36 may be generated by creating a copy of the faithful color image 33 and replacing the pixel values ​​of the R image 30 in the copy with the pixel values ​​of the IR image 34.

[0039] Next, the control device 14 outputs or displays on a screen the true color image 33, the modified color image 36, and the IR image 34 of the subject 3 (step 10).

[0040] In this embodiment, even if the working distance changes between visible light and infrared light imaging, by using a telecentric optical system as the imaging optical system, orthogonal projection images (orthoimages) of the same size and magnification can be obtained for both the visible light (R, G, B) image and the IR image. Therefore, the pixel positions of the visible light image and the corresponding pixel positions of the IR image accurately match. Therefore, the pixel values ​​of the transformed R image 35 (IR image 34) can be easily replaced with the pixel values ​​of the R image 30. In other words, the difference in image magnification between the visible light image and the infrared light image, which is common in conventional technology, does not occur. Therefore, the transformed R image 35 can be easily replaced with the R image 30 without any special work, processing, calculation, or the like to adjust for such a difference.

[0041] FIG. 6 shows a comparison of the faithful color image, IR image, and modified color image generated as a result of a test in which a faithful color image, an IR image, and a modified color image of a wooden tablet were generated using the image generation device 1 of this embodiment.

[0042] The wooden tablet used in this test had a character string written in ink on a wooden board, but the ink on the surface of the tablet had worn away or disappeared due to deterioration and wear. As a result, even when looking at a faithful color image of the tablet, the ink writing is almost unreadable. On the other hand, infrared light penetrates deeper into the tablet than visible light and is absorbed by the ink remaining inside the tablet. Therefore, in the IR image of the tablet, the ink writing appears darker (with lower pixel values) than the surrounding background, making the ink writing easier to read. However, because the IR image is a monotone image, it is not possible to read the ink writing while also observing the color of the tablet.

[0043] Of the three color component images of the faithful color image, the R component image has the brightness (pixel value) pattern closest to that of the infrared light image. The modified color image is created by replacing the R component image of this faithful color image with the IR image (using the pixel values ​​of the IR image as the pixel values ​​of the R component image). In this modified color image, the brightness pattern of the background area other than the ink-written area is close to that of the original faithful color image. In particular, as shown in Figure 3, the sensitivity curve of the imaging device 9 to IR light does not overlap with the sensitivity curves to R, G, and B visible light, and the sensitivity curve to R light does not substantially overlap with the sensitivity curves to G and B light. Therefore, the above substitution has virtually no effect on the G and B images of the modified color image. Therefore, the brightness pattern of the background area is quite close to that of the original faithful color image. On the other hand, because IR light is well absorbed in the ink-written area, the modified color image appears darker in the R component than in the original faithful color image, resulting in a color tone in which the G component is relatively more prominent. The human eye is most sensitive to green in the visible light spectrum. Therefore, when a person looks at the altered color image of the wooden tablet, the brightness pattern of the background part is not significantly different from the original faithful color image, but the green color in the ink writing part is more emphasized and appears clearer, making the ink writing easier to read.

[0044] Although not specifically shown in the drawings, generating a modified color image using the image forming apparatus 1 according to this embodiment offers the following advantages when studying ancient paintings. For example, ancient paintings painted in green or other colors on paper typically used verdigris (basic copper carbonate) as a green pigment. However, over time, verdigris pigments, especially on surfaces exposed to the elements, undergo chemical changes, degrading the green color component and introducing red components, resulting in a yellowish gray color, a mixture of green and red. The human eye is most sensitive to green in the visible light spectrum. Therefore, when viewing a faithful color image of a painting, the green color, to which humans are most sensitive, is weakened or lost, resulting in the painting appearing much duller in color than the original. Meanwhile, infrared light penetrates deeper into the pigments of a painting than visible light and is absorbed by the verdigris within, causing the original green areas to appear darker (lower pixel values) in the infrared light image of the painting. As mentioned above, of the three color component images of the faithful color image, the R component image has the brightness (pixel value) pattern closest to that of the infrared light image. Therefore, in a modified color image in which the R component image of the faithful color image is replaced with an infrared light image (using the pixel values ​​of the infrared light image as the pixel values ​​of the R component image), the R component becomes darker in areas where there is discolored patina pigment, making the G component relatively more noticeable. Meanwhile, in areas where there is no patina pigment, a brightness pattern similar to that of the original R component image is maintained, resulting in a color tone that is closer to that of the original painting.

[0045] The present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations. Furthermore, the dimensions, shapes, etc. of each component shown in the drawings are not necessarily accurately depicted and may be appropriately modified to emphasize the features of this embodiment.

[0046] For example, in the above-described embodiment, an example has been shown in which the R component image of a faithful color image is replaced with an infrared light image, but this is not limited to this. As a modified example, an ultraviolet light image may be captured using ultraviolet light, and this ultraviolet light image may be replaced with a B component image whose wavelength is closest to that of ultraviolet light. Furthermore, after capturing an infrared light image, an ultraviolet light image, and a faithful color image, the ultraviolet light image may be replaced with the B component image, and the infrared light image may be replaced with the R component image. Furthermore, depending on the purpose and application, the ultraviolet light image or the infrared light image may be replaced with the G component image.

[0047] Furthermore, for example, in the above-described embodiment, an example was described in which an image of a subject is generated using reflected light from the subject, but this is not limited to this. Instead of reflected light, a faithful color image, an infrared image, or an ultraviolet image may be generated using transmitted light from the subject (light that is irradiated from the back side of the subject and transmitted through the subject).

[0048] 1: Image generating device 9: Imaging device 10: Image sensor unit 11: Telecentric imaging unit 12, 13: Light source unit 14: Control device 30: R image 31: G image 32: B image 33: True color image 34: IR image 35: Modified R image 36: Modified color image

Claims

1. An image generating device comprising: an imaging device that generates R, G, B images by receiving visible light from a subject, and generates an invisible light image by receiving invisible light from the subject; and a control device that receives the R, G, B images and the invisible light image from the imaging device, generates a first color image having the R, G, B images as component images, and generates and outputs a second color image equivalent to the first color image in which the pixel values ​​of one of the R, G, B color component images of the first color image are replaced with the pixel values ​​of the invisible light image.

2. An image generating device according to claim 1, wherein the non-visible light is infrared light, the non-visible light image is an infrared light image, and the one color component image to be replaced is the R component image.

3. An image generating device according to claim 1, wherein the imaging device comprises a telecentric optical system that forms an orthoimage of a subject image with a constant magnification even if the working distance between the imaging device and the subject changes, and an image sensor unit that receives the subject image formed by the telecentric optical system and outputs pixel values ​​of each of the R image, G image, B image and non-visible light image.

4. An image generating device according to claim 3, wherein the working distance is set to a first distance for visible light so that a visible light image of the subject is formed on the light receiving surface of the image sensor unit, and the imaging device is driven to generate the R, G, B images; and the working distance is set to a second distance for invisible light so that a non-visible light image of the subject is formed on the light receiving surface of the image sensor unit, and the imaging device is driven to generate the non-visible light images.

5. An image generating device according to claim 1, wherein the imaging device has spectral sensitivity characteristics in which the sensitivity curve for the visible light and the sensitivity curve for the non-visible light do not overlap.

6. An image generating device according to claim 1, wherein the non-visible light is ultraviolet light, the non-visible light image is an ultraviolet light image, and the one color component image to be replaced is the B component image.

Images