Texture image acquisition device, texture image acquisition method, and imaging jig

Abstract

To easily acquire an image of a medium having a texture such as a fabric faithfully to the texture.SOLUTION: By using an imaging tool provided with a comparison pattern including at least a first region and a second region which surround a window portion configured as a region of a predetermined shape transmitting light and have different shades, an image including the medium of the window portion and the comparison pattern is captured in a state where the imaging tool is superimposed on the medium. Medium image data based on the image of the medium and comparison image data based on the image of the comparison pattern are acquired from the captured image, and the medium image data is corrected based on the image data of the first region and the second region included in the comparison image data. Then, the image of the texture of the medium is cut out from the corrected medium image data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to techniques for obtaining images of textures. [Background technology]

[0002] There are known techniques for reading the texture of an object that can be used as a recording medium. For example, Patent Document 1 below discloses a method for synthesizing texture image data with the brightness information of image image data, or synthesizing the chromaticity information of texture image data with the chromaticity information of image image data, thereby preserving the hue and saturation and obtaining a synthesized image of texture that does not have a large variation in the brightness of the overall image after synthesis. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 6-86045 [Patent Document 2] Japanese Patent Application Laid-Open No. 2008-000000 Summary of the Invention [Problem to be solved by the invention]

[0004] However, these technologies have issues such as the need for strict color management of the reference chart itself, and the difficulty of dealing with unevenness that occurs due to location, such as uneven lighting, when using gamma curve correction. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms or application examples.

[0006] (1) One aspect of the present disclosure is a texture image acquisition device that acquires a texture image of a medium. The texture image acquisition device includes: an imaging jig that includes a comparison pattern that surrounds a window portion configured as a region of a predetermined shape that transmits light and includes at least a first region and a second region that differ in shade; an imaging unit that captures an image including the medium in the window portion and the comparison pattern with the imaging jig superimposed on the medium; a correction unit that acquires, from the captured image, medium image data based on the image of the medium and comparison image data based on an image of the comparison pattern, and corrects the medium image data based on image data of the first region and the second region included in the comparison image data; and a cutting unit that cuts out an image of the texture of the medium from the corrected medium image data.

[0007] (2) Another aspect of the present disclosure is an imaging jig for imaging the texture of a medium. The imaging jig includes a window portion configured as a region of a predetermined shape provided in the center of the imaging jig, and a comparison pattern surrounding the window portion, the comparison pattern including at least a first region and a second region of different shading. Here, the first region circumscribes the window portion and surrounds it, the second region surrounds the first region outside the first region, and the respective peripheral shapes of the first region and the second region are similar shapes that share a center point with the window portion.

[0008] (3) The present disclosure can be implemented in various forms, such as a method for acquiring a texture image corresponding to a texture image acquisition device using this imaging jig, or a computer-executable program for acquiring a texture image of a medium. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram showing a schematic configuration of a texture image acquisition device according to an embodiment. [Figure 2] A cross-sectional view taken along the XZ plane at position II-II in Figure 1. [Figure 3]FIG. 10 is a plan view showing an example of a state in which the imaging jig is superimposed on the subject. [Figure 4] FIG. 2 is a functional block diagram of a texture image acquisition device. [Figure 5] FIG. 2 is an explanatory diagram showing an outline of a correction process performed in the texture image acquisition device. [Figure 6] FIG. 10 is an explanatory diagram showing an outline of correction processing. [Figure 7] 10 is a flowchart showing an example of a texture image acquisition processing routine. [Figure 8] FIG. 10 is an explanatory diagram showing an example of angle of view correction. [Figure 9] FIG. 10 is an explanatory diagram showing contrast correction. [Figure 10] FIG. 10 is an explanatory diagram showing an overview of brightness correction. [Figure 11] FIG. 10 is an explanatory diagram showing how a texture image is cut out. [Figure 12] 10A to 10C are explanatory diagrams illustrating modified examples 1 to 3 as other embodiments of the imaging jig. [Figure 13] 10A and 10B are explanatory diagrams illustrating modified examples 4 and 5 as other embodiments of the imaging jig. [Figure 14] FIG. 10 is an explanatory diagram illustrating another embodiment of the imaging jig. [Figure 15] FIG. 10 is an explanatory diagram illustrating a sixth modification as another embodiment of the imaging jig. DETAILED DESCRIPTION OF THE INVENTION

[0010] A. First embodiment: (A1) Overall structure: FIG. 1 is an explanatory diagram showing the schematic configuration of a texture image acquisition device 10 according to a first embodiment. As shown in the figure, the texture image acquisition device 10 includes an imaging unit 20 that captures the texture of a subject TM, an image processing device 30 that receives an image from the imaging unit 20 and acquires a texture image, a printer 40 that receives output from the image processing device 30 and prints the image on a print medium P, and an imaging jig 100 that is placed above the subject TM and assists in capturing the texture. As shown in the figure, the downward direction along the direction of gravity is the +Z direction, and the plane perpendicular to the Z direction is the XY plane. One of these planes is the +X direction, and the other direction perpendicular to this is the +Y direction. The directions opposite the +Z, +X, and +Y directions are referred to as the -Z, -X, and -Y directions, respectively. Furthermore, when the orientation is not important, the directions may simply be referred to as the Z, X, and Y directions. These X, Y, and Z directions are also shown appropriately in other figures.

[0011] The imaging jig 100 is used by placing it on the subject TM. The subject TM is placed roughly along the XY plane, and the imaging jig 100 is placed on top of it. This is shown in Figure 2. Figure 2 is a cross-sectional view taken along the XZ plane at position II-II in Figure 1. In the figure, a mounting base 110 is placed on a table TB, the subject TM is placed on top of that, and the imaging jig 100 is placed on top of that. The imaging jig 100 has a positive-facing window OP in its center, which is a polygonal area that transmits light. The imaging jig 100 needs only to be rigid enough to prevent distortion when placed on the subject TM, and various materials such as thin metal plates, synthetic resin sheets, and thick paper can be used. If the imaging jig 100 is made of a material that can be printed with a printer, a pattern may be printed on it using a printer 40 or other printing device. The shape and color of the window OP and the pattern around it will be described in detail later. The window portion OP may be a cutout in the imaging jig 100, or may be covered with thin transparent glass or synthetic resin. Alternatively, the first area AR1 may be printed with white ink on a transparent sheet, and the second area AR2 may be printed with black ink, with the window portion OP remaining as an unprinted area. In this case, the buffer area AB may also be printed with, for example, gray ink. When printing on a transparent sheet, it is preferable that the printed surface faces the imaging unit 20 so that light reflected by the transparent sheet does not directly enter the imaging unit 20.

[0012] In this embodiment, the imaging unit 20 is a mobile phone equipped with a camera 25. The imaging unit 20 is connected to the image processing device 30 via wireless communication and transmits captured image data and the like to the image processing device 30. The imaging unit 20 can also be a standalone camera instead of a mobile terminal such as a mobile phone. The connection between the imaging unit 20 and the image processing device 30 is not limited to wireless communication, and can also be a wired connection using a cable or the like. The imaging unit 20 can be held by a user to capture images, but as shown by the dashed dotted line in FIG. 2, the imaging unit 20 can also be fixed to a stand 22 connected to a mounting base 110. The stand 22 can also be in a form that can stand on its own.

[0013] (A2) Configuration of imaging jig 100: FIG. 3 is a plan view of the imaging jig 100. In the figure, the imaging jig 100 is placed on top of the subject TM. A window portion OP provided in the center of the imaging jig 100 is square-shaped, and three regions are provided around the outside of the window portion OP, concentric with the center point of the window portion OP. A first region AR1 is formed around the window portion OP. The first region AR1 has a rectangular outer periphery concentric with the window portion OP, and is arranged to circumscribe the window portion OP and surround it. In the first embodiment, the color of the first region AR1 is white. A buffer region AB is provided outside the first region AR1. The color of this buffer region AB is gray. A second region AR2 is formed outside the buffer region AB. The outer periphery of the second region AR2 is a rectangular outer periphery concentric with the window portion OP, and is formed outside the first region AR1 to surround the first region AR1. The color of the second region AR2 is black. Although an area of ​​the background color of the imaging jig 100 exists outside the second area AR2, this background area may be omitted. In the first embodiment, the area consisting of the first area AR1, the buffer area AB, and the second area AR2 constitutes the comparison pattern. The peripheral shapes of these areas are similar shapes that share a center point with the window portion OP. Note that the buffer area AB may not be included in the comparison pattern.

[0014] The first region AR1 is white, but if the imaging jig 100 is white paper, the background color may be used. The second region AR2 is black. Differences in the appearance of the subject TM due to factors such as lighting are corrected using the white of the first region AR1 and the black of the second region AR2 as references. While the specific details of this brightness correction will be described later, if the first region AR1 has L*=100 in the L*a*b* color space and the second region AR2 has L*=0 in the L*a*b* color space, the entire range of brightness L* can be easily corrected. The first region AR1 and the second region AR2 only need to have different shades—in this case, brightness L*—and the shades may be reversed. Furthermore, the values ​​of L* and L* may be other than 0 and 100. The brightness L* of each region may be measured under standard illumination. The buffer region AB is a buffer region provided between the first region AR1 and the second region AR2. This is because, when capturing an image using the imaging unit 20, if adjacent areas with significantly different brightness levels are present, functions such as edge extraction built into the imaging unit 20 may be activated, resulting in an image that does not accurately reflect the brightness of adjacent areas. When a smartphone or similar device is used as the imaging unit 20, such functions may be activated to capture a photo that looks good. The buffer area AB is located between the first area AR1 and the second area AR2 and suppresses or avoids functions such as shading correction associated with edge extraction. Therefore, in environments where such corrections do not occur, the buffer area AB may not be necessary. In the following description, to avoid cumbersome descriptions, the notation in the L*a*b* color system may be simply referred to as Lab.

[0015] (A3) Acquiring texture images: The functional configuration of the image processing device 30 is shown in FIG. 4. As shown, the image processing device 30 includes a communication unit 31 that communicates with the imaging unit 20, an image acquisition unit 33 that acquires image data from the imaging unit 20 via the communication unit 31, a correction processing unit 34 that performs various corrections on the acquired image, a cropping unit 36 ​​that crops out a texture image and outputs it to a memory 37, a pattern output unit 38 that prints a pattern of the imaging jig 100 using a printer 40, and a display 39 that displays the captured image acquired by the image acquisition unit 33, messages, and the like. The cropping unit 36 ​​not only stores the cropped texture image in the memory 37, but may also output it to the printer 40 and print the texture image on a print medium P for confirmation. The image processing device 30 includes an arithmetic and logic operation circuit 32 that includes a known CPU and storage device. The image acquisition unit 33, the correction processing unit 34, the cropping unit 36, and the like are realized by the operation of the CPU executing a predetermined program.

[0016] The communication unit 31 communicates with the imaging unit 20 via Bluetooth (registered trademark), WiFi (registered trademark), or the like. The image acquisition unit 33 acquires image data via the communication unit 31. The acquired image data includes a medium image including texture and an image of a comparison pattern. The correction processing unit 34 acquires medium image data based on an image of the subject TM and comparison image data based on an image of the comparison pattern of the imaging jig 100, and corrects the medium image data based on the image data of the first and second regions included in the comparison image data.

[0017] The cropping unit 36 ​​crops an image of the texture of the subject TM from the corrected medium image data and stores it in memory 37 or outputs it to the printer 40, which prints it on the print medium P. The memory 37 may be built into the image processing device 30 or may be an external USB memory stick. Here, a full-color inkjet printer is used as the printer 40. However, any type of printer capable of reproducing images with a predetermined gradation, such as a thermal dye-sublimation printer, may be used. A display may also be used instead of a printer. The printer 40 may print a comparison pattern for the imaging jig 100, as shown in the figure, as long as it can identify and reproduce device-independent colors, including shading, using an ICC profile 50. In this case, the pattern output unit 38 of the printer 40 outputs data for a comparison pattern in which one of the first area AR1 and the second area AR2 is white, the other is black, and the buffer area AB is gray. The printer 40 then prints the comparison pattern on the print medium P via the output ICC profile 50. Here, the colors of each region may be defined using the RGB color system. For example, white in the first region AR1 may be defined as RGB = (255, 255, 255), and black in the second region AR2 may be defined as RGB = (0, 0, 0). Examples of RGB color systems that can be used include well-known sRGB, AdobeRGB ("Adobe" is a registered trademark), and DisplayP3. Color values ​​in the L*a*b* color system or CMYK color system corresponding to these RGB values ​​may also be used. After printing, the imaging jig 100 can be easily manufactured by cutting out the window OP. The gray of the buffer region AB may be, for example, RGB = (128, 128, 128). However, since the buffer region AB in this embodiment is not intended for brightness correction, its color and brightness may be arbitrary as long as they are not identical to those of the first region AR1 and the second region AR2.

[0018] The captured image captured by the above-described imaging unit 20 contains various types of noise. In order to extract the texture of the subject TM, it is necessary to remove these noises. Hereinafter, noises that may be contained in the captured image will be described with reference to FIG. 5, but some of these noises do not necessarily require removal processing. This point will also be described. [1] Distortion of the angle of view: When the imaging unit 20 is not directly facing the subject TM, the shape of the subject TM defined by the square window OP is captured as a distorted shape, as shown in the noise image column. The texture of the subject TM itself is distorted in the same way as the outer shape. Therefore, the distortion of the angle of view is corrected to accurately reproduce the texture. The correction of the angle of view can be performed by a coordinate transformation such as an affine transformation, as will be described later. As shown in FIG. 2, if the imaging unit 20 is accurately positioned directly facing the subject TM and distortion of the angle of view is eliminated or minimized to a negligible degree, correction of the distortion of the angle of view may not be necessary.

[0019] [2] Low contrast: When the contrast of the image captured by the imaging unit 20 is lower than the actual contrast, the contrast is improved. The contrast can be improved by correcting the contrast of the acquired image, but it can also be improved by changing the exposure conditions (aperture and exposure time) of the imaging unit 20. If the imaging unit 20 is set to perform automatic exposure adjustment so that it captures images with sufficient contrast, correction on the image processing device 30 side may be omitted.

[0020] [3] Uneven lighting: Illumination of the subject TM is uneven. Illumination unevenness can be caused by a variety of factors, including the angle between the lighting light source and the subject TM, differences in the distance from the light source to various positions on the subject TM, the shape of the light source, variations in the light emission of the light source itself, and reflections by the imaging unit 20, etc. Such uneven lighting can be perceived as uneven brightness in the first area AR1 or the second area AR2, and can be corrected by performing brightness correction.

[0021] [4] Out-of-focus: As shown by the noise image, an out-of-focus image may result from a shift in focus position or an excessive exposure time during image capture by the image capture unit 20. In such cases, a sharpening process using an edge-enhancing filter, such as a differential filter like a Sobel filter, may be performed. If the out-of-focus image is caused by the image capture conditions of the image capture unit 20, it may be possible to change the image capture conditions and capture the image again using the image capture unit 20.

[0022] [5] Luminance noise: Luminance noise refers to noise that occurs in the luminance channel. Luminance noise generally occurs due to the high sensitivity of the imaging unit 20. When processing the image signal from the imaging unit 20, it is treated as noise occurring in the L component, which is the brightness component. Luminance noise is likely to occur when the ISO sensitivity of the imaging unit 20 is high. [6] High-sensitivity noise: High-sensitivity noise is noise that occurs when the sensitivity of the imaging unit 20 is too high, and is noise that causes non-existent colors to appear. Luminance noise and high-sensitivity noise can be reduced using a blurring filter. An averaging filter that takes the average value of a predetermined number of pixels can be used as the blurring filter. For luminance noise, if the Lab color system is used, it is sufficient to mainly average the lightness L. For high-sensitivity noise, if the Lab color system is used, it is sufficient to also average the chromaticity index ab. Correction processes for out-of-focus, luminance noise, high-sensitivity noise, etc. are not essential, and it is up to the user whether or not to perform them. Furthermore, the user may be allowed to decide whether or not to perform them.

[0023] An example of the correction process for the various types of noise described above is shown in FIG. 6. The captured image captured by the imaging unit 20 may contain distortion of the angle of view, inappropriate contrast, inappropriate brightness, inappropriate sharpness, etc. Therefore, the correction is performed by correcting the distortion of the angle of view, optimizing the contrast, optimizing the brightness, and optimizing the sharpness. The optimizing of the sharpness corresponds to performing either a sharpening process to increase the sharpness, or a blurring process to reduce the sharpness. This correction process is generally performed as follows: Angle of view correction → Contrast correction → Brightness correction → Sharpening or blurring This is done in the following order.

[0024] (A4) Texture image acquisition process: The texture image acquisition process performed by the texture image acquisition device 10 will be described with reference to FIG. 7. Prior to the texture image acquisition process, the above-described imaging jig 100 is prepared, and a subject TM whose texture is to be acquired is prepared. The illustrated texture image acquisition process is realized by a computer executing a program prepared in advance in the image processing device 30. The program may be recorded in a ROM or the like provided in the image processing device 30, or may be downloaded from an external site and executed as needed. In addition, a program for linking the imaging unit 20 and the like may be prepared in either the imaging unit 20 or the image processing device 30, and transmitted to the other party for execution as needed.

[0025] When the illustrated process starts, an imaging procedure setting process is first performed (step S101). The imaging procedure setting process includes setting the position and brightness of the lighting, setting the white balance, setting the position of the imaging unit 20 and the angle with respect to the subject TM, etc. It may also include input of data related to the imaging jig 100, such as the size of the window portion OP and colorimetric values ​​of the colors of the first area AR1 and the second area AR2, and setting the type of correction to be performed after imaging and filters.

[0026] Next, an imaging process is performed using a camera (step S111). Specifically, the camera 25 of the imaging unit 20 images the subject TM on which the imaging jig 100 is placed, and captures a captured image including the subject TM, which is a medium visible through the window portion OP corresponding to the window portion, and the comparison pattern on the imaging jig 100. It is then determined whether the captured image is the desired image (step S115). This determination may be made by displaying the captured image on the display 39 provided in the image processing device 30 and having the user decide whether to accept it, or by a CPU or the like provided in the image processing device 30 based on the size of the window portion OP, the brightness of the first region AR1, etc., of the captured image. If the captured image is not the desired image (step S115: "NO"), the process returns to step S101 and repeats the process from setting the imaging procedure (step S101).

[0027] If it is determined that the desired image has been obtained (step S115: "YES"), first, an affine transformation is applied to correct distortion in the angle of view (step S121). The affine transformation process is shown in FIG. 8. As shown in the figure, the outer shape of the imaging jig 100 is used here. The outer shape of the imaging jig 100 is a rectangle (square) concentric with the center of the window portion OP, and the affine transformation is performed using the coordinates of its four vertices af1 to af4. Specifically, if the coordinates before transformation are (x, y) and the coordinates after transformation are (x', y'), the affine transformation can be defined by the following equation (1).

[0028]

number

[0029] The coordinates (x, y) of the four corners af1 to af4 of the imaging jig 100's outline can be determined by edge detection using a Sobel filter or similar on the captured image. Furthermore, because the length of each side of the imaging jig 100 is known, the coordinates (x', y') of the four corner points af1' to af4' can be defined by referencing the image resolution. These four sets of coordinates can be used to formulate simultaneous equations or solve an optimization problem to determine the matrix parameters that make up the affine transformation, i.e., a to f in the above equation (1). After determining the matrix parameters, equation (1) can be applied to all pixels that make up the captured image to determine the transformed coordinates.

[0030] Correction of distortion of the angle of view by applying affine transformation is assumed to be applied by default, but if it is determined from the position coordinates of the four corners that correction is not necessary, the processing of step S121 may be skipped. Furthermore, depending on the intended use of the texture after acquisition, a certain level of resolution may be required. In such cases, it is preferable to perform resolution conversion at the same time as the affine transformation. To increase the resolution, for example, to double the resolution, a new pixel is added between adjacent pixels. The gradation value of the new pixel can be set by interpolation based on the gradation values ​​of the adjacent existing pixels. It should be noted that correction of distortion of the angle of view may also be performed using coordinate transformation methods other than affine transformation, such as parallel projection transformation and perspective transformation.

[0031] Next, contrast correction is applied to the image after the correction of the distortion of the angle of view (step S131). Contrast correction is performed as a measure to deal with cases where the dynamic range of the captured image is narrow and the image is low in contrast. An example of an image in which a low-contrast image has been corrected to increase the contrast is shown in FIG. 9. The figure compares the low-contrast image CNL with the corrected image CNH. It can be seen that the range of brightness L of the original image has been expanded by the correction.

[0032] In this embodiment, contrast correction is performed using the average brightness Lw of the first region AR1, which is a white region in the captured image, and the average brightness Lb of the second region AR2, which is a black rectangular region. For this reason, the outer shapes of the first region AR1, the second region AR2, and the window portion OP are identified from the image after affine transformation. The outer shapes of each region can be identified by recognizing the four corners of each rectangular region (including the buffer region AB) in the image after affine transformation using edge detection and acquiring their coordinates.

[0033] If the brightness of the captured image of pixel g that exists in the window portion OP area is taken as pixel brightness Lg, then by calculating using the following equation (2), the contrast of a low-contrast image can be increased, resulting in an image with improved contrast. Lg'=f(Lg) ={(Lg-Lb) / (Lw-Lb)}×(LW0-LB0) …(2) The variables in this equation (2) are as follows: In the following, "white" corresponds to the color of the first area AR1, and "black" corresponds to the color of the second area AR2. Lb: Black of the imaging jig 100 in the captured image Lw: White of the imaging jig 100 in the captured image LB0: Black of the imaging jig 100 itself LW0: White of the imaging jig 100 itself

[0034] As an example, assume that the captured image has low contrast, and the average brightness of the second region AR2 in the captured image is Lb=20, and the average brightness of the first region AR1 is Lw=60. On the other hand, the brightness of the first region AR1 itself in the imaging jig 100 is LW0=100 as described above, and the brightness of the second region AR2 itself is LB0=0. Therefore, when calculated using equation (2), Lg'={(Lg-20) / (60-20)}×(100-0) If the brightness Lg of a pixel in the captured image is set to a value of 60, then Lg′ = 100, and if the brightness Lg of a pixel in the captured image is set to a value of 20, then Lg′ = 0, which shows that the dynamic range of the image has been expanded and the contrast has been improved.

[0035] If the image captured by the imaging unit 20 is expressed in RGB values, the RGB values ​​of the captured image are first converted to the Lab color system using a color space profile, the lightness L is determined, contrast correction is performed in the Lab space, and after correction, the values ​​are returned to the RGB values ​​by inverse conversion of the color space profile. Of course, if the texture image is to be used in the Lab color system as is, it may be handled in the Lab color system until the end without performing inverse conversion.

[0036] Because the above correction stretches the gradation values ​​of each pixel, if the range of gradation values ​​in the original image is narrow, the correction may result in a perceived step-like change. In such cases, it is preferable to forgo the above low-contrast correction, return to step S101, and start the process over again from the capture. In this case, it is also effective to preview the captured image once, determine exposure correction parameters using the brightness values ​​of white in the first area AR1 and black in the second area AR2 contained in the image, and feed these parameters back into the capture settings (step S101).

[0037] After contrast correction is applied to the captured image (step S131), brightness correction is then applied (step S141). Brightness correction can be performed by the following steps <1> to <6>. Note that the captured image is treated as having been converted into the Lab color space. The brightness correction process is shown in FIG. 10. The following description will refer to this figure as appropriate. <1> The rectangular position of the first area AR1 is identified from the captured image by edge determination. Note that the image to be subjected to brightness correction is the image after the distortion of the angle of view has been corrected by affine transformation and the contrast has been corrected. Therefore, if the rectangular position of the first area AR1 has been identified during these corrections, that rectangular position can be used as is. The pixel positions within this first area AR1 are expressed as (x, y). <2> Using the brightness values ​​L(x,y) of the pixels that make up the image in the first area AR1, the average brightness of the four sides of the first area AR1 is calculated. In Figure 10, for the pixels in the first area AR1, the average brightness Uav of each pixel position on the top side, the average brightness Rav of each pixel position on the right side, the average brightness Bav of each pixel position on the bottom side, and the average brightness Lav of each pixel position on the left side are calculated. The average brightness for each pixel position (x,y) is expressed as Uav(x,y). (3) The average brightness of the four sides is further averaged to obtain the overall average Aav.

[0038] <4> From the average brightness values ​​Uav(x,y), Rav(x,y), Bav(x,y), and Lav(x,y) of the four sides, a two-dimensional mesh MS corresponding to all pixels in the area surrounded by the four sides, i.e., the window portion OP, is assumed, and the distribution Lip(x,y) of the brightness value L is calculated by interpolation. <5> The brightness correction value ΔL for correcting the brightness Lg(x, y) of the pixel g at the position (x, y) is calculated using the following equation (3). ΔL(x,y)=Aav-Lip(x,y) …(3) Here, Lip(x,y) is not the actual brightness value of the image within the window OP, but the distribution of brightness values ​​L obtained by interpolation from the average brightness values ​​Uav(x,y), Rav(x,y), Bav(x,y), and Lav(x,y) of the four sides. <6> The obtained brightness correction value ΔL(x, y) is added to the actual brightness value L(x, y) of each pixel g(x, y) of the image within the window portion OP. In this way, the lightness value L(x, y) of each pixel g(x, y) is corrected.

[0039] The brightness correction described above can be used to deal with brightness differences depending on the location, such as uneven lighting. Below, we will show an example of calculating the brightness correction value ΔL for a 2D mesh, assuming the size of the window OP is 3x3 pixels. In the middle of Figure 10, for ease of explanation, the average brightnesses Uav(x,y), Rav(x,y), Bav(x,y), and Lav(x,y) of the four sides are shown as 1x3 or 3x1, and the brightness value L interpolated from these is shown as a 3x3 distribution. In practice, the distribution can be expanded according to the number of pixels in the image within the window OP. If the brightness correction accuracy needs to be improved, the vertical direction of horizontally long sides, such as the average brightnesses Uav(x,y) and Bav(x,y), and the horizontal direction of vertically long sides, such as the average brightnesses Rav(x,y) and Lav(x,y), can be considered as a width corresponding to multiple pixels, and the representative brightness can be calculated by averaging the brightnesses of the pixels aligned in the width direction. Specifically, for example, if Uav(x,y) is composed of 10 pixels in the x direction and 100 pixels in the y direction, the brightness value L of 10 pixels is averaged to obtain 1x100 brightness value L, which is used as the average brightness Uav(x,y).

[0040] Here, when determining the brightness correction amount ΔL, information from the first region AR1, which is a white region, is used. This is because it is assumed that the print medium from which the texture is to be acquired is often primarily white, such as fabric. Therefore, if the print medium from which the texture is to be acquired has a background color, providing an area with color information similar to the background color in the imaging jig 100 enables more accurate correction. Of course, even if the subject TM has a background color, correction can be performed using information from the white first region AR1. Furthermore, while the four sides were determined from the first region AR1 adjacent to the window portion OP in this embodiment, they may also be determined from other regions as long as they are of a color other than completely black (L=0). Furthermore, in this embodiment, the brightness correction value ΔL is determined using the above-mentioned two-dimensional mesh method. However, it may also be defined in the form of a lookup table (LUT) or a gamma curve.

[0041] Following the correction of distortion of the angle of view due to affine transformation (step S121), application of contrast correction (step S131), and application of brightness correction (step S141) described above, a decision is made as to whether to perform image sharpening or blurring (step S145), and if the acquired captured image is out of focus, sharpening processing (step S151) is applied, and if luminance noise or high-sensitivity noise is observed in the captured image, blurring processing (step S161) is applied. These processes are mutually exclusive, so both processes are not performed.

[0042] The sharpening process (step S151) can be easily achieved by applying, for example, the Lucy-Richardson algorithm. The Lucy-Richardson algorithm is a method for sharpening a target out-of-focus image by multiplying the out-of-focus image by the ratio between the target out-of-focus image and an intentionally created blurred image. Since the Lucy-Richardson algorithm is a generalized method, a detailed description thereof will be omitted.

[0043] On the other hand, for the blurring process (step S161), it is effective to use a general blurring filter such as a Gaussian filter or a median filter. The filter size may be determined based on factors such as the size of the target texture image (the size of the window OP) and the texture shape. Generally, blurring can cause the characteristics of the resulting texture to be lost. For this reason, it is desirable to limit luminance noise and high-sensitivity noise to a level that does not significantly affect the texture. Specifically, it is preferable to set the filter size to approximately 3 x 3 as the default, and for the user to increase the size while viewing the processing results. Note that the processing results may be evaluated by a trained AI rather than by the user.

[0044] If the captured image is appropriate, neither sharpening nor blurring may be performed (step S145: “Not Required”), and the process may proceed to the next step S171. Sharpening and blurring may be disabled by default, and may be applied based on the user's judgment or the results of image analysis. Image analysis may be performed based on the pixel status of the first region AR1 and the second region AR2 of the imaging jig 100. For example, the edge strength of the first region AR1, etc. may be determined based on a threshold value, and if the edge strength is lower than the threshold value, sharpening may be applied. Alternatively, if the pixel variance in the first region AR1, etc. is large and luminance noise or high-sensitivity noise is observed, blurring may be applied. In either case, if possible, the process may return to step S101 and display a notification on the display 39 recommending the user to retake the image.

[0045] In step S171, an acquisition range is set, which is the range within which the texture image is to be cut out, and an image within that range is cut out (step S181). The texture image is an image of the subject TM that is visible through the window portion OP from the entire image of the imaging jig 100 that has been captured and to which various corrections have been applied. Rather than cutting out the entire range of the window portion OP as the texture image, a range slightly inside the window portion OP is set as the acquisition range for the texture image and cut out. This process is shown in FIG.

[0046] In the figure, the imaging jig 100 is primarily illuminated from the upper left. In this case, a shadow SD due to the thickness of the imaging jig 100 may be cast on the subject TM within the window OP, inside the window OP. When the lightness L* of each pixel is measured along a line segment JJ ​​passing through the center of the imaging jig 100 in the Y direction, a region SA within the window OP, where the lightness L* is slightly reduced, may be found. This is likely to be the shadow SD due to the thickness of the imaging jig 100 itself. Therefore, this region SA and a region SB of the same width on the opposite side of the center point are defined as a predetermined range to be removed, and the image of the rectangular region excluding this range is cropped as the texture image GTM. The cropping range may also be set as a fixed range, such as a predetermined number of pixels inside the boundary of the window OP.

[0047] After the texture image has been cut out within the set range, the texture image GTM is output to the memory 37 of the image processing device 30 (step S191). If the imaging unit 20 is a mobile terminal such as a smartphone, the texture image GTM may be output and stored on the mobile terminal itself, or may be saved on a server on a network via communication. The saved texture image GTM may be printed on a print medium P by the printer 40 for confirmation. After the texture image GTM has been output, this processing routine ends.

[0048] By executing the texture image acquisition processing routine described above, the texture image acquisition device 10 can acquire the texture of the subject TM using the imaging jig 100 while suppressing the effects of various noises that may be mixed in during imaging. In this embodiment, the effects of distortion of the angle of view, insufficient contrast, uneven brightness, out-of-focus, luminance noise, high-sensitivity noise, etc. can be suppressed. In particular, the effects of uneven lighting that occurs depending on the location and distance from the light source can be easily suppressed or eliminated. As a result, the obtained texture image GTM has the effects of various noises suppressed or removed, and can be treated as an accurate reproduction of the actual state of the subject TM.

[0049] Furthermore, when the first region AR1, second region AR2, etc. of the imaging jig 100 are printed by a printer 40 equipped with an ICC profile 50 and used as the imaging jig 100, the imaging jig 100 can be created at the desired timing, for example, each time a texture image is to be acquired. In this way, there is no need to worry about discoloration of the first region AR1, etc. of the imaging jig 100 or to manage the color development state. Naturally, the imaging jig 100 may be configured using a material that is resistant to fading or discoloration so that it can be used for a long period of time.

[0050] B. Other Embodiments: (B1) In the first embodiment, the window portion OP of the imaging jig 100 is square-shaped. However, it may be rectangular. It may also be polygonal, such as triangular or pentagonal. A circular shape cannot be used to correct the distortion of the angle of view because it is difficult to identify the direction of distortion in the angle of view. However, it can be used to correct a distortion other than the distortion of the angle of view, such as contrast correction. Furthermore, marking three or more points on the circular shape can be used to correct the angle of view. For example, as shown in Variation 1 of FIG. 12 , if the imaging jig 101 is circular, a black border FL is attached to the outer periphery of the circular window portion OP, and white notches MK are formed in three or more places within the border, the positions of the notches MK can be used to identify the direction of distortion in the angle of view and correct the distortion. Of course, as long as the correction of distortion in the angle of view is not based on the outer shape of the window portion OP, but on a polygonal shape, such as the outer shape of a rectangular imaging jig or the first area AR1, the shape of the window portion OP can be any shape.

[0051] (B2) In the above embodiment, the size and shape of the first region AR1, etc. are assumed to be given in the correction calculations, etc., performed by the image processing device 30. However, as shown in Variation 2 of FIG. 12, scales SCX and SCY with known intervals may be displayed inside the imaging jig 102 along the X and Y directions, and these scales SCX and SCY may be read from the image captured by the imaging unit 20 and used as the size of the first region AR1, etc.

[0052] (B3) As shown in Variation 3 of FIG. 12 , a code such as a QR code (registered trademark) may be recorded on the imaging jig 103. The image processing device 30 may use this code qq to recognize information about the imaging jig 103, such as the shape, size, or brightness of the first and second regions AR1 and AR2, from the image captured by the imaging unit 20. In this case, the necessary information may be stored directly in the code, or various information may be obtained from an external website based on a URL of an online site written in the code. The code qq is not limited to a two-dimensional code such as a QR code; it may also be a one-dimensional code such as a barcode. Alternatively, the information may be displayed as text and read by the image processing device 30 using an OCR function. Furthermore, the outline of the first region AR1 or the like may be printed in a predetermined color, and the color may be associated with the size of the outline.

[0053] (B4) In addition, as shown in FIG. 13 as a fourth modification, the imaging jig 104 may have a third region AR3 with a different shade or hue from the first region AR1 and the second region AR2. The third region AR3 may be located on the outermost periphery as shown, but its location is optional. The third region AR3 may be an intermediate gradation (normal gray) relative to the gradation values ​​of the first region AR1 and the second region AR2, thereby increasing the variety of noise adjustments that can be performed. For example, the color of the third region AR3 may be gray, such as RGB=(64,64,64), RGB=(128,128,128), or RGB=(192,192,192). The third region AR3 may also serve as the buffer region AB of the imaging jig 100 described above. Furthermore, by setting the third region AR3 to a specific color, it can be used for hue correction and other corrections. The area to be added is not limited to the third area, but may be increased to a fourth, fifth, etc.

[0054] (B5) Several imaging jigs 100-104 are shown, but the size of the window portion OP in each imaging jig may be large enough to be used as a texture image. For example, it may be about 5 cm or more. FIG. 13 shows a fifth modification in which the subject TM has a pattern, and the pattern repeats in the Y direction at a period wy. In this case, the length WY in the Y direction of the window portion OP provided in the imaging jig 105 is at least twice the period wy of the pattern in the Y direction, that is, WY>2·wy This relationship also applies to the dimension of the window portion OP in the X direction and the period of the pattern in the X direction.

[0055] (B6) In the above explanation, the first area AR1 and the second area AR2 are assumed to be white or black areas and are primarily used as a reference for contrast correction and brightness correction. However, the subject TM to be imaged may be colored or may have a colored pattern on a white background. In such cases, the white balance of the lighting may not be appropriate, resulting in color defects in the captured image. Such defects can be resolved by adjusting the color of the pattern on the imaging jig, for example, and by image processing in the image processing device 30.

[0056] Typical color defects of captured pixels include color cast and low saturation (dullness). How to deal with these defects will be explained below with reference to FIG. 14 as needed. <Color overlap> Color cast primarily occurs due to differences in the white balance between the illumination light source and the photographic equipment. The upper part of Figure 14 shows an example of a captured image with color cast. In this case, the photographic jig 106 is photographed with a non-white color, excluding the black second region AR2, including the white first region AR1, the buffer region AB, the background color AC of the photographic jig 106 outside the second region AR2, and the portion of the subject TM visible through the window OP. This is called color cast. To address this color cast, the image processing device 30 can perform a white balance correction. Since white balance correction in image processing is well-known, a detailed description will be omitted. However, if a gray region with a known brightness value L exists in the region surrounding the window OP, white balance correction can be easily performed using this region. If a gray region is not available, a white region such as the first region AR1 can be used to adjust the RGB balance so that saturation does not occur in the white region. Of course, the user may be prompted to change the color temperature of the lighting device or the white balance on the imaging unit 20 side and redo the process from the imaging process (FIG. 7, step S101).

[0057] <Low saturation (dullness)> Another problem, low saturation (dullness), is primarily caused by insufficient exposure. Therefore, if low saturation is detected, it is effective to prompt the user to retake the image without correcting it. The bottom panel of Figure 14 shows an example in which color regions are provided on the imaging jig 107. In the illustrated example, a red region Ard, a green region Agr, and a blue region Abl are provided, in order from the inside out, outside the second region AR2. The hue and saturation of these regions are measured in advance, or color regions specified in an ICC profile are created on a printing press. The imaging jig 107 with these color regions is placed over the subject TM and imaged using the imaging unit 20. Edge detection is used to identify red, green, and blue (RGB) regions from the captured image, and their RGB values ​​are acquired. The Lab values ​​are calculated for these RGB values ​​using an ICC profile. The measured Lab values ​​are compared with the Lab values ​​used when the color regions were created. The amount of saturation correction is determined, and the saturation correction is performed. This can eliminate low saturation (dullness). It should be noted that the colors of the color regions formed on the imaging jig 107 do not need to be limited to the RGB shown in the example. If the color of the subject TM is known in advance, highly accurate saturation correction can be achieved by forming the color region with a color similar to the color of the subject TM. Also, as with color cast, the user may be prompted to start the process over again from the imaging process (FIG. 7, step S101).

[0058] (B7) In the above embodiments and variations, the imaging unit 20 has been described as having a single imaging function. However, when a mobile device such as a smartphone is used as the imaging unit 20, the camera 25 mounted on the imaging unit 20 may have lenses corresponding to multiple angles of view and automatically switch the angle of view depending on the target object, the distance to the target object, and other factors. A camera with a switchable angle of view may be equipped with two or more lenses, such as a super-telephoto lens, a telephoto lens, a standard lens, a wide-angle lens, or an ultra-wide-angle lens. Using a mobile device such as a smartphone as the imaging unit 20 facilitates the acquisition of texture images. However, if the lens used automatically switches depending on the imaging distance during imaging, it may be difficult to properly acquire an image of the subject TM overlaid with the imaging jig 100. Based on the assumption of the presence of such a camera 25, an example in which a predetermined pattern is provided in advance on the imaging jig is shown in FIG. 15. Note that at least some of the angle of view switching may involve switching the camera lenses, as well as switching the cameras themselves when multiple cameras are installed.

[0059] In the illustrated imaging jig 108, a resolution evaluation pattern RM is provided in the margin outside the second area AR2—here, in one corner (here, the lower left) of the rectangular imaging jig 108. The resolution evaluation pattern RM has at least an edge with varying shades of light and dark. Here, multiple black circles are formed as concentric circles spaced apart at predetermined intervals. In this example, the imaging unit 20 includes a camera 25 with three different angles of view: telephoto, standard, and wide-angle. In auto mode, the camera 25 has a function for selecting one of these angles of view based on the distance from the subject TM or the set magnification. The image processing device 30 includes a program for performing a resolution assessment process that acquires the edge intensity of the resolution evaluation pattern RM included in the captured image captured by the imaging unit 20 and determines that the resolution is insufficient if the edge intensity is below a predetermined threshold.

[0060] In a texture image acquisition device 10 using such an imaging jig 108, this resolution assessment process is performed in steps S101 to S115 of the texture image acquisition processing routine shown in FIG. 7. In the resolution assessment process, when a mobile device such as a smartphone is used as the imaging unit 20 and the camera 25 is automatically switched to a wide-angle or ultra-wide-angle view, the resolution of the image acquired by the camera 25 is detected using a resolution evaluation pattern RM. If the resolution is insufficient, the user is notified and prompted to switch the camera's angle of view to the telephoto side and take the image again. In the illustrated resolution assessment pattern RM, the spacing between the concentric circles is constant, but the spacing between the concentric circles may be varied to enable multiple levels of resolution assessment. Furthermore, concentric rectangular patterns are not limited to concentric circles, and may also be used. Alternatively, the assessment pattern may be two straight lines of a predetermined length separated by a predetermined distance, which are printed on the edge of the imaging jig. The resolution may then be assessed based on the results of edge assessment of the two lines.

[0061] (B8) In each of the above embodiments, some of the configurations realized by hardware may be replaced with software. At least a portion of the configurations realized by software may also be realized by a discrete circuit configuration. Furthermore, when some or all of the functions of the present disclosure are realized by software, the software (computer program) may be provided in a form stored on a computer-readable recording medium. The term "computer-readable recording medium" is not limited to portable recording media such as floppy disks and CD-ROMs, but also includes internal storage devices within a computer, such as various RAMs and ROMs, and external storage devices fixed to a computer, such as a hard disk. In other words, the term "computer-readable recording medium" has a broad meaning, including any recording medium capable of fixing data packets, not just temporarily.

[0062] The present disclosure is not limited to the above-described embodiments and can be realized in various configurations without departing from the spirit thereof. For example, the technical features in the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention section can be appropriately replaced or combined to solve some or all of the above-described problems or achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0063] 10...texture image acquisition device, 20...imaging unit, 22...stand, 25...camera, 30...image processing device, 31...communication unit, 32...arithmetic logic operation circuit (CPU, memory), 33...image acquisition unit, 34...correction processing unit, 36...cutting unit, 37...memory, 38...pattern output unit, 39...display, 40...printer, 50...ICC profile, 100-108...imaging jig, 110...mounting table, AB...buffer area, AR1...first area, AR2...second area, AR3...third area, OP...window portion, P...printing medium, RM...resolution evaluation pattern, TM...subject

Claims

1. A texture image acquisition device for acquiring a texture image of a medium, comprising: an imaging jig having a comparison pattern surrounding a window portion configured as a region of a predetermined shape that transmits light and including at least a first region and a second region with different shading; an imaging unit that captures an image including the medium in the window portion and the comparison pattern while the imaging jig is placed on top of the medium; a correction unit that acquires medium image data based on an image of the medium and comparison image data based on an image of the comparison pattern from the captured image, and corrects the medium image data based on image data of the first region and the second region included in the comparison image data; a cutout unit that cuts out an image of the texture of the medium from the corrected medium image data; A texture image acquisition device comprising:

2. The texture image acquisition device according to claim 1 , wherein the correction unit corrects the medium image data based on the actual density of the comparison pattern and the comparison image data.

3. 2. The texture image acquisition device according to claim 1, wherein at least one of the first region and the second region included in the comparison pattern of the imaging jig is created by printing with a printer capable of identifying colors including the shading using an ICC profile.

4. The texture image acquisition device according to claim 1 , wherein the cropping unit crops, as the texture image, data of pixels present in a predetermined acquisition range including a center point of the window portion included in the corrected medium image data.

5. The texture image acquisition device according to claim 4 , wherein the acquisition range is an area inside the window portion that does not include at least some of the pixels present in a predetermined area inside the outer periphery of the window portion.

6. 2. The texture image acquisition device according to claim 1, wherein the object of correction by the correction unit includes at least one of distortion of the angle of view, unevenness caused by lighting, low contrast due to insufficient exposure, defocus, luminance noise, and high-sensitivity noise that may occur in the medium image data when the imaging unit captures the image.

7. the imaging jig includes a resolution evaluation pattern having at least an edge where the density changes; acquiring an edge intensity of the resolution evaluation pattern included in the captured image captured by the imaging unit, and performing a resolution determination process to determine that the resolution is insufficient if the edge intensity is equal to or less than a predetermined threshold value; The texture image acquisition device according to claim 1 .

8. the imaging unit is a portable terminal that includes cameras with multiple angles of view and selects one of the multiple angles of view depending on a distance to a subject or a magnification; performing the resolution determination process when the angle of view of the imaging unit is selected; The texture image acquisition device according to claim 7 .

9. 1. A method for obtaining a texture image of a medium, comprising: an imaging jig having a comparison pattern surrounding a window portion configured as a region of a predetermined shape that transmits light and including at least a first region and a second region with different shadings is placed on the medium, and an image including the medium in the window portion and the comparison pattern is captured; Acquiring medium image data based on an image of the medium and comparison image data based on an image of the comparison pattern from the captured image, and correcting the medium image data based on image data of the first region and the second region included in the comparison image data; obtaining an image of the texture of the medium from the corrected medium image data; How to obtain texture images.

10. 10. The method for acquiring a texture image according to claim 9, wherein the target of correction in the medium image data includes at least one of distortion of the angle of view, unevenness caused by lighting, low contrast due to insufficient exposure, defocus, luminance noise, and high-sensitivity noise that may occur in the medium image data during the image capture.

11. The method for acquiring a texture image according to claim 9, wherein the targets of the correction in the medium image data include distortion of the angle of view, inappropriate contrast, inappropriate brightness, and inappropriate sharpness that may occur in the medium image data during the image capture, and the correction is performed in the order of correction of the distortion of the angle of view, adjustment of the contrast, adjustment of the brightness, and adjustment of the sharpness.

12. 1. A computer-executable program for capturing a texture image of a medium, comprising: a function of capturing an image including the medium in the window portion and the comparison pattern from a subject in a state in which an imaging jig is superimposed on the medium, the imaging jig surrounding the window portion configured as a region of a predetermined shape that transmits light and including a comparison pattern including at least a first region and a second region with different shading; a function of acquiring medium image data based on an image of the medium and comparison image data based on an image of the comparison pattern from the captured image, and correcting the medium image data based on image data of the first region and the second region included in the comparison image data; a function of acquiring an image of the texture of the medium from the corrected medium image data; A program implemented by a computer.

13. An imaging tool for imaging a texture of a medium, comprising: a window portion provided at the center of the imaging jig and configured as an area of ​​a predetermined shape; a comparison pattern including at least a first region and a second region having different shadings, the comparison pattern surrounding the window portion; Equipped with the first region circumscribing the window portion and surrounding the window portion, the second region surrounds the first region on the outer side of the first region, the first region and the second region have similar outer circumferential shapes that share a center point with the window portion; Imaging jig.

14. The imaging jig according to claim 13 , wherein the window portion is a polygonal window portion provided at the center of the imaging jig.

15. The imaging jig according to claim 14 , wherein the polygonal shape is a rectangular shape.

16. the first region has a rectangular outer periphery and circumscribes the window portion to surround the window portion, The second region has a rectangular outer periphery and surrounds the first region on the outer side of the first region. The imaging jig according to claim 14.

17. The imaging jig according to claim 13 , wherein at least one of the first region and the second region is white or black.

18. 14. The imaging jig according to claim 13, wherein each side of the window portion has a length at least twice the width of the period of the repeated shading provided in the medium.

Images