Image processing device, image processing method, and image processing program

Abstract

This image processing device derives the intensity distribution of illumination on the basis of first image data for a portion of a color chart in a captured image obtained by imaging in a state of overlap between a color development member that develops color in a concentration distribution that corresponds to the amount of applied energy, said color development member developing color with a specific color for which the difference in spectral reflectance with respect to the wavelengths of each of a plurality of colors falls within a given range, and a color chart of a color for which the difference falls outside the given range. The image processing device performs shading correction of second image data for a portion of the color development member in the captured image on the basis of the intensity distribution of the illumination.

Description

Image processing device, image processing method, and image processing program

[0001] The present disclosure relates to an image processing device, an image processing method, and an image processing program.

[0002] A conventional technique for measuring the amount of energy is known, which uses a color-changing member that changes color depending on the amount of energy applied to it. For example, one such color-changing member is Prescale (registered trademark) (manufactured by Fujifilm Corporation), which changes color depending on the pressure applied to it.

[0003] Japanese Patent Application Laid-Open No. 2017-203653 discloses a technique for performing shading correction on a captured image using a concentric color chart when capturing an image of a pressure measurement film.

[0004] In the technology described in JP 2017-203653 A, parameters for shading correction are generated using a concentric color chart as a preliminary calibration. In other words, if the shooting conditions in the preliminary calibration and the shooting conditions for capturing an image of the pressure measurement film differ from each other, shading correction cannot be performed with high accuracy.

[0005] For this reason, it is conceivable to photograph an image in a state where a color-developing member and a correction member such as a color chart are overlapped. However, the technology described in JP 2017-203653 A does not take into consideration performing shading correction on a photographed image obtained by photographing an image in a state where a color-developing member and a correction member are overlapped.

[0006] The present disclosure has been made in consideration of the above circumstances, and aims to provide an image processing device, an image processing method, and an image processing program that can perform shading correction with high accuracy on an image obtained by capturing an image in which a color-producing member and a correction member are superimposed.

[0007] The image processing device of the first aspect is an image processing device having at least one processor, wherein the processor acquires a captured image obtained by capturing a state in which a color-producing member that produces color with a density distribution according to the amount of energy applied, a color in which the difference in spectral reflectance for each wavelength of the multiple colors falls within a certain range and that is different from the multiple colors, and a color chart of a color in which the difference falls outside the certain range and that is arranged in a predetermined distribution state, are superimposed, derives an illumination intensity distribution based on first image data of the color chart portion in the captured image, and performs shading correction on second image data of the color-producing member portion in the captured image based on the illumination intensity distribution.

[0008] In the image processing device of the second aspect, in the image processing device of the first aspect, the processor derives the distribution of the amount of energy applied to the color-producing member using the second image data after shading correction.

[0009] In the image processing device of a third aspect, in the image processing device of the first or second aspect, the specific color is gray.

[0010] An image processing device of a fourth aspect is an image processing device of any one of the first to third aspects, in which the distribution state of the color chart includes the arrangement position of the color chart and the area ratio of the color chart portion to the color-producing member portion in a partial image of the captured image.

[0011] The image processing method of the fifth aspect involves acquiring a captured image obtained by photographing a state in which a color-producing member that produces color with a density distribution according to the amount of energy applied, a color in which the difference in spectral reflectance for each wavelength of the multiple colors falls within a certain range and that is different from the multiple colors, and a color chart of a color in which the difference falls outside the certain range and that is arranged in a predetermined distribution state, and deriving an illumination intensity distribution based on first image data of the color chart portion in the captured image, and performing a process by a processor provided in the image processing device to perform shading correction on second image data of the color-producing member portion in the captured image based on the illumination intensity distribution.

[0012] The image processing program of the sixth aspect is for acquiring a captured image obtained by photographing a color-producing member that produces color with a density distribution according to the amount of energy applied, wherein the difference in spectral reflectance for each of the multiple colors falls within a certain range and that produces a specific color different from the multiple colors, and a color chart of a color where the difference falls outside the certain range and that is arranged in a predetermined distribution state, in an overlapping state, deriving an illumination intensity distribution based on first image data of the color chart portion in the captured image, and performing a process of shading correction on second image data of the color-producing member portion in the captured image based on the illumination intensity distribution.

[0013] According to the present disclosure, shading correction can be performed with high accuracy on a captured image obtained by capturing an image in a state where a color-developing member and a correction member are superimposed.

[0014] FIG. 1 is a block diagram showing an example of a schematic configuration of a pressure measurement system; FIG. 2 is a diagram showing an example of a color-producing member; FIG. 3 is a block diagram showing an example of a hardware configuration of an image processing device; FIG. 4 is a diagram showing an example of characteristic data; FIG. 5 is a diagram showing an example of a sheet-like member on which a color chart is formed; FIG. 6 is a diagram showing an example of a state in which a color-producing member and a color chart are superimposed; FIG. 7 is a diagram for explaining the size of a color chart; FIG. 8 is a block diagram showing an example of a functional configuration of an image processing device; and FIG. 9 is a flowchart showing an example of pressure measurement processing.

[0015] Hereinafter, an example of an embodiment of the technology of the present disclosure will be described in detail with reference to the drawings. In this embodiment, an example in which pressure is applied as energy to an object will be described. Examples of objects to be pressurized include plate-shaped metals and semiconductor wafers.

[0016] First, the configuration of a pressure measurement system 1 according to this embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the pressure measurement system 1 includes an image processing device 10. Examples of the image processing device 10 include a portable computer such as a smartphone or a tablet terminal. Note that the image processing device 10 may also be a stationary computer.

[0017] As shown in Fig. 2, the pressure measurement system 1 measures the amount of energy using a color-forming member 90 that, when energy (pressure in this embodiment) is applied, develops a color with a density distribution corresponding to the amount of energy applied. In the example of Fig. 2, the hatched areas indicate the colored areas. Specifically, the image processing device 10 uses a camera 40 (see Fig. 3) to capture an image of the color-forming member 90 in a state in which it has developed a color due to the application of energy, and derives the amount of energy applied to the color-forming member 90 from the captured image.

[0018] An example of the color-forming member 90 is Prescale (registered trademark) (manufactured by Fujifilm Corporation), which produces a color density corresponding to the applied pressure. The prescale is a sheet-like support coated with a color former containing microcapsules containing a colorless dye and a color developer. When pressure is applied to the prescale, the microcapsules are destroyed, and the colorless dye is adsorbed to the color developer, producing color. Furthermore, because the color former contains multiple types of microcapsules with different sizes and strengths, the amount of microcapsules destroyed varies depending on the applied pressure, resulting in different color density. Therefore, by observing the color density, the magnitude and pressure distribution of the pressure applied to the prescale can be measured.

[0019] When energy is applied to the color-producing member 90 according to this embodiment, the color-producing member 90 produces a specific color (hereinafter simply referred to as a "specific color") that is different from the multiple colors and in which the difference in spectral reflectance for each wavelength of the multiple colors falls within a certain range. Specifically, the color-producing member 90 produces a specific color in which the spectral reflectance for each wavelength of light of the three primary colors of light, red, green, and blue, is constant. An example of this specific color is gray. That is, in the color-producing member 90, the portion to which energy is not applied is white, and the density value of the image in the portion to which energy is applied decreases as the amount of energy increases, resulting in a color approaching black. Note that the "constant spectral reflectance for each wavelength of the multiple colors" may mean that the difference in spectral reflectance for each wavelength of the multiple colors is within a certain range, including a tolerance.

[0020] Next, the hardware configuration of the image processing device 10 according to this embodiment will be described with reference to Fig. 3. As shown in Fig. 3, the image processing device 10 includes a CPU (Central Processing Unit) 20, a memory 21 as a temporary storage area, and a non-volatile storage unit 22. The image processing device 10 also includes a display 23 such as a liquid crystal display, an input device 24 such as a touch panel, a network I / F (Interface) 25 connected to a network, and a camera 40. The CPU 20, the memory 21, the storage unit 22, the display 23, the input device 24, the network I / F 25, and the camera 40 are connected to a bus 27. The CPU 20 is an example of a processor according to the disclosed technology.

[0021] The storage unit 22 is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or the like. The storage unit 22 serves as a storage medium and stores an image processing program 30. The CPU 20 reads the image processing program 30 from the storage unit 22, loads it into the memory 21, and executes the loaded image processing program 30.

[0022] The storage unit 22 also stores characteristic data 32. Fig. 4 shows an example of the characteristic data 32. The characteristic data 32 is data that defines a predetermined relationship between the amount of energy applied to the color-forming member 90 and the density value of the color-forming member 90 included in an image obtained by photographing the color-forming member 90. As the amount of energy, a physical quantity corresponding to energy that can be measured using the color-forming member 90, such as a pressure value, can be appropriately applied. Note that although Fig. 4 shows that the pressure value and the density value are proportional, the relationship between the pressure value and the density value is not necessarily limited to a proportional relationship.

[0023] The camera 40 includes an image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The camera 40 captures an image of the color-producing member 90 and outputs the captured image to the CPU 20. A scanner may be used instead of the camera 40. In other words, capturing an image here also includes reading an image using a scanner.

[0024] The pressure measurement system 1 according to this embodiment uses a sheet-like member 92 shown in FIG. 5 . The sheet-like member 92 is, for example, a transparent sheet having a spectral transmittance that is as close to 100% as possible. As shown in FIG. 5 as an example, a plurality of color charts 94 are formed on the sheet-like member 92, each of which has a difference in spectral reflectance for each wavelength of the plurality of colors that falls outside the certain range. In this embodiment, the color chart 94 is formed of the most intense red, for example, a red with a gradation value of 255 in the case of 256 gradations. Note that although FIG. 5 illustrates a rectangular shape as the color chart 94, the shape of the color chart 94 is not limited to a rectangular shape.

[0025] 6 , in this embodiment, when the image processing device 10 photographs the color-producing member 90 using the camera 40, a sheet-like member 92 is superimposed on the color-producing member 90. The camera 40 photographs the color-producing member 90 with the color-producing member 90 and the plurality of color charts 94 superimposed on top of each other, and outputs the image obtained by photographing (hereinafter referred to as the “photographed image”) to the CPU 20.

[0026] The color charts 94 are arranged in a predetermined distribution. The distribution here includes the positions of the color charts 94. Specifically, as shown in Fig. 5, the color charts 94 are arranged at equal intervals along one side of the rectangular sheet-like member 92 and along a side perpendicular to that side, and since the intervals are known, the positions of the color charts 94 are also known. Note that it is sufficient that the positions of the color charts 94 are known, and the intervals between adjacent color charts 94 are not limited to being equal.

[0027] The distribution state of the multiple color charts 94 also includes the area ratio between the color chart 94 portion and the color-producing member 90 portion in a partial image of the captured image. A partial image according to this embodiment is an image equivalent to one pixel in the captured image. As shown in FIG. 7, the area of ​​the color chart 94 in the captured image according to this embodiment is smaller than the area of ​​the partial image equivalent to one pixel according to the resolution of the captured image. In this embodiment, the area ratio between the color chart 94 portion and the color-producing member 90 portion in this partial image equivalent to one pixel is known, as shown in the following equation (1). Color chart 94 portion: color-producing member 90 portion=s:1-s (1)

[0028] Note that, as long as the area ratio between the color chart 94 portion and the color-producing member 90 portion in the partial image of the captured image is known, the area of ​​the color chart 94 is not limited to the example shown in Fig. 7. For example, the area of ​​the color chart 94 may be an area equivalent to one pixel of the captured image. In this case, the partial image may be an image of multiple pixels, such as 3 x 3 pixels, with the color chart 94 portion at the center.

[0029] Next, the functional configuration of the image processing device 10 according to this embodiment will be described with reference to Fig. 8. As shown in Fig. 8, the image processing device 10 includes an imaging control unit 50, an acquisition unit 52, a first derivation unit 54, a correction unit 56, and a second derivation unit 58. The CPU 20 executes the image processing program 30, thereby functioning as the imaging control unit 50, the acquisition unit 52, the first derivation unit 54, the correction unit 56, and the second derivation unit 58.

[0030] The user inputs a shooting instruction via the input device 24 with the sheet-like member 92 superimposed on the color-producing member 90. When the user inputs the shooting instruction, the shooting control unit 50 controls the camera 40 to take a shooting image with the color-producing member 90 and the sheet-like member 92 superimposed on each other.

[0031] The acquisition unit 52 acquires a captured image obtained by capturing an image of the color-producing member 90 and the multiple color charts 94 formed on the sheet-like member 92 in an overlapping state under the control of the imaging control unit 50 .

[0032] The first derivation unit 54 derives the illumination intensity distribution based on the first image data of the color chart 94 portion in the captured image acquired by the acquisition unit 52. Details of the process of deriving the illumination intensity distribution by the first derivation unit 54 will be described below. Here, it is assumed that each pixel in the captured image has a pixel value for each of the colors red, green, and blue.

[0033] If the theoretical values ​​of each color on the color chart 94 are red Rm, green Gm, and blue Bm, the actual measured values ​​of each color on the color chart 94 in the captured image are affected by the lighting and are therefore expressed by the following equations (2) to (4). In equations (2) to (4), illk(x, y) represents the intensity distribution of the lighting when the captured image was captured, x represents, for example, the horizontal pixel position with the upper left corner of the captured image as the origin, and y represents, for example, the vertical pixel position with the upper left corner of the captured image as the origin. illk(x, y)×Rm (2) illk(x, y)×Gm (3) illk(x, y)×Bm (4)

[0034] If the theoretical values ​​of each color in the color-producing member 90 are red Rp, green Gp, and blue Bp, the actual measured values ​​of each color in the color-producing member 90 in the captured image are affected by lighting and are therefore expressed by the following equations (5) to (7): illk(x, y)×Rp (5) illk(x, y)×Gp (6) illk(x, y)×Bp (7)

[0035] According to the above formulas (1) to (7), the first image data of the portion of the color chart 94 in the captured image, i.e., the pixel values ​​Ri(x, y), Gi(x, y), and Bi(x, y) of each color of the pixel including the color chart 94 in the captured image, are expressed by the following formulas (8) to (10). As shown in formulas (8) to (10), the pixel values ​​Ri(x, y), Gi(x, y), and Bi(x, y) are respectively the actual measurement value of the portion of the color chart 94 in the pixel and the actual measurement value of the portion of the color-producing member 90 multiplied by the area ratio shown in formula (1) and then added together. Ri (x, y) = s x illk (x, y) x Rm + (1-s) x illk (x, y) x Rp... (8) Gi (x, y) = s x illk (x, y) x Gm + (1-s) x illk (x, y) x Gp... (9) Bi(x,y)=s×illk(x,y)×Bm+(1-s)×illk(x,y)×Bp...(10)

[0036] From equations (8) and (9), the difference between Ri(x, y) and Gi(x, y) is expressed by the following equation (11): Ri(x, y) - Gi(x, y) = {s × illk(x, y) × Rm + (1 - s) × illk(x, y) × Rp} - {s × illk(x, y) × Gm + (1 - s) × illk(x, y) × Gp} = illk(x, y) × {s × (Rm - Gm) + (1 - s) × (Rp - Gp)} (11)

[0037] As described above, the color-producing member 90 produces specific colors with constant spectral reflectances for each wavelength of red, green, and blue light, so Rp and Gp are equal. Therefore, (Rp - Gp) in equation (11) is 0, and the illumination intensity distribution illk(x, y) is expressed by the following equation (12): illk(x, y) = ((Ri(x, y) - Gi(x, y)) ÷ (s × (Rm - Gm)) ... (12) As described above, s is known. Furthermore, Rm and Gm can also be known by, for example, placing a reference patch image or the like outside the color-producing member 90 and capturing an image. Therefore, the first derivation unit 54 can derive the illumination intensity distribution illk(x, y) in accordance with equation (12) based on the first image data of the portion of the color chart 94 in the captured image.

[0038] The correction unit 56 performs shading correction on the second image data of the portion of the color-producing member 90 in the captured image acquired by the acquisition unit 52, based on the illumination intensity distribution derived by the first derivation unit 54. The shading correction performed by the correction unit 56 will be described in detail below.

[0039] Since illk(x, y) is proportional to Ri(x, y) - Gi(x, y), equation (12) can be expressed as the following equation (13): D(x, y) = Ri(x, y) - Gi(x, y) (13)

[0040] The correction unit 56 derives the in-plane average value Dave of D(x, y). Next, the correction unit 56 derives Ds(x, y) according to the following equation (14). Ds(x, y) represents the surface distribution of the deviation of D(x, y) from the average value, where the average value is 1. Ds(x, y) = D(x, y) ÷ Dave (14)

[0041] Next, the correction unit 56 derives a correction parameter SH(x, y) for shading correction according to the following equation (15): SH(x, y)=1÷Ds(x, y) (15)

[0042] Then, the correction unit 56 performs shading correction by multiplying the second image data by a correction parameter SH(x, y) in accordance with the following equations (16) to (18). Rs(x, y), Gs(x, y), and Bs(x, y) in equations (16) to (18) represent pixel values ​​of each color of the second image data after shading correction. Rs(x, y) = R(x, y) × SH(x, y) (16) Gs(x, y) = G(x, y) × SH(x, y) (17) Bs(x, y) = B(x, y) × SH(x, y) (18)

[0043] The second derivation unit 58 derives the pressure distribution applied to the color-forming member 90 using the characteristic data 32 and the second image data after shading correction by the correction unit 56. Specifically, the second derivation unit 58 derives the pressure distribution by converting the density into a pressure value using the characteristic data 32 for each pixel of the second image data after shading correction.

[0044] For pixels including the color chip 94, the second derivation unit 58 may derive the pressure value applied to the color-producing member 90 using characteristic data prepared in consideration of the density of the color chip 94 and the above-mentioned area ratio, separate from the characteristic data 32. Furthermore, for pixels including the color chip 94, the second derivation unit 58 may derive the pressure value by interpolating according to a known interpolation algorithm using, for example, eight neighboring pixels. The second derivation unit 58 may use the pressure value during the interpolation process, or may interpolate using pixel values ​​and then convert them into a pressure value using the characteristic data 32.

[0045] Next, the operation of the image processing device 10 according to this embodiment will be described with reference to Fig. 9. The CPU 20 executes the image processing program 30, thereby executing the pressure measurement process shown in Fig. 9. The pressure measurement process shown in Fig. 9 is executed, for example, when a user inputs an image capture instruction via the input device 24.

[0046] 9, the photographing control unit 50 controls the camera 40 to photograph an image in a state in which the color-producing member 90 and the sheet-like member 92 are superimposed. In step S12, the acquisition unit 52 acquires a photographed image obtained by photographing the color-producing member 90 and the plurality of color charts 94 formed on the sheet-like member 92 in a state in which they are superimposed, as controlled in step S10.

[0047] In step S14, the first derivation unit 54 derives the illumination intensity distribution based on the first image data of the portion of the color chart 94 in the photographed image acquired in step S12, as described above. In step S16, the correction unit 56 performs shading correction on the second image data of the portion of the color-producing member 90 in the photographed image acquired in step S12, based on the illumination intensity distribution derived in step S14, as described above.

[0048] In step S18, as described above, the second derivation unit 58 uses the characteristic data 32 and the second image data after the shading correction in step S16 to derive the pressure distribution applied to the color-forming member 90. When the processing of step S18 ends, the pressure measurement processing ends.

[0049] As described above, according to this embodiment, shading correction can be performed with high accuracy on a captured image obtained by capturing an image in a state where a color-developing member and a correction member are superimposed on each other.

[0050] In the above embodiment, a case where pressure is applied as the energy applied to the object is described, but the disclosed technology is not limited to this aspect. For example, a configuration in which heat or ultraviolet light is applied as the energy applied to the object may be used. When heat is applied as the energy applied to the object, Thermoscale (product name) (manufactured by Fujifilm Corporation), which changes color depending on the amount of heat, can be used as the color-changing member 90. When ultraviolet light is applied as the energy applied to the object, UVScale (product name) (manufactured by Fujifilm Corporation), which changes color depending on the amount of ultraviolet light, can be used as the color-changing member 90.

[0051] Furthermore, in the above embodiment, for example, the following various processors can be used as the hardware structure of a processing unit that executes various processes, such as each functional unit of the image processing device 10. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits, such as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing specific processes.

[0052] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0053] Examples of configuring multiple processing units with a single processor include: first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server computers, and this processor functions as multiple processing units; second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs); in this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0054] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0055] In the above embodiment, the image processing program 30 is pre-stored (installed) in the storage unit 22, but this is not limiting. The image processing program 30 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The image processing program 30 may also be downloaded from an external device via a network.

[0056] The disclosure of Japanese Patent Application No. 2023-146444, filed on September 8, 2023, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. An image processing device having at least one processor, wherein the processor acquires a captured image obtained by capturing a superimposed state of a color-producing member that produces color with a density distribution according to an amount of energy applied thereto, the color being different from the multiple colors, and the difference in spectral reflectance for each wavelength of the multiple colors being within a certain range, and a color chart of a color whose difference falls outside the certain range and that is arranged in a predetermined distribution state; derives an illumination intensity distribution based on first image data of a portion of the color chart in the captured image; and performs shading correction on second image data of a portion of the color-producing member in the captured image based on the intensity distribution.

2. The image processing device according to claim 1, wherein the processor derives a distribution of the amount of energy applied to the color-producing member using the second image data after the shading correction has been performed.

3. The image processing device according to claim 1 or 2, wherein the specific color is gray.

4. An image processing device according to claim 1 or 2, wherein the distribution state includes the arrangement position of the color chart and the area ratio of the color chart portion to the color-developing member portion in a partial image of the captured image.

5. An image processing method in which a processor provided in an image processing device executes the following processes: acquiring a captured image obtained by photographing a color-producing member that produces a color with a density distribution according to the amount of energy applied, the color being different from the multiple colors, and the difference in spectral reflectance for each wavelength of the multiple colors being within a certain range, and a color chart of a color whose difference is outside the certain range, the color chart being arranged in a predetermined distribution state; deriving an illumination intensity distribution based on first image data of a portion of the color chart in the captured image; and performing shading correction on second image data of the portion of the color-producing member in the captured image based on the intensity distribution.

6. An image processing program for causing a processor of an image processing device to execute the following processes: acquiring a captured image obtained by photographing a color-producing member that produces a color with a density distribution according to the amount of energy applied, the color being a specific color different from the multiple colors, the difference in spectral reflectance for each wavelength of the multiple colors being within a certain range, and a color chart of a color whose difference is outside the certain range and that is arranged in a predetermined distribution state; deriving an illumination intensity distribution based on first image data of a portion of the color chart in the captured image; and performing shading correction on second image data of a portion of the color-producing member in the captured image based on the intensity distribution.

Images