Image processing equipment, image processing methods, and recording media.

Abstract

The invention relates to an image processing device (10) using first image data in the first wavelength range, in which the variation in spectral reflectance with the amount of applied energy is within a certain range, in a photograph obtained by color-enhanced detail imaging which is color-enhanced with density distribution according to the amount of energy, to perform shading correction on second image data in the second wavelength range, in which the spectral reflectance varies with the amount of energy and is different from the first wavelength range, in the photograph.

Description

Image processing device, image processing method, and image processing program

[0001] The present disclosure relates to an image processing device, an image processing method, and an image processing program.

[0002] A conventional technique for measuring the amount of energy is known, which uses a color-changing member that changes color depending on the amount of energy applied to it. For example, one such color-changing member is Prescale (registered trademark) (manufactured by Fujifilm Corporation), which changes color depending on the pressure applied to it.

[0003] Japanese Patent Application Laid-Open No. 2017-203653 discloses a technique for performing shading correction on a captured image using a concentric color chart when capturing an image of a pressure measurement film.

[0004] However, the technology described in JP 2017-203653 A requires the use of a color chart, which is a member for shading correction, in order to perform shading correction on a captured image.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide an image processing device, an image processing method, and an image processing program that can perform shading correction on a captured image without using a shading correction component.

[0006] The image processing device of the first aspect is an image processing device having at least one processor, and the processor uses first image data of a first wavelength range in which the amount of change in spectral reflectance according to the amount of energy in a captured image obtained by photographing a color-producing component that produces color with a density distribution according to the amount of energy applied falls within a certain range, and performs shading correction on second image data of a second wavelength range different from the first wavelength range in which the spectral reflectance changes according to the amount of energy in the captured image.

[0007] In the image processing device of the second aspect, in the image processing device of the first aspect, the processor derives the distribution of the amount of energy applied to the color-producing member using the second image data after shading correction.

[0008] The image processing device of the third aspect is the image processing device of the first or second aspect, wherein the first wavelength range is a portion of the wavelength range on the long wavelength side of the red wavelength range, and the first image data is pixel values ​​of the red component in the captured image.

[0009] An image processing device according to a fourth aspect is the image processing device according to the third aspect, wherein the first wavelength range is a wavelength range of 640 nm or more and less than 700 nm.

[0010] An image processing device of a fifth aspect is an image processing device of any one of the first to fourth aspects, wherein the second wavelength range is at least one of a blue wavelength range and a green wavelength range, and the second image data is at least one of pixel values ​​of the blue component and pixel values ​​of the green component in the captured image.

[0011] An image processing device of a sixth aspect is an image processing device of any one of the first to fifth aspects, in which the color-producing member is a member that produces a color in a first wavelength range in which the change in spectral reflectance falls within a certain range regardless of the amount of energy.

[0012] An image processing device of a seventh aspect is an image processing device of any one of the first to sixth aspects, in which the captured image is an image captured through a filter that passes light in a first wavelength range and a second wavelength range and blocks light in a wavelength range between the first wavelength range and the second wavelength range.

[0013] The image processing method of the eighth aspect is performed by a processor provided in an image processing device, which uses first image data in a first wavelength range in which the amount of change in spectral reflectance in response to the amount of energy in a captured image obtained by photographing a color-producing member that exhibits color with a density distribution in response to the amount of energy applied falls within a certain range, and performs a process of performing shading correction on second image data in a second wavelength range different from the first wavelength range in which the spectral reflectance changes in response to the amount of energy in the captured image.

[0014] The image processing program of the ninth aspect is intended to cause a processor provided in an image processing device to execute a process of performing shading correction on second image data of a second wavelength range different from the first wavelength range in which the spectral reflectance changes according to the amount of energy in the captured image obtained by photographing a color-forming material that develops color with a density distribution according to the amount of energy applied, using first image data of a first wavelength range in which the amount of change in spectral reflectance according to the amount of energy in the captured image falls within a certain range.

[0015] According to the present disclosure, shading correction of a captured image can be performed without using a member for shading correction.

[0016] FIG. 1 is a block diagram showing an example of a schematic configuration of a pressure measurement system; FIG. 2 is a diagram showing an example of a color-forming member; FIG. 3 is a block diagram showing an example of a hardware configuration of an image processing device; FIG. 4 is a diagram showing an example of characteristic data; FIG. 5 is a graph showing an example of the spectral reflectance of a color-forming member according to the amount of energy applied; FIG. 6 is a block diagram showing an example of the functional configuration of an image processing device; and FIG. 7 is a flowchart showing an example of pressure measurement processing.

[0017] Hereinafter, an example of an embodiment of the technology of the present disclosure will be described in detail with reference to the drawings. In this embodiment, an example in which pressure is applied as energy to an object will be described. Examples of objects to be pressurized include plate-shaped metals and semiconductor wafers.

[0018] First, the configuration of a pressure measurement system 1 according to this embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the pressure measurement system 1 includes an image processing device 10. Examples of the image processing device 10 include a portable computer such as a smartphone or a tablet terminal. Note that the image processing device 10 may also be a stationary computer.

[0019] As shown in Fig. 2, the pressure measurement system 1 measures the amount of energy using a color-forming member 90 that, when energy (pressure in this embodiment) is applied, develops a color with a density distribution corresponding to the amount of energy applied. In the example of Fig. 2, the hatched areas indicate the colored areas. Specifically, the image processing device 10 uses a camera 40 (see Fig. 3) to capture an image of the color-forming member 90 in a state in which it has developed a color due to the application of energy, and derives the amount of energy applied to the color-forming member 90 from the captured image.

[0020] An example of the color-forming member 90 is Prescale (registered trademark) (manufactured by Fujifilm Corporation), which produces a color density corresponding to the applied pressure. The prescale is a sheet-like support coated with a color former containing microcapsules containing a colorless dye and a color developer. When pressure is applied to the prescale, the microcapsules are destroyed, and the colorless dye is adsorbed to the color developer, producing color. Furthermore, because the color former contains multiple types of microcapsules with different sizes and strengths, the amount of microcapsules destroyed varies depending on the applied pressure, resulting in different color density. Therefore, by observing the color density, the magnitude and pressure distribution of the pressure applied to the prescale can be measured.

[0021] Next, the hardware configuration of the image processing device 10 according to this embodiment will be described with reference to Fig. 3. As shown in Fig. 3, the image processing device 10 includes a CPU (Central Processing Unit) 20, a memory 21 as a temporary storage area, and a non-volatile storage unit 22. The image processing device 10 also includes a display 23 such as a liquid crystal display, an input device 24 such as a touch panel, a network I / F (Interface) 25 connected to a network, and a camera 40. The CPU 20, the memory 21, the storage unit 22, the display 23, the input device 24, the network I / F 25, and the camera 40 are connected to a bus 27. The CPU 20 is an example of a processor according to the disclosed technology.

[0022] The storage unit 22 is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or the like. The storage unit 22 serves as a storage medium and stores an image processing program 30. The CPU 20 reads the image processing program 30 from the storage unit 22, loads it into the memory 21, and executes the loaded image processing program 30.

[0023] The storage unit 22 also stores characteristic data 32. An example of the characteristic data 32 is shown in FIG. 4. The characteristic data 32 is data that defines a predetermined relationship between the amount of energy applied to the color-forming member 90 and the density of the color-forming member 90 included in an image obtained by photographing the color-forming member 90. As the amount of energy, a physical quantity corresponding to energy that can be measured using the color-forming member 90, such as a pressure value, can be appropriately applied. Note that although the pressure value and density are proportional in FIG. 4, the relationship between the pressure value and density is not necessarily limited to a proportional relationship.

[0024] The camera 40 includes an image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The camera 40 photographs the color-producing member 90 and outputs the image obtained by photographing (hereinafter referred to as the "photographed image") to the CPU 20. Note that a scanner may be used instead of the camera 40. In other words, photographing an image as used herein also includes reading by a scanner.

[0025] The color-forming member 90 develops a color at a density corresponding to the amount of energy applied thereto, and the spectral reflectance of the color-forming member 90 also changes according to the amount of energy applied. The spectral reflectance characteristics of the color-forming member 90 according to this embodiment will be described with reference to Fig. 5 . The vertical axis of Fig. 5 represents the spectral reflectance, and the horizontal axis represents the wavelength of light. Furthermore, Fig. 5 shows the spectral reflectance of the color-forming portion of the color-forming member 90 when five different levels of energy are applied to the color-forming member 90.

[0026] As shown in FIG. 5 , among the three primary colors of light, red, green, and blue, in a portion of the wavelength range on the long-wavelength side of the red wavelength range (hereinafter referred to as the “first wavelength range”), the amount of change in spectral reflectance depending on the amount of energy falls within a certain range. The “certain range of change” here refers to, for example, a range within an allowable error, such as a range in which the amount of change in spectral reflectance depending on the amount of energy falls within plus or minus 1%. In the example of FIG. 5 , the first wavelength range is indicated by a dashed rectangle. In this embodiment, an example will be described in which the blue wavelength range is from 400 nm to less than 500 nm, the green wavelength range is from 500 nm to less than 550 nm, and the red wavelength range is from 550 nm to less than 700 nm.

[0027] 5, in the color-producing member 90 according to this embodiment, the amount of change in spectral reflectance according to the amount of energy falls within a certain range in the wavelength range of 640 nm or more and less than 700 nm. In this embodiment, an example will be described in which the wavelength range of 640 nm or more and less than 700 nm is applied as the first wavelength range. In other words, the color-producing member 90 according to this embodiment is a member that produces a color in the first wavelength range in which the amount of change in spectral reflectance falls within a certain range regardless of the amount of energy applied.

[0028] Furthermore, in the color-producing member 90 according to this embodiment, the spectral reflectance changes depending on the amount of energy in the blue wavelength range and the green wavelength range. Hereinafter, the wavelength range in which the spectral reflectance changes depending on the amount of energy is referred to as the "second wavelength range." The second wavelength range is different from the first wavelength range. In the example of FIG. 5 , the second wavelength range is indicated by a dashed-dotted rectangle. In this embodiment, an example will be described in which the blue wavelength range and the green wavelength range, i.e., the range of 400 nm or more and less than 550 nm, are applied as the second wavelength range.

[0029] In this embodiment, the camera 40 captures an image of the color-producing member 90 through a filter that passes light in the first and second wavelength ranges and blocks light in a wavelength range between the first and second wavelength ranges. A multi-bandpass filter with a steep characteristic is preferable as such a filter. In this embodiment, a vapor deposition filter is used as the multi-bandpass filter. It is preferable that light is incident perpendicularly to the surface of the vapor deposition filter. Therefore, in this embodiment, the user attaches a filter to the lens of the camera 40 included in the image processing device 10 and presses an external lens against the filter, causing the camera 40 to capture an image of the color-producing member 90. This image capture method is called a collimation method, in which a filter is placed between the camera lens and the external lens. Thus, the captured image according to this embodiment is an image captured through a filter that passes light in the first and second wavelength ranges and blocks light in a wavelength range between the first and second wavelength ranges.

[0030] Next, the functional configuration of the image processing device 10 according to this embodiment will be described with reference to Fig. 6. As shown in Fig. 6, the image processing device 10 includes an imaging control unit 50, an acquisition unit 52, a correction unit 54, and a derivation unit 56. The CPU 20 executes the image processing program 30, thereby functioning as the imaging control unit 50, the acquisition unit 52, the correction unit 54, and the derivation unit 56.

[0031] As described above, the user inputs an instruction to photograph the color-producing member 90 through a filter that passes light in the first wavelength range and the second wavelength range and blocks light in a wavelength range between the first wavelength range and the second wavelength range via the input device 24. When the user inputs the instruction to photograph, the photographing control unit 50 controls the camera 40 to photograph an image of the color-producing member 90.

[0032] The acquisition unit 52 acquires a captured image obtained by capturing an image of the color-producing member 90 under the control of the imaging control unit 50 .

[0033] The correction unit 54 performs shading correction on second image data of a second wavelength range in the captured image using first image data of a first wavelength range in the captured image acquired by the acquisition unit 52. The correction unit 54 according to this embodiment uses pixel values ​​of the red component in the captured image, i.e., R (Red) pixel values, as the first image data. The correction unit 54 also uses pixel values ​​of the blue component in the captured image, i.e., B (Blue) pixel values, as the second image data. Hereinafter, the first image data will also be referred to as R(x, y), and the second image data will also be referred to as B(x, y). For example, x represents the horizontal pixel position with the upper left corner of the captured image as the origin. For example, y represents the vertical pixel position with the upper left corner of the captured image as the origin. The shading correction performed by the correction unit 54 will be described in detail below.

[0034] The colored portion of the first image data is a portion having a nearly constant spectral reflectance regardless of the amount of energy applied. This spectral reflectance is defined as r1. The correction unit 54 may use, as r1, a representative value such as the average or median of the spectral reflectance in the first wavelength range shown in FIG. 5. The spectral reflectance of the portion of the first image data to which no energy was applied and which did not develop color (hereinafter referred to as the "background portion") is defined as r2.

[0035] When r1 and r2 are approximately equal, that is, when the absolute value of the difference between r1 and r2 is equal to or less than a threshold value, the correction unit 54 derives a correction parameter SH(x, y) for shading correction according to the following formula (1). W1 in formula (1) represents a target value for shading correction, and for example, in the case of an image with 256 gradations, W1 is set to 255 / r1. SH(x, y)=W1 / R(x, y) (1)

[0036] If the first image data contains a large amount of noise, the correction unit 54 may perform a known noise reduction process on the first image data before deriving the correction parameter SH(x, y). Examples of the noise reduction process in this case include filtering using a median filter, an image averaging filter, or the like.

[0037] If the absolute value of the difference between r1 and r2 exceeds a threshold, the correction unit 54 performs a process of distinguishing between colored portions and background portions in the first image data. In this case, the correction unit 54 distinguishes between colored portions and background portions in the first image data by utilizing the fact that, for example, shading caused by the illumination intensity distribution and the lens is a gradual change, while the transition between the colored portions and the background portion is an extremely abrupt change.

[0038] The correction unit 54 then derives the correction parameter SH(x, y) according to the following equation (2). In equation (2), W1p represents a target value for shading correction of the colored portion, and for example, in the case of an image with 256 gradations, W1p is set to 255÷r1. Furthermore, in equation (2), W1w represents a target value for shading correction of the background portion, and for example, in the case of an image with 256 gradations, W1w is set to 255÷r2.

[0039]

[0040] The correction unit 54 performs shading correction by multiplying the second image data B(x, y) by a correction parameter SH(x, y) according to the following equation (3): Bs(x, y) in equation (3) represents the second image data after shading correction: Bs(x, y)=B(x, y)×SH(x, y) (3)

[0041] The correction unit 54 may use pixel values ​​of the green component in the captured image, i.e., pixel values ​​of G (Green), as the second image data. In this case, the correction unit 54 performs shading correction by multiplying the second image data G(x, y) by a correction parameter SH(x, y) in accordance with the following equation (4). Gs(x, y) in equation (4) represents the second image data after shading correction. Gs(x, y) = G(x, y) × SH(x, y) (4)

[0042] Furthermore, the correction unit 54 may use both the pixel values ​​of the blue component and the pixel values ​​of the green component in the captured image as the second image data. In this case, the correction unit 54 may use, as the second image data, data obtained by dividing the pixel value of the red component by the sum of the pixel values ​​of the blue component and the pixel value of the green component for each pixel in the captured image.

[0043] The derivation unit 56 derives the pressure distribution applied to the color-forming member 90 using the characteristic data 32 and the second image data after shading correction by the correction unit 54. Specifically, the derivation unit 56 derives the pressure distribution by converting the density into a pressure value using the characteristic data 32 for each pixel of the second image data after shading correction.

[0044] Next, the operation of the image processing device 10 according to this embodiment will be described with reference to Fig. 7. The CPU 20 executes the image processing program 30, thereby executing the pressure measurement process shown in Fig. 7. The pressure measurement process shown in Fig. 7 is executed, for example, when a user inputs an image capture instruction via the input device 24.

[0045] 7, as described above, the photographing control unit 50 controls the camera 40 to photograph an image of the color-forming member 90. In step S12, the acquisition unit 52 acquires a photographed image obtained by photographing the color-forming member 90 under the control in step S10.

[0046] In step S14, the correction unit 54 performs shading correction on the second image data in the second wavelength range of the captured image using the first image data in the first wavelength range of the captured image acquired in step S12, as described above. In step S16, the derivation unit 56 derives the pressure distribution applied to the color-producing member 90 using the characteristic data 32 and the second image data after the shading correction in step S14, as described above. When the processing of step S16 ends, the pressure measurement process ends.

[0047] As described above, according to this embodiment, shading correction of a captured image can be performed without using a member such as a color chart for shading correction.

[0048] In the above embodiment, either the blue wavelength range or the green wavelength range may be applied as the second wavelength range. When the blue wavelength range is applied as the second wavelength range, the correction unit 54 may use pixel values ​​of the blue component in the captured image as the second image data. When the green wavelength range is applied as the second wavelength range, the correction unit 54 may use pixel values ​​of the green component in the captured image as the second image data.

[0049] Furthermore, in the above embodiment, a case where pressure is applied as the energy applied to the object is described, but the disclosed technology is not limited to this aspect. For example, a configuration in which heat or ultraviolet light is applied as the energy applied to the object may be used. When heat is applied as the energy applied to the object, Thermoscale (product name) (manufactured by Fujifilm Corporation), which changes color depending on the amount of heat, can be used as the color-changing member 90. When ultraviolet light is applied as the energy applied to the object, UVScale (product name) (manufactured by Fujifilm Corporation), which changes color depending on the amount of ultraviolet light, can be used as the color-changing member 90.

[0050] Furthermore, in the above embodiment, for example, the following various processors can be used as the hardware structure of a processing unit that executes various processes, such as each functional unit of the image processing device 10. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits, such as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing specific processes.

[0051] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0052] Examples of configuring multiple processing units with a single processor include: first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server computers, and this processor functions as multiple processing units; second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs); in this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0053] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0054] In the above embodiment, the image processing program 30 is pre-stored (installed) in the storage unit 22, but this is not limiting. The image processing program 30 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The image processing program 30 may also be downloaded from an external device via a network.

[0055] The disclosure of Japanese Patent Application No. 2023-144649, filed on September 6, 2023, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. An image processing device having at least one processor, wherein the processor uses first image data in a first wavelength range in which the amount of change in spectral reflectance in response to the amount of energy in a captured image obtained by photographing a color-producing member that exhibits color with a density distribution in response to an applied amount of energy falls within a certain range, and performs shading correction on second image data in a second wavelength range different from the first wavelength range in which the spectral reflectance changes in response to the amount of energy in the captured image.

2. The image processing device according to claim 1, wherein the processor derives a distribution of the amount of energy applied to the color-producing member using the second image data after the shading correction has been performed.

3. An image processing device as described in claim 1 or claim 2, wherein the first wavelength range is a portion of the red wavelength range on the long wavelength side, and the first image data is pixel values ​​of the red component in the captured image.

4. The image processing device according to claim 3, wherein the first wavelength range is a wavelength range of 640 nm or more and less than 700 nm.

5. An image processing device as described in claim 1 or claim 2, wherein the second wavelength range is at least one of a blue wavelength range and a green wavelength range, and the second image data is at least one of pixel values ​​of a blue component and pixel values ​​of a green component in the captured image.

6. An image processing device according to claim 1 or 2, wherein the color-producing member is a member that produces a color in the first wavelength range such that the amount of change in spectral reflectance falls within the certain range regardless of the amount of energy.

7. An image processing device as described in claim 1 or claim 2, wherein the captured image is an image captured through a filter that passes light in the first wavelength range and the second wavelength range and blocks light in a wavelength range between the first wavelength range and the second wavelength range.

8. An image processing method in which a processor included in an image processing device performs a process of performing shading correction on second image data of a second wavelength range different from the first wavelength range in which the spectral reflectance changes depending on the amount of energy in a captured image obtained by photographing a color-producing member that exhibits color with a density distribution depending on the amount of energy applied thereto, using first image data of a first wavelength range in which the amount of change in spectral reflectance depending on the amount of energy in the captured image falls within a certain range.

9. An image processing program for causing a processor of an image processing device to execute a process of performing shading correction on second image data of a second wavelength range different from the first wavelength range in which the spectral reflectance changes according to the amount of energy in a captured image obtained by photographing a color-producing member that exhibits color with a density distribution according to the amount of applied energy, using first image data of a first wavelength range in which the amount of change in spectral reflectance according to the amount of energy in the captured image falls within a certain range.

Images