Image processing device, image processing method, and image processing program

Abstract

This image processing device: acquires information corresponding to a spatial frequency in a captured image acquired by capturing an image of a coloring member that develops a color that has a density distribution corresponding to an applied energy amount; and derives, on the basis of the acquired information, an energy amount applied to the coloring member for an area, in the captured image, where the density is below a first threshold value.

Description

Image processing device, image processing method, and image processing program

[0001] The present disclosure relates to an image processing device, an image processing method, and an image processing program.

[0002] A conventional technique for measuring the amount of energy is known, which uses a color-changing member that changes color depending on the amount of energy applied to it. For example, one such color-changing member is Prescale (registered trademark) (manufactured by Fujifilm Corporation), which changes color depending on the pressure applied to it.

[0003] Japanese Patent Application Laid-Open No. 2008-232665 discloses a technique for converting density values ​​obtained by reading a pressure measurement film in which a pressure-applied portion develops color with a scanner into pressure values ​​using a conversion table.

[0004] When using a color-forming material that changes color depending on the amount of energy applied, depending on the amount of energy applied, it may not be possible to accurately derive the amount of energy based on the density in the captured image obtained by photographing the color-forming material.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide an image processing device, an image processing method, and an image processing program that can accurately derive the amount of energy applied to a color-forming member that develops color according to the amount of energy applied.

[0006] The image processing device of the first aspect is an image processing device having at least one processor, which acquires information corresponding to the spatial frequency in a captured image obtained by photographing a color-producing member that produces color with a density distribution according to the amount of energy applied, and derives the amount of energy applied to the color-producing member based on the acquired information for parts of the captured image where the density is less than a first threshold value.

[0007] In a second aspect of the image processing device, in the image processing device of the first aspect, the processor derives the amount of energy applied to the color-forming member based on the density for parts of the captured image where the density is equal to or greater than a second threshold and is equal to or greater than a first threshold.

[0008] The image processing device of the third aspect is the image processing device of the second aspect, wherein the second threshold is greater than the first threshold, and for portions of the captured image where the density is equal to or greater than the first threshold and less than the second threshold, the amount of energy applied to the color-forming member is derived by combining a first amount of energy derived based on the acquired information and a second amount of energy derived based on the density.

[0009] The image processing device of the fourth aspect is the image processing device of the third aspect, and for parts of the captured image where the density is equal to or greater than a first threshold and less than a second threshold, the first energy amount is weighted by a weighting coefficient that decreases as the density increases, and the second energy amount is weighted by a weighting coefficient that increases as the density increases, thereby combining the first energy amount and the second energy amount.

[0010] An image processing device of a fifth aspect is the image processing device of any one of the first to fourth aspects, wherein the information corresponding to the spatial frequency in the captured image is a component of the captured image whose spatial frequency is equal to or greater than a certain value.

[0011] The sixth aspect of the image processing method involves acquiring information corresponding to the spatial frequency in a captured image obtained by photographing a color-producing member that produces color with a density distribution according to the amount of energy applied, and having a processor provided in the image processing device execute a process to derive the amount of energy applied to the color-producing member based on the acquired information for portions of the captured image where the density is less than a first threshold value.

[0012] The seventh aspect of the image processing program is for causing a processor provided in an image processing device to acquire information corresponding to the spatial frequency in a captured image obtained by photographing a color-producing member that produces color with a density distribution according to the amount of energy applied, and for parts of the captured image where the density is less than a first threshold, to execute a process of deriving the amount of energy applied to the color-producing member based on the acquired information.

[0013] According to the present disclosure, it is possible to accurately derive the amount of energy applied to a color-forming member that develops color in accordance with the amount of energy applied.

[0014] FIG. 1 is a block diagram showing an example of a schematic configuration of a pressure measurement system; FIG. 2 is a diagram showing an example of a color-forming member; FIG. 3 is a block diagram showing an example of a hardware configuration of an image processing device; FIG. 4 is a graph showing an example of a relationship between applied pressure and density of a captured image; FIG. 5 is a diagram showing an example of weighting coefficient data; FIG. 6 is a block diagram showing an example of a functional configuration of an image processing device; and FIG. 7 is a flowchart showing an example of pressure measurement processing.

[0015] Hereinafter, an example of an embodiment of the technology of the present disclosure will be described in detail with reference to the drawings. In this embodiment, an example in which pressure is applied as energy to an object will be described. Examples of objects to be pressurized include plate-shaped metals and semiconductor wafers.

[0016] First, the configuration of a pressure measurement system 1 according to this embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the pressure measurement system 1 includes an image processing device 10. Examples of the image processing device 10 include a portable computer such as a smartphone or a tablet terminal. Note that the image processing device 10 may also be a stationary computer.

[0017] As shown in Fig. 2, the pressure measurement system 1 measures the amount of energy using a color-forming member 90 that, when energy (pressure in this embodiment) is applied, develops a color with a density distribution corresponding to the amount of energy applied. In the example of Fig. 2, the hatched areas indicate the colored areas. Specifically, the image processing device 10 uses a camera 40 (see Fig. 3) to capture an image of the color-forming member 90 in a state in which it has developed color due to the application of energy, and derives the amount of energy applied to the color-forming member 90 from the captured image (hereinafter referred to as the "captured image").

[0018] An example of the color-forming member 90 is Prescale (registered trademark) (manufactured by Fujifilm Corporation), which produces a color density corresponding to the applied pressure. The prescale is a sheet-like support coated with a color former containing microcapsules containing a colorless dye and a color developer. When pressure is applied to the prescale, the microcapsules are destroyed, and the colorless dye is adsorbed to the color developer, producing color. Furthermore, because the color former contains multiple types of microcapsules with different sizes and strengths, the amount of microcapsules destroyed varies depending on the applied pressure, resulting in different color density. Therefore, by observing the color density, the magnitude and pressure distribution of the pressure applied to the prescale can be measured.

[0019] Next, the hardware configuration of the image processing device 10 according to this embodiment will be described with reference to Fig. 3. As shown in Fig. 3, the image processing device 10 includes a CPU (Central Processing Unit) 20, a memory 21 as a temporary storage area, and a non-volatile storage unit 22. The image processing device 10 also includes a display 23 such as a liquid crystal display, an input device 24 such as a touch panel, a network I / F (Interface) 25 connected to a network, and a camera 40. The CPU 20, the memory 21, the storage unit 22, the display 23, the input device 24, the network I / F 25, and the camera 40 are connected to a bus 27. The CPU 20 is an example of a processor according to the disclosed technology.

[0020] The storage unit 22 is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or the like. The storage unit 22 serves as a storage medium and stores an image processing program 30. The CPU 20 reads the image processing program 30 from the storage unit 22, loads it into the memory 21, and executes the loaded image processing program 30.

[0021] The storage unit 22 also stores first characteristic data 32, second characteristic data 34, and weighting coefficient data 36. The first characteristic data 32, second characteristic data 34, and weighting coefficient data 36 will be described in detail later.

[0022] The camera 40 includes an image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The camera 40 captures an image of the color-producing member 90 and outputs the captured image to the CPU 20. A scanner may be used instead of the camera 40. In other words, capturing an image here also includes reading with a scanner.

[0023] Next, referring to FIG. 4 , the relationship between the pressure applied to the color-forming member 90 according to this embodiment and the density of the color-forming portion in a captured image obtained by capturing a photograph of the color-forming member 90 that has developed color in response to the pressure will be described. The vertical axis of FIG. 4 represents pressure, and the horizontal axis represents density. As shown in FIG. 4 , there is a relationship between density and pressure. However, in areas below density d0, the increase in pressure relative to the increase in density is more rapid than in areas above density d1. Therefore, when converting the density of a captured image to a pressure value, if the density contains an error, the error in the pressure value corresponding to the density error will be larger in areas below density d0 than in areas above density d1. In this case, the pressure applied to the color-forming member 90 cannot be accurately derived.

[0024] Furthermore, the color-forming member 90 according to this embodiment has a characteristic that, in areas where the density is less than d0, the smaller the applied pressure, the higher the granularity of the captured image. Granularity refers to the degree of roughness of the image. The image processing device 10 according to this embodiment has a function of deriving the pressure value applied to the color-forming member 90 using the first characteristic data 32, the second characteristic data 34, and the weighting coefficient data 36.

[0025] The first characteristic data 32 is data that defines a predetermined relationship between the amount of energy applied to the color-producing member 90 and the granularity of an image obtained by photographing the color-producing member 90. As the amount of energy, a physical quantity corresponding to energy that can be measured using the color-producing member 90, such as a pressure value, can be appropriately applied.

[0026] The second characteristic data 34 is data that defines a predetermined relationship between the amount of energy applied to the color-producing member 90 and the density of the color-producing member 90 in a captured image obtained by capturing an image of the color-producing member 90. As the amount of energy, a physical quantity corresponding to energy that can be measured using the color-producing member 90, such as a pressure value, can be appropriately applied.

[0027] The weighting coefficient data 36 defines weighting coefficients used to weight the first characteristic data 32 and the second characteristic data 34. Fig. 5 shows an example of the weighting coefficient data 36. The vertical axis of Fig. 5 represents the weighting coefficient, and the horizontal axis represents the concentration. In Fig. 5, the dashed dotted line represents the weighting coefficient for the first characteristic data 32 (hereinafter referred to as the "first weighting coefficient"), and the solid line represents the weighting coefficient for the second characteristic data 34 (hereinafter referred to as the "second weighting coefficient").

[0028] The first weighting coefficient is 1.0 when the concentration is less than d0, 0 when the concentration is d1 or greater, and decreases as the concentration increases from d0 to d1. The second weighting coefficient is 0 when the concentration is less than d0, 1 when the concentration is d1 or greater, and increases as the concentration increases from d0 to d1. That is, only the first characteristic data 32 is used when the concentration is less than d0, and only the second characteristic data 34 is used when the concentration is d1 or greater.

[0029] Next, the functional configuration of the image processing device 10 according to this embodiment will be described with reference to Fig. 6. As shown in Fig. 6, the image processing device 10 includes an imaging control unit 50, an acquisition unit 52, and a derivation unit 54. The CPU 20 executes the image processing program 30 to function as the imaging control unit 50, the acquisition unit 52, and the derivation unit 54.

[0030] The user inputs an instruction to photograph the color-forming member 90 via the input device 24. When the user inputs the instruction to photograph, the photographing control unit 50 controls the camera 40 to photograph an image of the color-forming member 90.

[0031] The acquisition unit 52 acquires a captured image obtained by capturing an image of the color-producing member 90 under the control of the imaging control unit 50 .

[0032] The derivation unit 54 acquires information corresponding to the spatial frequency of the captured image (hereinafter referred to as "spatial frequency information") from the captured image acquired by the acquisition unit 52. Specifically, the derivation unit 54 acquires the spatial frequency information by applying a filter to the captured image that passes spatial frequency components above a certain level. That is, the spatial frequency information according to this embodiment is the spatial frequency components in the captured image that are above a certain level. In this case, the lower limit of the spatial frequency to be extracted is determined in advance through experiments or the like. The spatial frequency information acquired in this manner represents the granularity of the image.

[0033] For example, in the present embodiment, the detection target for granular structures in a captured image has a diameter of 0.1 mm or more and less than 0.5 mm. In this case, the derivation unit 54 acquires spatial frequency information by applying a filter capable of extracting spatial frequency components higher than 0.5 [cycle / mm] to the captured image.

[0034] For example, if the resolution of the captured image is 300 dpi (dots per inch), the derivation unit 54 can use a Laplacian filter with a size of 3 × 3 pixels. The derivation unit 54 may also use a spatial filter that reduces high-frequency components in order to distinguish between noise contained in the captured image and the granular structure to be detected. For example, the derivation unit 54 can achieve both noise reduction and granular structure extraction by calculating the difference between the result of applying an average filter with a size of 7 × 7 pixels and the result of applying an average filter with a size of 3 × 3 pixels to a captured image with a resolution of 300 dpi.

[0035] Furthermore, for example, the derivation unit 54 may apply an edge detection filter such as a Prewitt filter to the captured image. In this case, the derivation unit 54 can extract the granular structure by calculating the sum of the absolute values ​​of the outputs of the Prewitt filters in the horizontal and vertical directions of the captured image. The Prewitt filter may have a size of, for example, 5x5 pixels or more and 9x9 pixels or less.

[0036] For a portion of the captured image where the density is less than the first threshold, the derivation unit 54 derives the pressure value applied to the color-forming member 90 based on the acquired spatial frequency information. The first threshold is, for example, the density d0 described above. Specifically, for a pixel in the captured image where the density is less than the first threshold, the derivation unit 54 derives the pressure value by converting the spatial frequency information into a pressure value using the first characteristic data 32.

[0037] Furthermore, for portions of the captured image where the density is equal to or greater than a second threshold value that is greater than the density less than the first threshold value, the derivation unit 54 derives the pressure value applied to the color-forming member 90 based on the density. The second threshold value is, for example, the density d1 described above. Specifically, for pixels in the captured image where the density is equal to or greater than the second threshold value, the derivation unit 54 derives the pressure value by converting the density into a pressure value using the second characteristic data 34. Note that the second threshold value may be equal to the first threshold value.

[0038] Furthermore, for portions of the captured image where the density is equal to or greater than the first threshold and less than the second threshold, the derivation unit 54 combines a first pressure value derived based on spatial frequency information and a second pressure value derived based on density to derive the pressure value applied to the color-forming member 90. Specifically, the derivation unit 54 converts spatial frequency information into a pressure value using the first characteristic data 32 to derive the first pressure value. The derivation unit 54 also converts density into a pressure value using the second characteristic data 34 to derive the second pressure value. The derivation unit 54 then weights the first pressure value with a first weighting coefficient, which decreases as the density increases, and weights the second pressure value with a second weighting coefficient, which increases as the density increases, using the weighting coefficient data 36. In this way, the derivation unit 54 combines the first pressure value and the second pressure value.

[0039] Next, the operation of the image processing device 10 according to this embodiment will be described with reference to Fig. 7. The CPU 20 executes the image processing program 30, thereby executing the pressure measurement process shown in Fig. 7. The pressure measurement process shown in Fig. 7 is executed, for example, when a user inputs an image capture instruction via the input device 24.

[0040] 7, the imaging control unit 50 controls the camera 40 to capture an image of the color-forming member 90. In step S12, the acquisition unit 52 acquires a captured image obtained by capturing an image of the color-forming member 90 under the control of step S10.

[0041] In step S14, the derivation unit 54 acquires spatial frequency information from the captured image acquired in step S12, as described above. In step S16, the derivation unit 54 derives the pressure value applied to the color-forming member 90 based on the spatial frequency information for portions of the captured image where the density is less than the first threshold, as described above. Furthermore, the derivation unit 54 derives the pressure value applied to the color-forming member 90 based on the density for portions of the captured image where the density is equal to or greater than the second threshold, as described above. Furthermore, the derivation unit 54 derives the pressure value applied to the color-forming member 90 by combining the first pressure value derived based on the spatial frequency information and the second pressure value derived based on the density for portions of the captured image where the density is equal to or greater than the first threshold and less than the second threshold, as described above. When the processing of step S16 ends, the pressure measurement processing ends.

[0042] As described above, according to this embodiment, the amount of energy applied to the color-forming member 90 can be derived with high accuracy.

[0043] In the above embodiment, the camera 40 may capture the coloring member 90 and the patch image in a state in which a black patch image is arranged outside the coloring member 90. In this case, the shooting control unit 50 may determine the focus state based on the contrast value of the patch image portion in the captured image captured by the camera 40. For example, if the degree of blur of the captured image is within a correctable range, the shooting control unit 50 may perform correction on the captured image to reduce the degree of blur. Furthermore, for example, if the degree of blur of the captured image is outside the correctable range, the shooting control unit 50 may notify the user that they should retake the image by displaying a message on the display 23 prompting them to take the image again.

[0044] Furthermore, in the above embodiment, a case where pressure is applied as the energy applied to the object is described, but the disclosed technology is not limited to this aspect. For example, a configuration in which heat or ultraviolet light is applied as the energy applied to the object may be used. When heat is applied as the energy applied to the object, Thermoscale (product name) (manufactured by Fujifilm Corporation), which changes color depending on the amount of heat, can be used as the color-changing member 90. When ultraviolet light is applied as the energy applied to the object, UVScale (product name) (manufactured by Fujifilm Corporation), which changes color depending on the amount of ultraviolet light, can be used as the color-changing member 90.

[0045] Furthermore, in the above embodiment, for example, the following various processors can be used as the hardware structure of a processing unit that executes various processes, such as each functional unit of the image processing device 10. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits, such as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing specific processes.

[0046] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0047] Examples of configuring multiple processing units with a single processor include: first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server computers, and this processor functions as multiple processing units; second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs); in this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0048] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0049] In the above embodiment, the image processing program 30 is pre-stored (installed) in the storage unit 22, but this is not limiting. The image processing program 30 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The image processing program 30 may also be downloaded from an external device via a network.

[0050] The disclosure of Japanese Patent Application No. 2023-144648, filed on September 6, 2023, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. An image processing device having at least one processor, wherein the processor acquires information corresponding to the spatial frequency in a captured image obtained by photographing a color-producing member that exhibits a color with a density distribution according to an amount of energy applied thereto, and derives the amount of energy applied to the color-producing member based on the acquired information for portions of the captured image where the density is less than a first threshold value.

2. The image processing device according to claim 1, wherein the processor derives the amount of energy applied to the color-producing member based on the density for portions of the captured image where the density is equal to or greater than a second threshold value and is equal to or greater than the first threshold value.

3. The image processing device of claim 2, wherein the second threshold is a value greater than the first threshold, and for portions of the captured image where the density is equal to or greater than the first threshold and less than the second threshold, the amount of energy applied to the color-forming member is derived by combining a first amount of energy derived based on the acquired information and a second amount of energy derived based on the density.

4. The image processing device of claim 3, wherein for portions of the captured image where the density is equal to or greater than the first threshold value and less than the second threshold value, the first energy amount is weighted by a weighting coefficient that decreases as the density increases, and the second energy amount is weighted by a weighting coefficient that increases as the density increases, thereby combining the first energy amount and the second energy amount.

5. An image processing device according to any one of claims 1 to 4, wherein the information corresponding to the spatial frequency in the captured image is a component in the captured image whose spatial frequency is equal to or higher than a certain value.

6. An image processing method in which a processor included in an image processing device executes the following process: acquiring information corresponding to the spatial frequency in a captured image obtained by photographing a color-producing member that exhibits a color with a density distribution according to the amount of energy applied thereto; and deriving the amount of energy applied to the color-producing member based on the acquired information for portions of the captured image where the density is less than a first threshold value.

7. An image processing program for causing a processor of an image processing device to execute a process of acquiring information corresponding to the spatial frequency in a captured image obtained by photographing a color-producing member that exhibits a color with a density distribution according to the amount of energy applied thereto, and deriving the amount of energy applied to the color-producing member based on the acquired information for portions of the captured image where the density is less than a first threshold value.

Images