Image processing method, image processing program, image processing device, image display system, image printing system, image analysis system, and method for manufacturing printed matter

Abstract

To provide an image processing method configured to reduce a processing time in applying a secondary B-spline base function to each pixel in a z direction of an xyz orthogonal coordinate system, an image processing program, an image processing apparatus, an image display system, an image printing system, an image analysis system, and a method of manufacturing a printed matter.SOLUTION: An image processing method includes: setting each reference point in an xyz orthogonal coordinate system of pixel values of pixels, to each of a plurality of pixels constituting a two-dimensional image in an xy orthogonal coordinate system (S10); applying Gaussian filtering to the xy orthogonal coordinate system, to each of a plurality of aggregates formed of some of the plurality of pixels, around a target pixel located at the center of each aggregate (S20); applying a secondary B-spline base function along a z axis of the xyz orthogonal coordinate system to each pixel, using a statistic specified from the two-dimensional image as a reference width (S40); and replacing a pixel value of each of the plurality of target pixels by each maximum value of a multiplication value distribution to which the secondary B-spline base function has been applied (S50).SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an image processing method, an image processing program, an image processing device, an image display system, an image printing system, an image analysis system, and a method for producing printed matter. [Background technology]

[0002] It is known that images captured using an imaging device such as a digital camera contain noise. Images captured in dark places are particularly prone to noise. Figures 1A to 1C show images containing noise, with Figure 1A being an example of an original image, Figure 1B being an example of an image containing salt-and-pepper noise, and Figure 1C being an example of an image containing Gaussian noise. The more noise there is in an image, the lower the image quality and recognition rate. Taking a picture with a digital camera equipped with a high-performance, large-capacity image sensor can reduce the rate of noise in the image, but it is difficult to reduce the rate of noise to 0%. Therefore, in order to improve the image quality and recognition rate of images taken with an imaging device such as a digital camera, it is necessary to remove or reduce noise.

[0003] As a general method for removing or reducing noise, an image processing method has been proposed (see Non-Patent Document 1) in which a captured image is blurred using a Gaussian filter as a low-pass filter. While a Gaussian filter can reduce noise from the original image, it also blurs edges. When edges are blurred, the perceived resolution decreases, and so does the image quality. In other words, the method proposed in Non-Patent Document 1 has difficulty achieving both noise reduction and edge preservation. In recent years, an image processing method using a bilateral filter, which is a Gaussian filter with weights of normal distribution (see Non-Patent Document 2), has been used. While a bilateral filter excels in preserving edges relative to the original image, it is less effective in reducing noise. Furthermore, a bilateral filter can reduce noise by adjusting its parameters, but at the same time blurs edges. In other words, the method proposed in Non-Patent Document 2 has difficulty achieving both noise reduction and edge preservation. In addition to these filters, there are also adaptive bilateral filters (see Non-Patent Document 3) and non-local means filters (see Non-Patent Document 4), but none of these filters can simultaneously remove noise and preserve edges.

[0004] The following image processing method has been proposed as a method for achieving both noise reduction and edge preservation, which could not be achieved by the methods using the filters in Non-Patent Documents 1 to 4. This image processing method (see Non-Patent Document 5) uses a fast M-estimation Gaussian filter for images (hereinafter referred to as FMGFI), which combines a Gaussian filter and a fast M-estimation method. This image processing method makes it possible to remove spike noise and preserve steps due to the characteristics of the fast M-estimation method, and therefore makes it possible to remove noise and preserve edges at the same time. 2A to 2C are images output by processing the noisy image of FIG. 1B using the image processing method of Non-Patent Document 3, with FIG. 2A being an example of an image output with the basic width BW set to 1, FIG. 2B being an example of an image output with the basic width BW set to 21, and FIG. 2C being an example of an image output with the basic width BW set to 51. It can be seen that in the case of FIG. 2B (basic width BW=21), it is possible to achieve both noise removal and edge preservation, whereas in the case of FIG. 2A (basic width BW=1), it is possible to preserve edges but not remove noise, and in the case of FIG. 2C (basic width BW=51), it is possible to remove noise but not preserve edges. [Prior art documents] [Non-patent literature]

[0005] [Non-licensed Document 1] D.Chowdhury, SKDas, S.Nandy, A.Chakraborty, R.Goswami, A.Chakraborty, 2019 International Conference on Opto-Electronics and Applied Optics (Optronix) "An Atomic Technique For Removal Of Gaussian Noise From A Noisy Gray Scale Image Using LowPass-Convoluted Gaussian Filter", published on March 18, 2019, pp.1-6 [Non-licensed Document 2] S.Parism, P.Kornprobst, J.Tumblin, F.Durand, Bilateral Filtering, "Theory and Applications Foundations and Trends in Computer Graphics and Vision", Vol.4, No.4, pp.1-74, 2009. [Non-licensed Document 3] X.Changzhen, C.Licong, P.Yigui, 2010 International Conference on Measuring Technology and Mechatronics Automation "An Adaptive Bilateral Filtering Algorithm and Its Application in Edge Detection", pp.440-443, 2010. [Non-licensed Document 4] A.Buades, B.Coll, JMMorel, A non-local algorithm for image denoising, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), Vol. 2, pp. 60-65, 2005. [Non-Patent Document 5] Yuki Kondo, Ichiro Yoshida, Munetoshi Numata, Yamato Koshimizu, Journal of the Japan Society for Precision Engineering (2020, Vol. 86, No. 12), "Study on Edge-Preserving Noise Reduction Filter Using Fast M-Estimation Method", Japan Society for Precision Engineering, December 1, 2020, pp. 1034-1041 Summary of the Invention [Problem to be solved by the invention]

[0006] As mentioned above, the image processing method using the fast M-estimation Gaussian filter for images proposed in Non-Patent Document 5 can achieve both noise removal and edge preservation. To achieve this, the basis width BW of the quadratic B-spline basis function must be set to an appropriate value, as shown in Figure 2B (basis width BW = 21). However, Non-Patent Document 3 does not propose a specific method for setting the basic width BW of the quadratic B-spline basis function to an appropriate value, so the basic width BW had to be set manually (by an operator observing and determining while changing the basic width BW).

[0007] One of the objects of the present invention is to provide an image processing method that requires less processing time than a method in which the base width is manually set when applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz Cartesian coordinate system. [Means for solving the problem]

[0008] The image processing method of the first aspect includes: a first step of setting reference points in an xyz orthogonal coordinate system for pixel values ​​of each pixel in each of a plurality of pixels that form a two-dimensional image in an xy orthogonal coordinate system; a second step of applying Gaussian filtering to each of a plurality of groups of pixels formed from a portion of the plurality of pixels in an xy orthogonal coordinate system, with a pixel of interest located at the center of each group as a center; a third step of applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz Cartesian coordinate system using a statistical value determined from the two-dimensional image as a basic width; The pixel values ​​of each of the plurality of target pixels are 、 Multiplication value distribution to which the quadratic B-spline basis function is applied The pixel value at which of each The fourth step is to replace Includes:

[0009] The image processing method of the second aspect includes: a first step of setting reference points in an xyz orthogonal coordinate system for pixel values ​​of each pixel in each of a plurality of pixels that form a two-dimensional image in an xy orthogonal coordinate system; a second step of applying Gaussian filtering to each of a plurality of groups of pixels formed from a portion of the plurality of pixels in an xy orthogonal coordinate system, with a pixel of interest located at the center of each group as a center; a third step of applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz orthogonal coordinate system, using a statistical value of each aggregate identified from the pixel of interest and a plurality of peripheral pixels adjacent to the pixel of interest as each basic width; The pixel values ​​of each of the plurality of target pixels are 、 Multiplication value distribution to which the quadratic B-spline basis function is applied The pixel value at which of each The fourth step is to replace Includes:

[0010] The image processing method of the third aspect includes: An image processing method according to a second aspect, comprising: The statistical value is set to the statistical value of each group specified by the pixel of interest and the plurality of peripheral pixels.

[0011] The image processing program of the first aspect comprises: On the computer, a first function for setting, for each of a plurality of pixels constituting a two-dimensional image in an xy orthogonal coordinate system, reference points in an xyz orthogonal coordinate system for pixel values ​​of each pixel; a second function of applying Gaussian filtering to each of a plurality of groups of pixels formed by a portion of the plurality of pixels in an xy orthogonal coordinate system with a pixel of interest located at the center of each group as a center; a third function for applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz Cartesian coordinate system, using a statistical value determined from the two-dimensional image as a base width; and The pixel values ​​of each of the plurality of target pixels are 、 Multiplication value distribution to which the quadratic B-spline basis function is applied The pixel value at which of each The fourth function, which replaces Execute the following.

[0012] The image processing program of the second aspect is On the computer, a first function for setting, for each of a plurality of pixels constituting a two-dimensional image in an xy orthogonal coordinate system, reference points in an xyz orthogonal coordinate system for pixel values ​​of each pixel; a second function of applying Gaussian filtering to each of a plurality of groups of pixels formed by a portion of the plurality of pixels in an xy orthogonal coordinate system with a pixel of interest located at the center of each group as a center; a third function for applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz Cartesian coordinate system, using a statistical value of each cluster identified from the pixel of interest and a plurality of surrounding pixels adjacent to the pixel of interest as each basic width; The pixel values ​​of each of the plurality of target pixels are 、 Multiplication value distribution to which the quadratic B-spline basis function is applied The pixel value at which of each The fourth function, which replaces Execute the following.

[0013] The image processing program of the third aspect is An image processing program according to a second aspect, The statistical value is set to the standard deviation of each cluster identified from the pixel of interest and the plurality of surrounding pixels.

[0014] An image processing device according to one aspect includes: a storage unit that stores an image processing program according to any one of the first to third aspects; an execution unit configured to be able to communicate with the storage unit and to execute the first function, the second function, the third function, and the fourth function in accordance with the image processing program; Equipped with.

[0015] An image display system according to one embodiment includes: an image processing device according to an embodiment; a display device that displays an output image that is output as a two-dimensional image by the execution unit executing the first function, the second function, the third function, and the fourth function; and Equipped with.

[0016] An image printing system according to one embodiment includes: an image processing device according to an embodiment; a printing device that prints, on a medium, an output image that is output by the execution unit executing the first function, the second function, the third function, and the fourth function on a two-dimensional image; Equipped with.

[0017] An image analysis system according to one embodiment includes: an image processing device according to an embodiment; an imaging device that captures an image of an object, the imaging device being configured to be able to communicate with the image processing device and transmitting a captured two-dimensional image of the object to the image processing device; an analysis device that analyzes a shape of the object using an output image output by the execution unit when the first function, the second function, the third function, and the fourth function are executed on a two-dimensional image of the object; Equipped with.

[0018] A method for producing a printed matter according to one embodiment includes the steps of: Using an image printing system of one embodiment, The output image is printed on a medium. [Effects of the Invention]

[0019] The image processing method of the first aspect requires a shorter processing time than a method of manually setting the base width when applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz Cartesian coordinate system.

[0020] The image processing method of the second aspect can perform optimal image processing for each of multiple pixels, compared to a method in which a statistical value identified from the entire two-dimensional image is used as a basic width and a quadratic B-spline basis function is applied to each pixel along the z-axis of an xyz Cartesian coordinate system.

[0021] The image processing method of the third aspect can perform optimal image processing for each of a plurality of pixels using simple statistical values.

[0022] The image processing program of the first aspect requires a shorter processing time than a method of manually setting the base width when applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz Cartesian coordinate system.

[0023] The image processing program of the second embodiment can perform optimal image processing for each of multiple pixels, compared to a method in which a statistical value identified from the entire two-dimensional image is used as a basic width and a quadratic B-spline basis function is applied to each pixel along the z-axis of an xyz Cartesian coordinate system.

[0024] The image processing program of the third aspect can perform optimal image processing for each of a plurality of pixels using simple statistical values.

[0025] The image processing device of one embodiment has a shorter processing time than an image processing device that manually sets the base width when applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz Cartesian coordinate system.

[0026] In one embodiment, the image display system takes less time from the start of image processing to the completion of display than an image display system in which the base width is manually set when applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz Cartesian coordinate system.

[0027] In one embodiment, the image printing system takes less time from the start of image processing to the end of printing than an image printing system that manually sets the base width when applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz Cartesian coordinate system.

[0028] In one embodiment, the image analysis system takes less time from the start of image processing to the end of image analysis than an image analysis system in which the base width is manually set when applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz Cartesian coordinate system.

[0029] One embodiment of the method for producing a printed material requires less time from the start of image processing to the end of printing than a method for printing an output image on a medium using an image printing system that manually sets the base width when applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz Cartesian coordinate system. [Brief explanation of the drawings]

[0030] [Figure 1A] An example of an original image (source image) taken using a digital camera is shown below. [Figure 1B] An example of an image containing salt and pepper noise (an image in which salt and pepper noise has been added to the image in FIG. 1A) is shown below. [Figure 1C] An example of an image containing Gaussian noise (an image in which Gaussian noise has been added to the image of FIG. 1A) is shown. [Figure 2A] An example of an image output by an image processing method using a fast M-estimation Gaussian filter for images, in which the basic width is set to BW=1, is shown. [Figure 2B] An example of an image output by an image processing method using a fast M-estimation Gaussian filter for images, in which the basic width is set to BW=21, is shown. [Figure 2C] An example of an image output by an image processing method using a fast M-estimation Gaussian filter for images, where the image is output with the basic width set to BW=51, is shown. [Figure 3] FIG. 1 is a schematic diagram of an image display system according to a first embodiment. [Figure 4] FIG. 2 is a diagram for explaining the characteristics of a quadratic B-spline basis function used in the abwFMGFI algorithm stored in the image processing program of the first embodiment. [Figure 5] FIG. 2 is a diagram illustrating an algorithm of abwFMGFI according to the first embodiment. [Figure 6A] FIG. 3 is a diagram simply illustrating the processing performed in the first step S10 of the first embodiment. [Figure 6B] 1B is a diagram for specifically explaining the processing performed in the first step S10 of the first embodiment, and is an example of a bird's-eye view obtained by expanding the original image of FIG. 1A by one dimension. [Figure 6C] 1B is a diagram for specifically explaining the processing performed in the first step S10 of the first embodiment, and is another example of a bird's-eye view obtained by expanding the original image of FIG. 1A by one dimension. FIG. [Figure 6D] FIG. 2 is a three-dimensional diagram illustrating the processing performed in the first step S10 of the first embodiment. [Figure 7A] FIG. 10 is a diagram simply illustrating the processing performed in the second step S20 of the first embodiment. [Figure 7B] 10 shows an example of the value of the Gaussian function in the second step S20 of the first embodiment. [Figure 7C] 10 shows a schematic diagram of when GF is applied to each pixel region (3×3) in the second step S20 of the first embodiment. [Figure 8] FIG. 10 is a schematic diagram for explaining the determination of the basic width BW performed in the third step S30 of the first embodiment as preprocessing for the fourth step S40, and the application of a quadratic B-spline basis function to a corresponding reference value in the fourth step S40 based on the basic width BW determined in the third step S30. [Figure 9]FIG. 10 is a schematic diagram for explaining application of a quadratic B-spline basis function to a reference value based on the basic width BW determined in the third step S30 in the fourth step S40 of the first embodiment. [Figure 10A] FIG. 10 is a schematic diagram for explaining calculation of an output value from a multiplication value distribution in the z direction after the processing of the first step S10 to the fourth step S40 in the fifth step S50 of the first embodiment. [Figure 10B] FIG. 10 is a schematic diagram for explaining the process of deriving an output value when the target pixel region does not contain noise, in the calculation of the output value performed in the fifth step S50 of the first embodiment. [Figure 10C] FIG. 10 is a schematic diagram for explaining the process of deriving an output value when a pixel of interest in a target pixel region is noise, in the calculation of the output value performed in fifth step S50 of the first embodiment. [Figure 10D] FIG. 10 is a schematic diagram for explaining the process of deriving an output value when noise is contained in peripheral pixels of a target pixel region in the calculation of the output value performed in fifth step S50 of the first embodiment. [Figure 10E] FIG. 10 is a schematic diagram for explaining the process of deriving an output value when the pixel of interest and surrounding pixels in the target pixel region are edges and the surrounding pixels contain noise, in the calculation of the output value performed in the fifth step S50 of the first embodiment. [Figure 11A] This figure shows 12 types of sample images used in an experiment to remove salt and pepper noise. [Figure 11B] FIG. 11 is a diagram showing the results of an experiment to remove salt-and-pepper noise in order to compare the first embodiment with a general filter, and shows the original image of the first sample image, an image with noise added, and images in which each filter has been applied to the image with noise added. [Figure 11C] FIG. 11 is a diagram showing the results of an experiment to remove salt-and-pepper noise to compare the first embodiment with a general filter, and shows the original image of the second sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 11D]FIG. 11 is a diagram showing the results of an experiment to remove salt-and-pepper noise in order to compare the first embodiment with a general filter, and shows the original image of the third sample image, an image with noise added, and images in which each filter has been applied to the image with noise added. [Figure 11E] FIG. 11 is a diagram showing the results of an experiment to remove salt-and-pepper noise in order to compare the first embodiment with a general filter, and shows the original image of the fourth sample image, an image with noise added, and images in which each filter has been applied to the image with noise added. [Figure 11F] FIG. 11 is a diagram showing the results of an experiment to remove salt-and-pepper noise in order to compare the first embodiment with a general filter, and shows the original image of the fifth sample image, an image with noise added, and images in which each filter has been applied to the image with noise added. [Figure 11G] FIG. 10 is a diagram showing the results of an experiment to remove salt-and-pepper noise in order to compare the first embodiment with a general filter, and shows the original image of the sixth sample image, an image with noise added, and images in which each filter has been applied to the image with noise added. [Figure 11H] FIG. 10 is a diagram showing the results of an experiment to remove salt-and-pepper noise in order to compare the first embodiment with a general filter, and shows the original image of the seventh sample image, an image with noise added, and images in which each filter has been applied to the image with noise added. [Figure 11I] FIG. 11 is a diagram showing the results of an experiment to remove salt-and-pepper noise in order to compare the first embodiment with a general filter, and shows the original image of the eighth sample image, an image with noise added, and images in which each filter has been applied to the image with noise added. [Figure 11J] FIG. 11 is a diagram showing the results of an experiment to remove salt-and-pepper noise in order to compare the first embodiment with a general filter, and shows the original image of the ninth sample image, an image with noise added, and images in which each filter has been applied to the image with noise added. [Figure 11K]FIG. 11 is a diagram showing the results of an experiment to remove salt-and-pepper noise in order to compare the first embodiment with a general filter, and shows the original image of the tenth sample image, an image with noise added, and images in which each filter has been applied to the image with noise added. [Figure 11L] FIG. 11 shows images showing the results of an experiment to remove salt-and-pepper noise in order to compare the first embodiment with a general filter, including the original image of the 11th sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 11M] FIG. 12 is a diagram showing the results of an experiment to remove salt-and-pepper noise in order to compare the first embodiment with a general filter, and shows the original image of the 12th sample image, an image with noise added, and images in which each filter has been applied to the image with noise added. [Figure 11N] 10 is a table showing a list of RMSE values ​​of each sample image of each filter in an experiment to remove salt and pepper noise for comparing the first embodiment with a general filter. [Figure 12A] FIG. 11 shows images showing the results of a salt-and-pepper noise removal experiment to compare the first embodiment with FMGFI, including the original image of the first sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 12B] FIG. 10 is a diagram showing the results of a salt-and-pepper noise removal experiment to compare the first embodiment with FMGFI, including the original image of the second sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 12C] FIG. 10 is a diagram showing the results of an experiment to remove salt-and-pepper noise to compare the first embodiment with FMGFI, including the original image of the third sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 12D] FIG. 10 is a diagram showing the results of an experiment to remove salt-and-pepper noise to compare the first embodiment with FMGFI, showing the original image of the fourth sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 12E]FIG. 11 shows images showing the results of an experiment to remove salt-and-pepper noise to compare the first embodiment with FMGFI, including the original image of the fifth sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 12F] FIG. 10 is a diagram showing the results of an experiment to remove salt-and-pepper noise to compare the first embodiment with FMGFI, including the original image of the sixth sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 12G] FIG. 10 is a diagram showing the results of an experiment to remove salt-and-pepper noise to compare the first embodiment with FMGFI, including the original image of the seventh sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 12H] 10A and 10B are images showing the results of an experiment to remove salt-and-pepper noise to compare the first embodiment with FMGFI, and are diagrams showing the original image of the eighth sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 12I] FIG. 11 shows images showing the results of a salt-and-pepper noise removal experiment to compare the first embodiment with FMGFI, including the original image of the ninth sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 12J] FIG. 11 shows images showing the results of a salt-and-pepper noise removal experiment to compare the first embodiment with FMGFI, including the original image of the 10th sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 12K] FIG. 11 shows images showing the results of an experiment to remove salt-and-pepper noise to compare the first embodiment with FMGFI, including the original image of the 11th sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 12L] FIG. 11 shows images showing the results of an experiment to remove salt-and-pepper noise to compare the first embodiment with FMGFI, including the original image of the 12th sample image, an image with noise added, and images obtained by applying each filter to the image with noise added. [Figure 12M] 10 is a table showing a list of RMSE values ​​of each sample image of each filter in an experiment to remove salt and pepper noise for comparing the first embodiment with FMGFI. [Figure 13A] FIG. 10 is a diagram showing the results of an edge preservation experiment to compare the first embodiment with a general filter, and shows the original image of the first sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 13B] FIG. 11 shows images showing the results of an edge preservation experiment to compare the first embodiment with a general filter, including the original image of the second sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 13C] FIG. 11 shows images showing the results of an edge preservation experiment to compare the first embodiment with a general filter, including the original image of the third sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 13D] FIG. 11 shows images showing the results of an edge preservation experiment to compare the first embodiment with a general filter, including the original image of the fourth sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 13E] FIG. 11 shows images showing the results of an edge preservation experiment to compare the first embodiment with a general filter, including the original image of the fifth sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 13F]FIG. 11 shows images showing the results of an edge preservation experiment to compare the first embodiment with a general filter, including the original image of the sixth sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 13G] FIG. 11 shows images showing the results of an edge preservation experiment to compare the first embodiment with a general filter, including the original image of the seventh sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 13H] FIG. 11 shows images showing the results of an edge preservation experiment to compare the first embodiment with a general filter, including the original image of the eighth sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 13I] FIG. 11 shows images showing the results of an edge preservation experiment to compare the first embodiment with a general filter, including the original image of the ninth sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 13J] FIG. 11 shows images showing the results of an edge preservation experiment to compare the first embodiment with a general filter, including the original image of the 10th sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 13K] FIG. 11 shows images showing the results of an edge preservation experiment to compare the first embodiment with a general filter, including the original image of the 11th sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 13L]This is a diagram showing the results of an edge preservation experiment to compare the first embodiment with a general filter, and is a diagram showing the original image of the 12th sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 13M] 10 is a table showing a list of RMSE values ​​of each sample image of each filter in an edge preservation experiment for comparing the first embodiment with a general filter. [Figure 14A] This figure shows images showing the results of an edge preservation experiment to compare the first embodiment with FMGFI, including the original image of the first sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 14B] This figure shows images showing the results of an edge preservation experiment to compare the first embodiment with FMGFI, including the original image of the second sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 14C] This figure shows images showing the results of an edge preservation experiment to compare the first embodiment with FMGFI, including the original image of the third sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 14D] This figure shows images showing the results of an edge preservation experiment to compare the first embodiment with FMGFI, including the original image of the fourth sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 14E] This figure shows images showing the results of an edge preservation experiment to compare the first embodiment with FMGFI, including the original image of the fifth sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 14F] This figure shows images showing the results of an edge preservation experiment to compare the first embodiment with FMGFI, including the original image of the sixth sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 14G] This figure shows images showing the results of an edge preservation experiment to compare the first embodiment with FMGFI, including the original image of the seventh sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 14H] This figure shows images showing the results of an edge preservation experiment to compare the first embodiment with FMGFI, including the original image of the eighth sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 14I] This figure shows images showing the results of an edge preservation experiment to compare the first embodiment with FMGFI, including the original image of the 9th sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 14J] This figure shows images showing the results of an edge preservation experiment to compare the first embodiment with FMGFI, including the original image of the 10th sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 14K] This figure shows images showing the results of an edge preservation experiment to compare the first embodiment with FMGFI, including the original image of the 11th sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 14L]This figure shows images showing the results of an edge preservation experiment to compare the first embodiment with FMGFI, including the original image of the 12th sample image, an image with a grid (or lattice or edge) added, and images in which each filter has been applied to the image with the grid (or lattice or edge) added. [Figure 14M] 10 is a table showing a list of RMSE values ​​of each sample image of each filter in an edge preservation experiment for comparing the first embodiment with FMGFI. [Figure 15A] 10 is a table showing the specifications of a PC used in a measurement experiment of filter processing time for comparing the first embodiment with FMGFI. [Figure 15B] 10 is a table showing the results of a measurement experiment of filter processing time for comparing the first embodiment with FMGFI. [Figure 16] FIG. 10 is a schematic diagram of an image printing system according to a second embodiment. [Figure 17A] FIG. 10 is a schematic diagram of an image analysis system according to a third embodiment. [Figure 17B] FIG. 10 is a diagram showing an algorithm for image analysis according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0031] Overview Hereinafter, several embodiments and several modified examples thereof will be described. First, each of the embodiments described below will be described in the order in which they are described. Next, several modified examples will be described. Please note that in this specification, components having equivalent functions are denoted by the same or similar reference numerals in each drawing referred to in different embodiments, etc. ---------------------------------------------- Embodiment Specific application ---------------------------------------------- First embodiment: Image display system Second embodiment: Image printing system Third embodiment: Image analysis system ----------------------------------------------

[0032] First Embodiment The functions, configuration, and actions of the image display system 10 of the first embodiment, as well as the effects of the first embodiment, will be described below in the order of description with reference to the drawings.

[0033] <Functions, Configurations, and Actions of the Image Display System of the First Embodiment> 3 is a schematic diagram of an image display system 10 according to this embodiment. The image display system 10 includes a camera 20 (an example of an image capturing device), a display 30 (an example of a display device), and an image processing engine 40 (an example of an image processing device).

[0034] [Camera and Display] One example of camera 20 is a CCD camera. Camera 20 has a function of capturing an image of an object (for example, the image of a person shown in FIG. 1A) and a function of transmitting a two-dimensional image of the captured object or data thereof to image processing engine 40. Therefore, camera 20 is set to be able to communicate with image processing engine 40. The display 30 has a function of displaying a two-dimensional image (output image) that has been subjected to image processing by a calculation unit 42 (an example of a computer and an execution unit) that constitutes an image processing engine 40 (described later) on a two-dimensional image (original image) transmitted from the camera 20. Specific image processing by the image processing engine 40 will be described later.

[0035] [Image processing engine] 1, the image processing engine 40 has a calculation unit 42, an input unit 44, a storage unit 46, and an output unit 48. The image processing engine 40 has a function of applying image processing, which will be described later, to a two-dimensional image (original image) and outputting a two-dimensional image (processed image) in which noise has been removed from the original image. In this embodiment, the original image is, for example, a two-dimensional image captured by the camera 20, and the two-dimensional image in which noise has been removed by image processing performed by the image processing engine 40 is, for example, output to and displayed on the display 30.

[0036] The input unit 44 has a function of receiving an input signal from outside. In the case of this embodiment, as an example, the outside is the camera 20, and the input signal from the outside corresponds to data of the original image captured by the camera 20 (data of a two-dimensional image). The output unit 48 has a function of outputting to the outside the result of image processing applied by the calculation unit 42 to data input to the image processing engine 40. In the case of this embodiment, as an example, the outside is the display 30, and the result corresponds to data of an image (processed image) in which noise has been removed by applying image processing to the original image by the image processing engine 40. The calculation unit 42 is, for example, a CPU (Central Processing Unit). The calculation unit 42 has a function of performing image processing (a function of applying image processing to an input original image) and a function of performing overall control of the image processing engine 40. Here, solid arrows in Fig. 3 indicate the flow of data, and dashed arrows indicate the flow of control. In order for the calculation unit 42 to perform image processing, a program describing the procedure for that processing is required, and this program is stored in the storage unit 46. Here, the program used by the calculation unit 42 to perform image processing is referred to as an image processing program 46P (see FIG. 3).

[0037] [Image processing method: Image processing method using abwFMGFI] Next, the image processing method of this embodiment, i.e., the algorithm (processing flow) of the image processing program 46P stored in the storage unit 46, will be described with reference to the drawings. Here, the image processing method of this embodiment is an image processing method using FMGFI disclosed in Non-Patent Document 5, which is equipped with an automatic determination algorithm for basic width BW, which will be described later. In the following description, the image processing method of this embodiment will be referred to as abwFMGFI (where "abw" stands for "with an Automatic Determination Algorithm for Basic Width"). Below, we will explain the outline of the high-speed M-estimation method related to abwFMGFI and the purpose of inventing abwFMGFI, and then explain the algorithm of abwFMGFI stored in the image processing program 46P.

[0038] <Outline of the fast M-estimation method related to abwFMGFI and the purpose of inventing abwFMGFI> Fast M-estimation, one of the robust estimation methods, uses a quadratic B-spline basis function as a loss function to find the evaluation value that maximizes the sum of the loss function. Figure 4 illustrates the characteristics of quadratic B-spline basis functions. As shown in this figure, quadratic B-spline basis functions have the characteristic that the weighting function within the central base width BW coincides with a quadratic function, while the weighting function outside the base width BW converges to 0. In other words, if the data is noise-free, only the central base width BW of the quadratic B-spline basis function is used in the calculation. As a result, the quadratic B-spline basis function functions only as a quadratic function. Therefore, if the data is noise-free, the estimates from fast M-estimation will coincide with those from the least-squares method.

[0039] Next, we will explain the output value when image processing is performed by combining the fast M-estimation method and a Gaussian filter (hereinafter referred to as a GF). The GF can be considered as a weighted least-squares method with a weight value of 1. Therefore, when image processing is performed by combining the fast M-estimation method and a GF, the output value tends to match the output value of the GF if the image does not contain noise. On the other hand, when a pixel of interest (the pixel of interest) is noisy and its value significantly differs from that of the pixels surrounding the pixel of interest (the peripheral pixels), two multiplication value distributions are formed: one for the noise and one for the peripheral pixels. Since the sum of the multiplication values ​​for the peripheral pixels tends to be larger than the sum of the values ​​of the multiplication value distribution for the noise, the output value is calculated only from the peripheral pixels and is not or is less affected by noise. When the pixel of interest is an edge, two multiplication value distributions are formed adjacent to the edge according to the pixel value. In this case, due to the properties of the GF, the sum of the multiplication values ​​tends to be larger at the edge including the pixel of interest. Therefore, the output value is likely to be the pixel value on the edge side including the pixel of interest. As a result, image processing combining the fast M-estimation method and a GF is expected to preserve edges. In order to achieve both noise removal and edge preservation, the setting of the basic width BW is extremely important.

[0040] <abwFMGFI algorithm> 5 shows the algorithm of abwFMGFI stored in the image processing program 46P. The processing flow of abwFMGFI consists of the following five steps (first step S10 to fifth step S50) performed in the order listed. Note that the first step S10 and second step S20 correspond to the process for realizing FMGFI, which is the premise of abwFMGFI.

[0041] (1st step) Fig. 6A is a simplified diagram showing the processing performed in the first step S10 (an example of the first process). Figs. 6B to 6D are diagrams specifically explaining the processing performed in the first step S10. The first step S10 is a step of setting a reference point in a cell corresponding to the x coordinate, y coordinate, and pixel value z of the original image. In FMGFI, a filter is applied to the pixel value z direction in addition to the x and y directions of the image, so a 3D array is required for a 2D grayscale image, for example. In other words, a 1D expansion process is required to turn a 2D grayscale image into a 3D array. Here, Figures 6B and 6C are bird's-eye views obtained by expanding the original image of Figure 1A by one dimension, and Figure 6D is a three-dimensional view of the view of Figure 6A, showing reference points set in each cell corresponding to the x coordinate, y coordinate, and pixel value z of the original image. The first step S10 is one process of the filtering process performed by the calculation unit 42 (see FIG. 3) that constitutes the image processing engine 40, and in this embodiment, the function of performing the first step S10 by the calculation unit 42 is referred to as the first function.

[0042] (Second step) FIG. 7A is a simplified diagram illustrating the processing performed in the second step S20. The second step S20 (an example of the second process) is a step in which a GF is applied in the x and y directions, centered on the reference point set in the first step S10. In a typical GF, a value is output by convolving a normalized Gaussian function with a pixel value. However, in the FMGFI used in this step, the value of the Gaussian function before normalization is added to the value of the corresponding cell. Here, FIG. 7B shows examples of Gaussian function values ​​for each pixel region (3×3), (5×5), and (7×7) (examples of collections), and FIG. 7C shows a schematic diagram of applying GF to a pixel region (3×3) as an example. The second step S20 is one process of the filtering process performed by the calculation unit 42 (see FIG. 3), and in this embodiment, the function of the calculation unit 42 to perform the second step S20 is referred to as the second function.

[0043] (Third Step) The third step S30 is a step for determining a basic width BW, which is performed as preprocessing for the next step (fourth step S40), and FIG. 8 is a schematic diagram for explaining this. Here, the combination of the third step S30 and the fourth step S40, which will be described later, is an example of a third process. Here, the combination of the third step S30 and the fourth step S40 is one process of the filter processing performed by the calculation unit 42 (see FIG. 3), and in this embodiment, the function of the calculation unit 42 to perform this combination is referred to as the third function.

[0044] The purpose of this step is as follows: When FMGFI is applied to an image containing noise, the results of applying FMGFI vary greatly depending on the size of the basic width BW. The larger the basic width BW, the more likely it is that visual noise in the original image will be reduced and edges will be blurred. In contrast, the smaller the basic width BW, the more likely it is that pixel values ​​of the original image will be preserved, and therefore noise will not be removed. Due to the characteristics of applying FMGFI to such images containing noise, in order to achieve both noise removal and edge preservation in FMGFI, it is necessary to set the basic width BW to an optimal value. However, the conventional FMGFI disclosed in the aforementioned Non-Patent Document 5 does not present a specific method for setting the basic width BW to an optimal value. If a human (for example, a designer) were to determine the value of the basic width BW and then determine the basic width BW by a human judgment of the result, there would be a problem of increased processing time due to the human work time. Therefore, in this step, the optimum basic width BW is automatically determined.

[0045] Next, a method for automatically determining the optimum basic width BW, which has been devised by the inventors of the present application, will be described, taking into consideration the characteristics of applying FMGFI to an image containing the above-mentioned noise. The specific idea behind this method is that "when noise is contained within the filter width range in an image, the basic width BW should be automatically increased to remove the noise. Conversely, when noise is not contained within the filter width range, the basic width BW should be automatically decreased." The method for "automatically" determining the basic width BW uses statistical values ​​identified by the original image. In this embodiment, an example of the statistical value is the standard deviation. As shown in FIG. 8, the standard deviation is calculated from the pixel of interest and surrounding pixels within the filter width range. Since abwFMGFI calculates the standard deviation for each pixel, the basic width BW can be determined for each pixel. Furthermore, because quadratic B-spline basis functions are symmetrical (one side and the other side are symmetrical in the z direction with respect to a reference value), half the standard deviation is used as the basic width BW for one half of the pixel. In this case, the basic width BW is an integer, and if the value of half the standard deviation is a decimal, it is rounded up or down. The two middle diagrams in FIG. 8 show conceptual diagrams for calculating the basic width BW when noise is present and when it is not present within the filter width range. As shown in FIG. 8, in this step, the basic width BW is calculated for each pixel, allowing the basic width BW to be varied and automatically determined depending on the presence or absence of noise. In the example of Figure 8, the diagram on the left side of the middle row contains noise within the filter width range, so the basic width BW is large at 21 levels, while the diagram on the right side does not contain noise within the filter width range, so the basic width BW is small at 2 levels. As described above, as a result of performing this step, if noise is included within the filter width range, the basic width BW becomes larger, thereby improving denoising performance (noise removal performance). On the other hand, if noise is not included within the filter width range, the basic width BW becomes smaller, thereby improving the reproducibility of the original image. In other words, this step makes it possible to automatically determine the optimal basic width BW and automatically apply FMGFI to the original image.

[0046] (Step 4) FIG. 9 is a schematic diagram illustrating the application of quadratic B-spline basis functions to corresponding reference values ​​based on the basic width BW determined in the third step S30. The fourth step S40 is a step of applying the quadratic B-spline basis functions along the z-axis based on the basic width BW determined in the third step S30. Here, the application of the quadratic B-spline basis functions is realized by convolving a box linear filter three times, for example. The weighting function value of the box linear filter is 1, and the width is twice the basic width BW. The convolution integral process is performed using integers. The middle and bottom diagrams in FIG. 8 illustrate the application of quadratic B-spline basis functions based on each basic width BW determined in the third step S30. When implementing abwFMGFI using numerical analysis software such as MATLAB (trademark pending by MathWorks, Inc., USA), the third step S30 is executed in advance and the values ​​of the three box linear filters are tabulated, thereby making it possible to simultaneously calculate the first step S10, the second step S20, and the fifth step S50 (described later). This allows for even faster processing in this embodiment. As described above, in this step, "application of quadratic B-spline basis functions is realized by convolving a box linear filter three times, for example." The technical significance of "convolving three times" is as follows. Specifically, convolving a box linear filter once is equivalent to convolving with a rectangular weight function. Convolving a box linear filter twice is equivalent to convolving with a triangular weight function. In other words, convolving a box linear filter once or twice cannot be equivalent to a weight function in the form of a quadratic B-spline basis function. In contrast, convolving a box linear filter three times, as in this step, is equivalent to a weight function in the form of a quadratic B-spline basis function. Here, even if a box linear filter is convolved four or more times, a weight function in the form of a quadratic B-spline basis function can be derived. However, when the number of convolutions is three, the time (calculation speed) to derive a weight function in the form of a quadratic B-spline basis function is exponential (10 n) can be significantly faster. The above is the technical significance of "convolving three times." Note that in this step, "application of quadratic B-spline basis functions is realized by convolving a box linear filter three times as an example," but if speed is not a consideration, convolution may be performed four or more times.

[0047] (5th step) The fifth step S50 (an example of the fourth process) is a step of calculating an output value from the multiplication value distribution in the z direction after applying the quadratic B-spline basis function in the fourth step S40. Here, FIG. 10A is a schematic diagram for explaining the calculation of the output value from the multiplication value distribution in the z direction in the fifth step S50 after the processing of the first step S10 to the fourth step S40. Here, the fifth step S50 is one process of the filtering performed by the calculation unit 42 (see FIG. 3), and in this embodiment, the function of performing the fifth step S50 by the calculation unit 42 is referred to as the fourth function. In this step, as shown in FIG. 10A, the pixel value z with the maximum multiplication value in the z direction for each x and y coordinate is set as the output value. That is, the pixel values ​​of each of the multiple pixels are replaced with the maximum values ​​to which the quadratic B-spline basis function is applied. Here, if the multiplication values ​​for multiple pixel values ​​z are the same, a box filter of any width is applied once only to the pixel values ​​z with the same multiplication value, and the pixel value z with the maximum multiplication value is set as the output value. Furthermore, if the multiplication values ​​for multiple pixel values ​​z are the same, the pixel value z closest to the corresponding pixel value in the original image is output. The reason for doing as described above in this step is based on the "process of deriving the basic width BW and output value depending on the presence or absence of noise and edges within the filter width range" below.

[0048] "The process of deriving the basic width BW and output value depending on the presence or absence of noise and edges within the filter width range" 10B to 10E are diagrams showing pixel values ​​within the filter width range, the determined basic width BW, the multiplication value distribution after applying the quadratic B-spline basis function, and the output value depending on the presence or absence of noise and edges. Specifically, they are as follows. 10B is a schematic diagram for explaining the process of deriving the output value when "the target pixel region does not contain noise." In this case, since the target pixel and its surrounding pixels do not contain noise, the degree of variation in pixel values ​​is small, and the standard deviation is small, so the basic width BW tends to be small. The influence of surrounding pixels on the target pixel is small, so there is a high possibility that the target pixel will be output as is. Therefore, by applying this step based on the above idea when "no noise is contained in the target pixel region," it is possible to achieve high reproducibility of the original image. FIG. 10C is a schematic diagram for explaining the process of deriving the output value when the pixel of interest in the target pixel region is noise. In this case, since the pixel of interest is noise, the degree of variation in pixel values ​​increases, leading to a larger standard deviation, and therefore a larger basic width BW. The influence of surrounding pixels on the pixel of interest is large, and the multiplication value for the surrounding pixels tends to be larger than the multiplication value for the pixel of interest. Therefore, the output value is calculated only from the surrounding pixels. Therefore, by applying this step based on the above idea to "when the pixel of interest in the target pixel region is noise," noise can be removed. FIG. 10D is a schematic diagram for explaining the process of deriving an output value when "noise is contained in the surrounding pixels of the target pixel region." In this case, noise is contained in the surrounding pixels, which increases the degree of variation in pixel values ​​and the standard deviation, and therefore the basic width BW tends to be larger. The surrounding pixels have a large effect on noise, and the multiplication value for the surrounding pixels tends to be larger than the multiplication value for the noise. Therefore, the output value is calculated only from the surrounding pixels and is not affected by noise. Therefore, by applying this step based on the above idea when "noise is included in the peripheral pixels of the target pixel region," it is possible to achieve high reproducibility of the original image. Figure 10E is a schematic diagram for explaining the process of deriving the output value when the pixel of interest and its surrounding pixels in the target pixel region are edges and the surrounding pixels are noise. When the pixel of interest and its surrounding pixels are edges, the pixel values ​​vary more and the standard deviation increases, so the basic width BW tends to be larger. The influence of the pixels that make up each edge is significant, forming a multiplication value distribution consisting of two peaks in the z direction. Due to the characteristics of the GF, the multiplication value is larger for the pixel values ​​that make up the edge side including the pixel of interest, so the output value of the edge side including the pixel of interest is obtained. Therefore, by applying this step based on the above idea when "the pixel of interest and the surrounding pixels in the target pixel region are edges and the surrounding pixels are noise," it is possible to preserve the edges.

[0049] The above is a description of the functions, configuration, and actions of the image display system 10 of this embodiment.

[0050] <Effects of the first embodiment> Next, the effects of this embodiment will be described with reference to the results of several experiments described below.

[0051] [First effect] The effect of this embodiment is that the noise removal performance for noise (salt and pepper noise) is superior to that of NLMF, GF, MF, BF, and ABF.

[0052] (Method of salt and pepper noise removal experiment) In this experiment, we compared the denoising performance (noise removal performance) of each filter on sample images with salt-and-pepper noise. Salt-and-pepper noise was added by replacing the pixel values ​​of randomly selected pixels in the original image with either level 0 or level 255. In this experiment, salt-and-pepper noise was added so that the noise ratio to all pixels in the sample image was 0.5%. Denoising performance was compared using visual inspection and root mean square error (RMSE). In this experiment, RMSE was calculated from the difference in pixel values ​​between the original image and the image after applying the filter. A lower RMSE value indicates a higher degree of reproduction of the original image. A higher degree of reproduction of the original image indicates higher denoising performance and edge preservation performance. The filters used in this experiment were abwFMGFI, NLMF (non-local means filter), GF, MF (median filter), BF (bilateral filter), ABF (adaptive bilateral filter), and FMGFI. As shown in Figure 11A, the sample images used in this experiment are 12 types of images in total, including one type of geometric image and 11 types of photographic images. The size of each sample image is 256 x 256 pixels. The size of each filter is 3 x 3 pixels. To avoid implementation errors in the experiment, OpenCV 4.5.3 and 2.4.13 functions (see the following two URLs (URL1 and URL2)) are used to implement NLMF, GF, MF, BF, and ABF. The parameter values ​​of each filter are the same as those in the aforementioned non-patent document 5. The basic width BW of FMGFI is 1, 3, 5, 11, 21, and 51 levels. URL1:<https: / / docs.opencv.org / 2.4 / modules / imgproc / doc / filtering.html> URL2:<https: / / docs.opencv.org / 2.4 / modules / photo / doc / denoising.html>

[0053] (Results and discussion of salt and pepper noise removal experiment) Below, the results of applying the filter of this embodiment (abwFMGFI) are compared with the results of applying the commonly used filters NLMF, GF, MF, BF, and ABF. Here, FIGS. 11B to 11M show the original image of each sample image, an image with noise added, and an image obtained by applying each filter to the noise-added image. The table in FIG. 11N lists the RMSE values ​​of each sample image after each filter. In the table in FIG. 11N, RMSE values ​​for images in which noise has not been completely removed or where edges are blurred are underlined.

[0054] From the visual evaluation of FIGS. 11B to 11M, the following can be confirmed. It can be seen that only abwFMGFI achieves both noise removal and edge preservation in all sample images. MF can remove noise but the edges are blurred. GF can reduce noise but does not remove it, and the entire image is strongly blurred. The results of NLMF, BF and ABF show little change to the original image and are unable to remove noise. Furthermore, the following can be confirmed from the table in Figure 11N. The RMSE value of abwFMGFI is the lowest except for "milkdrop" with MF applied. However, when "milkdrop" with MF applied is visually evaluated, the entire image is blurred, and the deviation of the RMSE value from abwFMGFI is small at -1.7%. From these results, it cannot be said that MF has high denoising performance based on the RMSE value alone. From the above, it can be said that abwFMGFI has better denoising performance against salt-and-pepper noise than NLMF, GF, MF, BF, and ABF. In other words, this embodiment has better noise removal performance against salt-and-pepper noise while maintaining edge preservation performance compared to other comparative filters (NLMF, GF, MF, BF, and ABF). This concludes the explanation of the first effect.

[0055] [Second effect] The second effect is that the present embodiment is superior to FMGFI in terms of noise removal performance against noise (salt and pepper noise). This effect will be confirmed by an experiment similar to the experiment confirming the first effect.

[0056] The results of applying the filter of this embodiment (abwFMGFI) are compared with the results of applying the filter of a comparative embodiment, FMGFI. Figures 12A to 12L show the original image of each sample image, an image with noise added, and an image obtained by applying each filter to the noise-added image. The table in Figure 12M shows the RMSE values ​​of each sample image after each filter. In the table in Figure 12M, RMSE values ​​for images in which noise has not been completely removed or where edges are blurred are underlined.

[0057] The following can be confirmed from visual evaluation of FIGS. 12A to 12L. · abwFMGFI and FMGFI (basic width BW=11, 21 levels) were confirmed to be able to remove noise and preserve edges in all sample images. The filtering results of FMGFI (basic width BW=1 level) show that noise remains in all sample images. FMGFI (basic width BW=3, 5 levels) is able to remove noise from the "Geometric pattern" image, but noise remains in the other sample images. FMGFI (basic width BW=51 level) blurs the edges. Furthermore, the following can be confirmed from the table in Figure 12M. ·abwFMGFI has the lowest RMSE value among the filters whose denoising performance was confirmed. From the above, it can be said that this embodiment (abwFMGFI) has higher denoising performance against salt and pepper noise than any of the FMGFIs with basic widths BW in this experiment. This concludes the explanation of the second effect.

[0058] [Third effect] The effect of this embodiment is that the edge preservation performance is superior to that of NLMF, GF, MF, BF, and ABF.

[0059] (Method of edge preservation experiment) In this experiment, we compare the edge preservation performance of each filter for each sample image with a grid added. A grid is added by replacing all of the columns or rows of the original image with level 1. In this experiment, a grid with a thickness of 3 pixels and a spacing of 16 pixels is added to each row and column. The filters, filter conditions, and sample images used in this experiment are the same as those in the first effect confirmation experiment described above. In this experiment, evaluation is performed using visual inspection and RMSE values. In this case, the RMSE value is calculated from the difference in pixel values ​​between the image with the grid added and the image with the filter applied to the image with the grid added.

[0060] (Results and discussion of edge preservation experiments) The results of applying the filter of this embodiment (abwFMGFI) are compared with the results of applying the commonly used filters NLMF, GF, MF, BF, and ABF. Figures 13A to 13L show sample images with a grid applied to them and images with each filter applied to the grid-applied images. The table in Figure 13M shows the RMSE values ​​of each sample image after applying each filter. In this table, RMSE values ​​for images with uneven or blurred edges are underlined.

[0061] From the visual evaluation of FIGS. 13A to 13L, the following can be confirmed. ·abwFMGFI, NLMF and ABF confirm the preservation of edges. ·GF and BF blur the edges, and MF expands the lattice intersections. Furthermore, the following can be confirmed from the table in Figure 13M. ·abwFMGFI has the lowest RMSE value among the filters whose denoising performance was confirmed. From the above, it can be said that the edge preservation performance of abwFMGFI is at the same level as NLMF and ABF, and is higher than GF, MF, and BF. This concludes the explanation of the third effect.

[0062] [Fourth Effect] The effect of this embodiment is that it is superior to FMGFI in terms of edge preservation performance. This effect is confirmed by an experiment similar to the experiment that confirmed the third effect.

[0063] The results of applying the filter of this embodiment (abwFMGFI) are compared with the results of applying the filter of FMGFI. Figures 14A to 14L show images in which a grid is applied to each sample image and images in which each filter is applied to the grid-applied image. The table in Figure 14M shows the RMSE values ​​of each sample image after applying each filter. In this table, RMSE values ​​for images with uneven or blurred edges are underlined.

[0064] From the visual evaluation of FIGS. 14A to 14L, the following can be confirmed. In all sample images, abwFMGFI and FMGFI (basic width BW = 21 levels) show high edge preservation performance. Furthermore, comparison shows that abwFMGFI preserves edges better than FMGFI (basic width BW = 21 levels). ·FMGFI (basic width BW=1 level) makes the edges uneven in all sample images. FMGFI (basic width BW=3, 5, 11 levels) preserves edges in the "Geometric pattern" but makes the edges uneven in other sample images. Also, FMGFI (basic width BW=51 levels) blurs the edges in all sample images. Furthermore, the following can be confirmed from the table in Figure 14M. ·abwFMGFI has a lower RMSE value than FMGFI of any of the basic widths BW. From the above, it can be said that abwFMGFI has better edge preservation performance than FMGFI. This concludes the explanation of the fourth effect.

[0065] [Fifth Effect] The effect of this embodiment (abwFMGFI) is that it is superior to FMGFI in terms of processing speed.

[0066] (Method of experiment to measure processing speed) In this experiment, we compare the average processing time (hereinafter referred to as the average processing time) required when applying the abwFMGFI filter and the FMGFI filter as a comparison form 10 times each. The image size used in this experiment is 256 x 256 pixels. The conditions for each filter in this experiment are the same as those in the salt-and-pepper noise removal experiment described above. The table in Figure 15A shows the specifications and programming language of the PC (Personal Computer) used in this experiment.

[0067] (Results and considerations of processing speed measurement experiment) The experimental results of the average processing time of this embodiment (abwFMGFI) and the average processing time of the comparative embodiment (FMGFI) are compared. The table in FIG. 15B shows the average processing time of abwFMGFI and the average processing time of FMGFI. The average processing time of FMGFI increases as the basic width BW increases. The average processing time of abwFMGFI is between the condition where the basic width BW of FMGFI is 21 levels and the condition where it is 51 levels. Looking only at the results in the table of Figure 15B, it seems that abwFMGFI is not superior to FMGFI in terms of processing speed, but in the case of FMGFI, multiple basic widths BW must be manually set and the images processed under these conditions must be visually evaluated by a human judge. In fact, selecting the optimal basic width BW using visual inspection and the RMSE value takes at least 5 minutes per image. In contrast, in the case of abwFMGFI, the basic width BW can be determined automatically and the filter is applied only once, so under the conditions in the table of Figure 15A, it only takes the time shown in the table of Figure 15B (0.359 seconds). Therefore, abwFMGFI requires less effort and shorter processing time than FMGFI. It may seem possible to obtain quality equivalent to abwFMGFI by having an operator determine each basic width BW for each pixel region through visual observation and then apply FMGFI, but this is not realistic. This is because this method involves the operator determining each basic width BW, and therefore the quality cannot be said to be stable due to variations between operators.

[0068] The above is a description of the effects of the first embodiment. The above is also a description of the first embodiment.

[0069] Second Embodiment Next, the functions, configuration, and operation of the image printing system 10A of the second embodiment, as well as the effects of the second embodiment, will be described in the order of description with reference to FIG.

[0070] <Functions, Configurations, and Actions of the Image Printing System of the Second Embodiment> 16 is a schematic diagram of an image printing system 10A of this embodiment. The image printing system 10A includes a printer 30A (an example of a printing device) instead of the display 30 of the image processing system 10 of the first embodiment. This completes the differences between this embodiment and the first embodiment.

[0071] 〔printer〕 Printer 30A has the function of printing onto a medium an output image that is output when a first function, a second function, a third function, and a fourth function are executed on a two-dimensional image by calculation unit 42 that constitutes image processing engine 40. The type of printer 30A is not important as long as it has a configuration that can perform this function.

[0072] <Effects of the second embodiment> As shown in Figures 3 and 16, this embodiment includes the image processing engine 40 of the first embodiment. Therefore, the image printing system 10A of this embodiment takes a shorter time from the start of image processing to the end of printing than an image printing system (not shown) that processes images using FMGFI. Accordingly, a method for producing a printed product that also uses the image printing system 10A of this embodiment takes a shorter time from the start of image processing to the end of printing than a method for producing a printed product that uses an image printing system that processes images using FMGFI. Other effects of this embodiment are based on the effects of the first embodiment. That is, compared to an image processing system (not shown) that processes images using NLMF, GF, MF, BF, and ABF, the image printing system 10A of this embodiment can print images in which noise has been removed and edges have been preserved, for example.

[0073] The above is the description of the second embodiment.

[0074] Third Embodiment Next, the functions, configuration, and actions of the image analysis system 10B of the third embodiment, as well as the effects of the third embodiment, will be described in the order shown in FIG. 17A and FIG. 17B.

[0075] <Functions, Configuration, and Actions of the Image Analysis System of the Third Embodiment> 17A is a schematic diagram of an image analysis system 10B of this embodiment. The image analysis system 10B includes an image analysis engine 40B (an example of an analysis device) instead of the image analysis engine 40 of the image processing system 10 of the first embodiment. Furthermore, the image analysis system 10B includes a notification unit 30B instead of the display 30. This completes the differences between this embodiment and the first embodiment.

[0076] [Image analysis engine] As shown in FIGS. 3 and 17, the image analysis engine 40B stores an image analysis program 46Q and authentication image data OD in the storage unit 46 of the image analysis engine 40 of the first embodiment. As an example, the image analysis program 46Q is a program for analyzing an output image processed by the calculation unit 42 using abwFMGFI based on the image processing program 46P and an image based on the authentication image data OD, and determining whether the two match. For example, if the image captured by the camera 20 is a face image, the authentication image data OD is data of an image of the face previously captured. As described above, the image analysis engine 40B has the function of analyzing the shape of the object using the output image output by the calculation unit 42 when the first, second, third and fourth functions are executed on a two-dimensional image of the object (in the present embodiment, an image of a face photographed by the camera 20, as an example) and the authentication image data OD stored in the memory unit 46, and determining whether the two match.

[0077] 17B shows an algorithm when image processing program 46P and image analysis program 46Q cooperate with each other. As shown in this figure, in this algorithm (processing flow), camera 20 captures a facial image (step S100), then calculation unit 42 uses image processing program 46P to perform image processing by abwFMGFI on the facial image acquired in step S100 (step S110), then calculation unit 42 compares the output image that has been image processed based on abwFMGFI with authentication image data OD stored in storage unit 46 (step S120) and determines whether the two match (step S130), and if a positive determination is made in step S130, notification unit 30B notifies that the two match, and if a negative determination is made, notification unit 30B notifies that the two do not match.

[0078] <Effects of the third embodiment> As shown in FIGS. 3 and 17A, the image analysis engine 40B of this embodiment includes the image processing engine 40 of the first embodiment described above. Therefore, the image analysis system 10B of this embodiment takes a shorter time from the start of image processing to the end of image analysis than an image analysis system (not shown) that processes images using FMGFI. Also, the image analysis system 10B of this embodiment can perform image authentication using an image from which noise has been removed and edges have been preserved, and therefore can perform more accurate image analysis or authentication than image analysis systems (not shown) that process images using NLMF, GF, MF, BF, and ABF. Other effects of this embodiment are based on the effects of the first embodiment.

[0079] The above is the description of the third embodiment.

[0080] <<Multiple Modifications>> As described above, the present invention has been described using the above-mentioned multiple embodiments as examples, but the present invention is not limited to these embodiments. The technical scope of the present invention also includes, for example, multiple modified examples described below.

[0081] In each embodiment, the abwFMGFI is applied to a predetermined image region (3 × 3 in the example of FIG. 7C) for the entire range of a two-dimensional image (original image). However, the image region may be selectable by the user on the software. For example, the image processing of each embodiment may be applied only to an image region that includes an edge portion of the original image.

[0082] In each embodiment, the statistical value (statistical information of the image) used in calculating the reference value BW has been described as the standard deviation within each filter width range. However, the statistical value does not have to be the standard deviation as long as it is a parameter that can clarify characteristics such as the variation and distribution of pixel values ​​within that filter width range. For example, the statistical value may be a value obtained by calculating the range of pixel values, a four-directional range, an absolute value average, or other calculations. Furthermore, when calculating the reference value BW using the standard deviation as the statistical value, the reference value BW is set to 1 / 2 of the standard deviation, i.e., the coefficient is set to 1 / 2. However, when using a statistical value other than the standard deviation, this coefficient may be set to a coefficient different from that in each embodiment.

[0083] In the first embodiment, the camera 20 is described as being included in the components of the image display system 10. However, the camera 20 is not an essential component.

[0084] In the second embodiment, the components of the image printing system 10A are described as including the camera 20. However, the camera 20 is not an essential component. For example, it is sufficient if image data to be subjected to image processing by abwFMGFI is input to the input unit 44.

[0085] In the third embodiment, the components of the image analysis system 10B have been described as including the camera 20 and the notification unit 30B. However, the camera 20 and the notification unit 30B are not essential components. For example, it is sufficient that image data to be subjected to image processing by abwFMGFI is input to the input unit 44, and the notification unit 30B may be an external device (a component other than the image analysis system 10B).

[0086] In the third embodiment, the image analysis system 10B has been described assuming that the object to which abwFMGFI is applied is an image captured by the camera 20 (see FIG. 17B). However, the object to which abwFMGFI is applied may also be an image based on the authentication image data OD. Furthermore, the object to which abwFMGFI is applied may also be both (an image captured by the camera 20 and an image based on the authentication image data OD). Furthermore, the user may be able to select whether to apply abwFMGFI to one or both.

[0087] <<Additional Notes>> As described above, the present invention has been described with reference to each embodiment, but if the image processing method of the present invention is expressed more simply than the image processing method of the first aspect described above, it can be specified as follows. (An invention that further simplifies the image processing method of the first aspect) applying a quadratic B-spline basis function to pixel values ​​of a plurality of pixels of the two-dimensional image to which the Gaussian filter processing has been applied in an orthogonal direction of the two-dimensional image, using a statistical value identified from the two-dimensional image as a basic width; replacing each pixel value of the plurality of pixels with a respective maximum value to which the quadratic B-spline basis function has been applied; Image processing methods. Furthermore, the image processing method of the present invention can be specified as follows, if expressed in an even simpler way than the image processing program of the first aspect described above. (An invention that further simplifies the image processing program of the first aspect) On the computer, a function of applying a quadratic B-spline basis function to pixel values ​​of a plurality of pixels of a two-dimensional image to which Gaussian filtering has been applied in an orthogonal direction of the two-dimensional image, using a statistical value identified from the two-dimensional image as a basic width; a function of replacing each pixel value of the plurality of pixels with each maximum value to which the quadratic B-spline basis function is applied; Execute Image processing program. [Explanation of symbols]

[0088] 10 Image display system 10A Image Printing System 10B Image Analysis System 20 Camera (an example of a photographic device) 30 Display (an example of a display device) 30A Printer (an example of a printing device) 30B Notification Department 40 Image processing engine (an example of an image processing device) 40B Image analysis engine (an example of an image analysis device) 42 Calculation unit (an example of an execution unit) 44 Input section 46 Memory section 46P Image Processing Program 46Q Image analysis program 48 Output section OD authentication image data S10 First step (an example of the first process) S20 Second step (an example of the second process) S30 Third step (an example of part of the third process) S40 4th step (an example of the remaining part of the 3rd step) S50 5th step (an example of the 4th step)

Claims

1. a first step of setting reference points in an xyz orthogonal coordinate system for pixel values ​​of each pixel in each of a plurality of pixels constituting a two-dimensional image in an xy orthogonal coordinate system; a second step of applying Gaussian filtering to each of a plurality of groups of pixels formed from a portion of the plurality of pixels in an xy orthogonal coordinate system with a pixel of interest located at the center of each group as a center; a third step of applying a quadratic B-spline basis function to each pixel along the z-axis of an x-y-z orthogonal coordinate system using a statistical value determined from the two-dimensional image as a basic width; a fourth step of replacing the pixel values ​​of the plurality of pixels of interest with the pixel values ​​of the pixels of interest that have the largest multiplication value distribution to which the quadratic B-spline basis function is applied; An image processing method comprising:

2. a first step of setting reference points in an xyz orthogonal coordinate system for pixel values ​​of each pixel in each of a plurality of pixels constituting a two-dimensional image in an xy orthogonal coordinate system; a second step of applying Gaussian filtering to each of a plurality of groups of pixels formed from a portion of the plurality of pixels in an xy orthogonal coordinate system with a pixel of interest located at the center of each group as a center; a third step of applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz orthogonal coordinate system, using a statistical value of each aggregate identified from the pixel of interest and a plurality of peripheral pixels adjacent to the pixel of interest as each basic width; a fourth step of replacing the pixel values ​​of the plurality of pixels of interest with the pixel values ​​of the pixels of interest that have the largest multiplication value distribution to which the quadratic B-spline basis function is applied; An image processing method comprising:

3. the statistical value is set to a statistical value of each group identified from the target pixel and the plurality of peripheral pixels; The image processing method according to claim 2 .

4. On the computer, a first function of setting, for each of a plurality of pixels constituting a two-dimensional image in an xy orthogonal coordinate system, reference points in an xyz orthogonal coordinate system for pixel values ​​of each pixel; a second function of applying Gaussian filtering to each of a plurality of groups of pixels formed by a portion of the plurality of pixels in an xy orthogonal coordinate system with a pixel of interest located at the center of each group as a center; a third function for applying a quadratic B-spline basis function to each pixel along the z-axis of an x-y-z Cartesian coordinate system, using a statistical value determined from the two-dimensional image as a base width; and a fourth function of replacing the pixel values ​​of the plurality of pixels of interest with the pixel values ​​of the largest multiplication value distributions to which the quadratic B-spline basis functions are applied; Execute Image processing program.

5. On the computer, a first function of setting, for each of a plurality of pixels constituting a two-dimensional image in an xy orthogonal coordinate system, reference points in an xyz orthogonal coordinate system for pixel values ​​of each pixel; a second function of applying Gaussian filtering to each of a plurality of groups of pixels formed by a portion of the plurality of pixels in an xy orthogonal coordinate system with a pixel of interest located at the center of each group as a center; a third function of applying a quadratic B-spline basis function to each pixel along the z-axis of an xyz orthogonal coordinate system, using a statistical value of each cluster identified from the pixel of interest and a plurality of surrounding pixels adjacent to the pixel of interest as each basic width; and a fourth function of replacing the pixel values ​​of the plurality of pixels of interest with the pixel values ​​of the largest multiplication value distributions to which the quadratic B-spline basis functions are applied; Execute Image processing program.

6. the statistical value is set to a statistical value of each group identified from the target pixel and the plurality of peripheral pixels; The image processing program according to claim 5 .

7. a storage unit that stores the image processing program according to any one of claims 4 to 6; an execution unit configured to be able to communicate with the storage unit and to execute the first function, the second function, the third function, and the fourth function in accordance with the image processing program; An image processing device comprising:

8. The image processing device according to claim 7 ; a display device that displays an output image that is output as a two-dimensional image by the execution unit executing the first function, the second function, the third function, and the fourth function; and An image display system comprising:

9. The image processing device according to claim 7 ; a printing device that prints, on a medium, an output image that is output by the execution unit executing the first function, the second function, the third function, and the fourth function on a two-dimensional image; An image printing system comprising:

10. The image processing device according to claim 7 ; an imaging device that captures an image of an object, the imaging device being configured to be able to communicate with the image processing device and transmitting a captured two-dimensional image of the object to the image processing device; an analysis device that analyzes a shape of the object using an output image output by the execution unit when the first function, the second function, the third function, and the fourth function are executed on a two-dimensional image of the object; An image analysis system comprising:

11. Using the image printing system according to claim 9, printing the output image on a medium; Methods for producing printed materials.

Images