Information processing device, information processing method, recording medium, imaging system, and imaging method

Abstract

The purpose of the present invention is to improve a technique related to noise processing of an image. This information processing device comprises an extraction unit and a noise information generation unit. The extraction unit extracts one or more partial images. The extraction of the one or more partial images is performed on the basis of one or more images. Each partial image has a smaller number of pixels than the one image in which the partial image is included. The noise information generation unit generates noise information. The one or more partial images are used in the generation of the noise information. The noise information is information indicating noise included in the one or more images.

Description

Information processing device, information processing method, recording medium, imaging system, and imaging method

[0001] The present disclosure relates to an information processing device, an information processing method, a recording medium, an imaging system, and an imaging method.

[0002] There are two types of noise contained in images captured by cameras, etc.: random noise and fixed pattern noise. Fixed pattern noise depends on the characteristics of the image sensor and is difficult to reduce by processing such as averaging.

[0003] Japanese Patent Application Laid-Open No. 2006-129999 describes detecting fixed pattern noise output from an imaging device driven without exposure, and also describes removing fixed pattern noise contained in an output signal obtained when the imaging device captures subject light.

[0004] Japanese Unexamined Patent Publication No. 6-197285

[0005] The present disclosure aims to improve upon the techniques described in the prior art documents mentioned above.

[0006] The information processing device in this disclosure comprises: an extraction means for extracting, based on one or more images, one or more partial images each having a smaller number of pixels than one image included in the one or more images; and a noise information generation means for generating, based on the one or more partial images, information indicating noise included in the one or more images, wherein the noise information generation means generates, for a partial image included in the one or more partial images, relational information indicating the relative values ​​of each of a plurality of pixel values ​​included in the partial image, based on the pixel value of a reference pixel included in the partial image, and generates information indicating the noise using the generated relational information.

[0007] The information processing method disclosed herein is a method in which one or more computers extract, based on one or more images, one or more partial images each having a smaller number of pixels than an image included in the one or more images, generate relationship information for each partial image included in the one or more partial images that indicates the relative values ​​of each of multiple pixel values ​​included in the partial image, based on the pixel value of a reference pixel included in the partial image, and use the generated relationship information to generate information indicating noise included in the one or more images.

[0008] The recording medium in this disclosure is a computer-readable recording medium having a program recorded thereon, the program causing the computer to function as: an extraction means for extracting, based on one or more images, one or more partial images each having a smaller number of pixels than an image included in the one or more images; and a noise information generation means for generating information indicating noise included in the one or more images using the one or more partial images, the noise information generation means generating relationship information for a partial image included in the one or more partial images that indicates the relative values ​​of each of a plurality of pixel values ​​included in the partial image, based on the pixel value of a reference pixel included in the partial image, and generating information indicating the noise using the generated relationship information.

[0009] The imaging system in this disclosure comprises an imaging sensor and an information processing device, wherein the information processing device comprises: an extraction means for extracting, based on one or more images obtained using the imaging sensor, one or more partial images each having a smaller number of pixels than an image included in the one or more images; and a noise information generation means for using the one or more partial images to generate information indicating noise included in the one or more images, wherein the noise information generation means generates relational information for a partial image included in the one or more partial images that indicates the relative values ​​of each of a plurality of pixel values ​​included in the partial image, based on the pixel value of a reference pixel included in the partial image, and generates information indicating the noise using the generated relational information.

[0010] The imaging method disclosed herein is a method of capturing one or more images with an imaging sensor, extracting one or more partial images based on the one or more images, each having a smaller number of pixels than one of the images included in the one or more images, generating relational information for each partial image included in the one or more partial images that indicates the relative values ​​of each of multiple pixel values ​​included in the partial image, based on the pixel value of a reference pixel included in the partial image, and using the generated relational information to generate information indicating noise included in the one or more images.

[0011] 1 is a diagram illustrating an example of a configuration of an information processing device according to the present disclosure. FIG. 2 is a diagram illustrating an example of a computer for realizing an information processing device. FIG. 3 is a diagram illustrating an example of an information processing method according to the present disclosure. FIG. 4 is a diagram illustrating an example of a configuration of an imaging system according to the present disclosure. FIG. 5 is a diagram illustrating an example of an imaging method according to the present disclosure. FIG. 6 is a diagram illustrating an example of processing related to one partial image. FIG. 7 is a diagram illustrating an example of processing related to a plurality of imaging scenes. FIG. 8 is a diagram illustrating an example of a relationship between a plurality of partial images. FIG. 9 is a diagram illustrating integration of statistical relationship information. FIG. 10 is a flowchart illustrating in detail a flow of processing performed by an information processing device according to the present disclosure. FIG. 11 is a diagram illustrating a configuration of an information processing device and an imaging system including a noise reduction unit. FIG. 12 is a diagram illustrating a configuration of an information processing device and an imaging system including a determination unit. FIG. 13 is a flowchart illustrating a flow of processing by an information processing device to generate noise information for each combination of temperature and exposure time. FIG. 14 is a diagram illustrating a configuration of an information processing device and an imaging system including an authentication unit. FIG. 15 is a flowchart illustrating a flow of processing by an information processing device to perform authentication. FIG. 16 is a flowchart illustrating a flow of processing by an extraction unit to extract one or more partial images based on a detected region of interest. 1 is a flowchart illustrating a processing flow of an information processing device that acquires information from a target imaging sensor and specifies the shape of a partial image. 2 is a diagram illustrating an example of an information processing device and an imaging system that include a control unit. 3 is a diagram illustrating an example of an imaging system that includes a diffusion plate. 4 is a diagram illustrating an example of an imaging system that includes a drive unit.

[0012] Hereinafter, in this disclosure, the drawings relate to one or more embodiments. In addition, in all drawings, similar components are designated by similar reference numerals and descriptions thereof will be omitted where appropriate.

[0013] In the following description, when a pixel in one image or information corresponds to a pixel in another image or information, it means that the position (coordinates) of the pixel in both images or information is the same.

[0014] First Embodiment FIG. 1 is a diagram illustrating an example of the configuration of an information processing device 10 according to the present disclosure. As illustrated in FIG. 1, the information processing device 10 includes an extraction unit 110 and a noise information generation unit 130. The extraction unit 110 extracts one or more partial images based on one or more images. The partial images have fewer pixels than a single image included in the one or more images. The noise information generation unit 130 generates noise information using the one or more partial images. The noise information is information indicating noise included in the one or more images. The noise information generation unit 130 generates relationship information for the partial images included in the one or more partial images. The relationship information indicates the relative values ​​of each of the multiple pixel values ​​included in the partial image, with the pixel value of a reference pixel included in the partial image as a reference. The noise information generation unit 130 then generates noise information using the generated relationship information.

[0015] According to this information processing device 10, noise estimation is possible without the need to capture an image in a state where there is no light input (a so-called dark frame).

[0016] According to this information processing device 10, noise estimation is possible using a normal image that is not a dark frame. Capturing a dark frame often requires a mechanical mechanism for blocking light. In contrast, according to this information processing device 10, since it is not necessary to capture a dark frame, such a mechanical mechanism is not required. Therefore, compared to devices that require a mechanical mechanism for blocking light, the device can be more easily miniaturized. Furthermore, compared to devices that require a mechanical mechanism for blocking light, the risk of failure can be reduced. Additionally, manufacturing costs can be reduced.

[0017] The hardware configuration of the information processing device 10 will be described below. Each functional component of the information processing device 10 (extraction unit 110 and noise information generation unit 130) may be realized by hardware that realizes the functional component (e.g., a hardwired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). Below, a case where each functional component of the information processing device 10 is realized by a combination of hardware and software will be further described.

[0018] FIG. 2 is a diagram illustrating a computer 1000 for realizing the information processing device 10. The computer 1000 is any computer. For example, the computer 1000 is a system on chip (SoC), a personal computer (PC), a server machine, a tablet terminal, a smartphone, or the like. The computer 1000 may be a dedicated computer designed to realize the information processing device 10, or may be a general-purpose computer. Furthermore, the information processing device 10 may be realized by a single computer 1000 or by a combination of multiple computers 1000.

[0019] The computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path through which the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 transmit and receive data to and from each other. However, the method of interconnecting the processor 1040 and other components is not limited to bus connection. The processor 1040 may be any of various processors, such as a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory 1060 is a main storage device implemented using a random access memory (RAM) or the like. The storage device 1080 is an auxiliary storage device implemented using a hard disk, a solid state drive (SSD), a memory card, a read-only memory (ROM), or the like.

[0020] The input / output interface 1100 is an interface for connecting the computer 1000 to an input / output device. For example, an input device such as a keyboard and an output device such as a display are connected to the input / output interface 1100. The input / output interface 1100 may be connected to the input device or output device via a wireless connection or a wired connection.

[0021] The network interface 1120 is an interface for connecting the computer 1000 to a network. This communication network may be, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The network interface 1120 may be connected to the network wirelessly or by wire.

[0022] The storage device 1080 stores program modules that realize the various functional components of the information processing device 10. The processor 1040 reads these program modules into the memory 1060 and executes them to realize the functions corresponding to the respective program modules.

[0023] FIG. 3 is a diagram showing an example of an information processing method according to the present disclosure. This information processing method includes steps S20, S30, and S40. In step S20, one or more computers extract one or more partial images based on one or more images. The partial images have fewer pixels than an image included in the one or more images. In step S30, the one or more computers generate relationship information for the partial images included in the one or more partial images. The relationship information indicates the relative values ​​of each of multiple pixel values ​​included in the partial image, with the pixel value of a reference pixel included in the partial image as a reference. In step S40, the one or more computers use the generated relationship information to generate noise information. The noise information is information indicating noise included in the one or more images.

[0024] FIG. 4 is a diagram showing an example of the configuration of an imaging system 20 according to the present disclosure. The imaging system 20 includes an imaging sensor 200 and an information processing device 10. The information processing device 10 includes an extraction unit 110 and a noise information generation unit 130. The extraction unit 110 extracts one or more partial images based on one or more images acquired using the imaging sensor 200. The partial images have fewer pixels than a single image included in the one or more images. The noise information generation unit 130 generates noise information using the one or more partial images. The noise information indicates noise included in the one or more images. The noise information generation unit 130 generates relationship information for each partial image included in the one or more partial images. The relationship information indicates the relative values ​​of each of the multiple pixel values ​​included in the partial image, with the pixel value of a reference pixel included in the partial image as the reference. The noise information generation unit 130 then generates noise information using the generated relationship information.

[0025] The image sensor 200 is communicably connected to the information processing device 10 via an input / output interface 1100 or a network interface 1120 of a computer 1000 that constitutes the information processing device 10 .

[0026] FIG. 5 is a diagram showing an example of an imaging method according to the present disclosure. This imaging method includes steps S10, S20, S30, and S40. In step S10, one or more images are captured by the image sensor 200. In step S20, one or more computers extract one or more partial images based on the one or more images. Each partial image has fewer pixels than one image included in . In step S30, the one or more computers generate relationship information for each partial image included in the one or more partial images. The relationship information indicates the relative values ​​of each of multiple pixel values ​​included in the partial image, with the pixel value of a reference pixel included in the partial image as a reference. In step S40, the one or more computers use the generated relationship information to generate noise information. The noise information is information indicating noise included in the one or more images.

[0027] According to the information processing method, imaging system 20, and imaging method disclosed herein, noise estimation is possible without the need to capture an image in a state where there is no light input, as with the information processing device 10 disclosed herein.

[0028] Among noises contained in images, fixed pattern noise is caused by variations in the characteristics of each pixel in the image sensor, and the noise pattern varies for each individual image sensor. Therefore, in order to reduce fixed pattern noise from images, it has been necessary to confirm the noise pattern of the fixed pattern noise for each image sensor in advance, for example by acquiring a dark frame.

[0029] According to the information processing device 10, information processing method, imaging system 20, and imaging method disclosed herein, fixed pattern noise is suitably estimated. That is, the noise information generated by the noise information generation unit 130 is information indicating fixed pattern noise contained in one or more images. That is, the information processing device 10 estimates fixed pattern noise commonly contained in one or more images. The estimated noise information can be used, for example, to reduce noise in the image.

[0030] Hereinafter, detailed examples of the information processing device 10 and the imaging system 20 will be described.

[0031] The extraction unit 110 acquires one or more images. The one or more images acquired by the extraction unit 110 may be color images such as RGB images, or may be black and white images. The one or more images acquired by the extraction unit 110 may be frame images constituting a video. Alternatively, the one or more images acquired by the extraction unit 110 may be infrared images obtained by an infrared camera such as a shortwave infrared camera. In particular, since fixed pattern noise is dominant noise in shortwave infrared cameras, it is preferable to perform noise estimation and noise reduction using the information processing device 10 according to this disclosure.

[0032] The one or more images acquired by the extraction unit 110 may be, for example, images used for foreign substance inspection, product inspection, monitoring, or authentication (e.g., biometric authentication such as face authentication, iris authentication, fingerprint authentication, vein authentication, or palm print authentication). The one or more images acquired by the extraction unit 110 may or may not include an image specially captured for noise estimation. The one or more images acquired by the extraction unit 110 may be, for example, images taken during operation in inspection, monitoring, or authentication.

[0033] The one or more images acquired by the extraction unit 110 may be images acquired by an image sensor in a state where light is input. The one or more images acquired by the extraction unit 110 may not include a dark frame.

[0034] The information processing device 10 performs noise estimation for each imaging sensor. That is, the one or more images used to generate one piece of noise information are all images captured by the same imaging sensor. Hereinafter, the imaging sensor for which noise estimation is performed may be specifically referred to as the "target imaging sensor." Furthermore, it is preferable that the number of pixels in the one or more images used to generate one piece of noise information is the same.

[0035] The extraction unit 110 may acquire one or more images from an imaging device, may acquire them from a device other than the information processing device 10, or may read and acquire them from a storage unit provided inside or outside the information processing device 10. When this storage unit is provided inside the information processing device 10, this storage unit is realized by a storage device 1080 of the computer 1000 that realizes the information processing device 10.

[0036] In the imaging system 20 according to this disclosure, the extraction unit 110 acquires one or more images captured by the imaging sensor 200. The imaging sensor 200 corresponds to a target imaging sensor. The imaging system 20 may include an imaging device equipped with the imaging sensor 200, and the extraction unit 110 may acquire one or more images from the imaging device.

[0037] Examples of imaging devices include visible light cameras, infrared cameras, etc. Examples of visible light cameras include RGB cameras and black and white cameras. Examples of infrared cameras include short wave infrared cameras, near infrared cameras, etc.

[0038] Examples of the imaging sensor 200 include semiconductor image sensors, such as complementary metal oxide semiconductor (CMOS) image sensors and charge coupled device (CCD) image sensors.

[0039] The process executed by the information processing device 10 will be described in detail below.

[0040] FIG. 6 is a diagram for explaining an example of processing for one partial image 32. In FIG.

[0041] The extraction unit 110 generates an average image 31 by averaging one or more images 30 captured at different times. The average image 31 is a time-averaged image of the same captured scene. The meaning of the captured scene will be described in detail later. A time-averaged image of multiple images 30 has less random noise than each of the original images 30. The number of images 30 used to generate the average image 31 is not particularly limited, but is preferably 100 or more, and more preferably 500 or more. Note that when the extraction unit 110 uses only one image 30, the average image 31 is that image 30 itself.

[0042] The extraction unit 110 extracts one or more partial images 32 from the generated average image 31. The partial image 32 is an image obtained by extracting a partial region of the average image 31 and is composed of a plurality of pixels 320. The number of pixels in the partial image 32 is smaller than the number of pixels in the average image 31. In the example of FIG. 6 , the partial image 32 is composed of nine pixels 320 arranged in a 3×3 matrix. However, the shape and size of the partial image 32 are not particularly limited. For example, the partial image 32 may be composed of 25 pixels 320 arranged in a 5×5 matrix. The shape of the partial image 32 may be, for example, a square or a rectangle. The partial image 32 may have any shape composed of a combination of a plurality of pixels 320. The shape and size of the partial image 32 are represented by the combination of a plurality of pixels 320.

[0043] The shape and size of the partial image 32 are predetermined, and the extraction unit 110 can extract one or more partial images 32 based on the predetermined shape and size.

[0044] The partial image 32 can be said to correspond to one area in the image 30. In other words, the partial image 32 can be said to correspond to one area on the light receiving surface of the imaging sensor. The partial image 32 can be associated with position information indicating its position within the image 30. The position information is information indicating the position within the average image 31 from which the partial image 32 was extracted. The position information can be, for example, information indicating the coordinates of a reference pixel 320t (described later) within the average image 31 (i.e., within the image 30).

[0045] As will be described in detail later, the extraction unit 110 preferably extracts multiple partial images 32 from the average image 31 while allowing overlapping in the average image 31. This enables noise estimation over a wide range. When multiple partial images 32 are extracted from the average image 31, it is preferable that the multiple partial images 32 have the same size and shape.

[0046] The noise information generation unit 130 generates, for each of one or more partial images 32, relationship information 33 indicating the relative relationship between multiple pixel values ​​included in the partial image 32. Specifically, the relationship information 33 is information indicating the relative value of each pixel value, with one of the pixel values ​​included in the partial image 32 used as a reference. That is, for each of one or more partial images 32, the noise information generation unit 130 generates relationship information indicating the relative value of each of multiple pixel values ​​included in the partial image 32, with the pixel value of a reference pixel 320t included in the partial image 32 used as a reference. By generating the relationship information 33 indicating the relative relationship between multiple pixel values, it is possible to identify the amount of local change in noise. In the example of FIG. 6 , the reference pixel 320 is the pixel 320 located at the center of the partial image 32. The noise information generation unit 130 then generates noise information using the generated relationship information 33.

[0047] A specific description will be given below. The pixel value s' of the reference pixel 320t ｔ The signal component due to the light incident on the image sensor is s ｔ and the fixed pattern noise component is expressed as n ｔ When expressed as s', ｔ =s ｔ +n ｔ Similarly, the pixel value s' of a pixel 320i other than the reference pixel 320t included in the partial image 32 is ｉ The signal component due to the light incident on the image sensor is s ｉ and the fixed pattern noise component is expressed as n ｉ When expressed as s', ｉ =s ｉ +n ｉHere, assuming that there is no large fluctuation in the signal component due to the light incident on the image sensor in the partial image 32, that is, that the region corresponding to the partial image 32 in the image scene is smooth, then s ｔ =s ｉ Under this assumption, s' ｉ -s' ｔ = n ｉ -n ｔ That is, the pixel value s′ of the reference pixel 320t is ｔ from the pixel value s' of a pixel 320i other than the reference pixel 320t. ｉ The relative value obtained by subtracting 1 can be regarded as the relative value of the magnitude of the fixed pattern noise in pixel 320i with respect to the magnitude of the fixed pattern noise in reference pixel 320t. This relative value can also be said to be the amount of local change in the fixed pattern noise.

[0048] The relationship information 33 obtained by such processing indicates the relative pixel value in the partial image 32 for each pixel 320. The pixel configuration of the relationship information 33 is the same as that of the partial image 32 used to generate the relationship information 33. The relative value of each pixel 320 is the relative value of the pixel value of that pixel 320 with respect to the pixel value of a reference pixel 320t in the partial image 32. In other words, the relative value of each pixel 320 is the pixel value of that pixel 320 in the partial image 32 minus the pixel value of the reference pixel 320t in the partial image 32. In other words, the relationship information 33 indicates zero as the value for the reference pixel 320t, and indicates zero, a positive value, or a negative value as the value for each pixel 320 other than the reference pixel 320t. That is, the relationship information 33 indicates whether the pixel value of each pixel 320 in the partial image 32 is greater than or smaller than the pixel value of the reference pixel 320t, or whether the pixel value of each pixel 320 is the same as the pixel value of the reference pixel 320t.

[0049] Which pixel 320 among the plurality of pixels 320 constituting the partial image 32 is to be the reference pixel 320t is determined in advance. The reference pixel 320t may be any pixel 320 among the plurality of pixels 320 constituting the partial image 32, but it is preferable that the reference pixel 320t is not a pixel 320 constituting the outer edge of the partial image 32. That is, it is preferable that the reference pixel 320t in the partial image 32 is surrounded entirely by eight other pixels 320. It is also more preferable that the reference pixel 320t is a pixel 320 located at the center of the partial image 32. However, for a partial image 32 extracted using a pixel 320 constituting the outer edge of the average image 31 as the reference pixel 320t, the pixel 320 may be a pixel 320 constituting the outer edge of the partial image 32. In summary, if the reference pixel 320t does not correspond to a pixel constituting the outer edge of the average image 31, it is preferable that the reference pixel 320t is surrounded entirely by eight other pixels 320.

[0050] The relationship information 33 can be associated with the same position information as the position information associated with the partial image 32 used to generate the relationship information 33 .

[0051] 7 is a diagram illustrating an example of processing related to a plurality of imaging scenes. The extraction unit 110 uses a plurality of images 30 of different imaging scenes to extract one or more partial images 32 for each imaging scene. The noise information generation unit 130 then generates relationship information 33 for each imaging scene. Furthermore, the noise information generation unit 130 estimates the distribution of noise change amounts in one or more images 30 using the plurality of relationship information 33 obtained from the plurality of partial images 32 that are located at the same position in one or more images 30.

[0052] In this way, by estimating noise using a plurality of images 30 captured in different scenes, the influence of components dependent on the captured scene is reduced, and the accuracy of noise estimation can be improved.

[0053] For example, different imaging scenes are imaging scenes that differ in at least one of the imaging target area, the state of the imaging target area, and the imaging conditions. For example, by capturing images 30 multiple times with at least one of the position and orientation of the imaging device that captures the images 30 different from one another, multiple images 30 with different imaging target areas can be obtained. Examples of the state of the imaging target area include the brightness of the imaging target area, the position of an object (e.g., a person, living thing, or vehicle) located within the imaging target area, and the orientation of the object located within the imaging target area. Examples of imaging conditions include the aperture value, angle of view, and exposure time of the imaging device that captures the images 30. In particular, it is preferable that the extraction unit 110 use multiple images 30 with at least different imaging target areas from one another to extract one or more partial images 32 for each imaging scene.

[0054] Using the method described above, the extraction unit 110 extracts one or more partial images 32 for each imaging scene. Then, the noise information generation unit 130 generates relationship information 33 for each imaging scene. That is, each piece of relationship information 33 is generated using one or more images 30 from the same imaging scene. For the sake of explanation, a collection of one or more images 30 from the same imaging scene is called an image set. The extraction unit 110 uses multiple image sets from different imaging scenes to extract one or more partial images 32 for each imaging scene.

[0055] In the example of FIG. 7 , relationship information 33a is generated using one or more images 30a from imaging scene A. Meanwhile, relationship information 33b is generated using one or more images 30b from imaging scene B. One or more images 30a and one or more images 30b are images captured using the same imaging sensor (target imaging sensor). Here, relationship information 33a and relationship information 33b are generated using partial images 32 corresponding to the same position in image 30. In this manner, relationship information 33 is generated for each of a plurality of imaging scenes. That is, multiple pieces of relationship information 33 are obtained for one region in image 30 (i.e., a region corresponding to one partial image 32). In other words, multiple pieces of relationship information 33 are obtained for one region on the light receiving surface of the imaging sensor.

[0056] The noise information generation unit 130 performs statistical processing on the plurality of pieces of relationship information 33 to generate statistical relationship information 34 for each position in one or more images 30. This statistical processing is, for example, processing to average the relative values ​​indicated in the relationship information 33 for each pixel, or processing to identify the most frequent value of the relative values ​​indicated in the relationship information 33. By performing statistical processing on the plurality of pieces of relationship information 33 by the noise information generation unit 130, components independent of the captured scene can be extracted, thereby improving the accuracy of noise estimation.

[0057] The statistical processing is performed on a set of multiple pieces of relationship information 33 that are associated with the same position information. As a result, one piece of statistical relationship information 34 is obtained for one region in the image 30 (i.e., a region corresponding to one partial image 32). In other words, one piece of statistical relationship information 34 is obtained for one region on the light-receiving surface of the image sensor. The statistical relationship information 34 can be associated with the same position information as the position information associated with the multiple pieces of relationship information 33 used to generate the statistical relationship information 34.

[0058] The pixel configuration of the statistical relationship information 34 is the same as the pixel configuration of the relationship information 33 used to generate the statistical relationship information 34. Each pixel of the statistical relationship information 34 indicates a statistical value of the relative values ​​indicated for the corresponding pixels in the above-described plurality of pieces of relationship information 33. Specifically, in the statistical relationship information 34, zero is indicated as the value for the reference pixel 320t, and zero, a positive value, or a negative value is indicated as the value for each pixel 320 other than the reference pixel 320t.

[0059] The number of captured scenes for obtaining the statistical information 34 is not particularly limited, but is preferably 10 or more, and more preferably 100 or more.

[0060] However, the noise information generator 130 may generate noise information using an image of one imaging scene. In this case, the relation information obtained from the image of the one imaging scene may be used as statistical relation information in subsequent processing.

[0061] The noise information generating unit 130 similarly generates statistical relationship information 34 for multiple regions within the image 30 and integrates them.

[0062] FIG. 8 is a diagram illustrating the relationship between a plurality of partial images 32, and FIG. 9 is a diagram for explaining the integration of statistical relationship information 34.

[0063] In extracting the partial images 32, the extraction unit 110 extracts the partial images 32 from multiple positions in the average image 31, as illustrated in FIG. 8 . In FIG. 8 , reference pixels 320t are indicated by black rectangles. Furthermore, boundaries between pixels are indicated by dotted lines. The multiple partial images 32 are preferably positioned such that, in the average image 31, each partial image 32 overlaps with another partial image 32 at least in part. For example, the extraction unit 110 extracts the partial images 32 such that each of the multiple pixels constituting the average image 31 is set as the reference pixel 320t. However, the extraction unit 110 does not necessarily need to set all of the multiple pixels constituting the average image 31 as the reference pixel 320t. The extraction unit 110 may set at least some of the multiple pixels constituting the average image 31 as the reference pixel 320t. The multiple pixels constituting the average image 31 may include pixels that are not included in any of the partial images 32. In addition, when a partial image 32 is extracted using a pixel 320 that forms the outer edge of the average image 31 as a reference pixel 320t, the shape and size of the partial image 32 may be different from the shape and size of a partial image 32 extracted using a pixel 320 that does not form the outer edge as a reference pixel 320t.

[0064] As described above, the noise information generation unit 130 generates statistical relationship information 34 for each piece of position information of the partial image 32. Then, the noise information generation unit 130 estimates the noise change amount distribution using the generated statistical relationship information 34. The noise information generation unit 130 can generate change amount distribution information 35 that indicates the estimated distribution of the noise change amount.

[0065] When the extraction unit 110 extracts a plurality of partial images 32 having different position information from one another and the noise information generation unit 130 generates a plurality of pieces of statistical relationship information 34 having different position information from one another, the noise information generation unit 130 combines and integrates the plurality of pieces of statistical relationship information 34 based on the position information associated with each piece of statistical relationship information 34, and estimates the noise change distribution. When the noise information generation unit 130 generates change amount distribution information 35 using the plurality of pieces of statistical relationship information 34 having different position information from one another, distribution information over a range wider than each of the partial images 32 is obtained as the change amount distribution information 35. Consequently, noise distribution information over a range wider than each of the partial images 32 is obtained.

[0066] 9 , the noise information generating unit 130 generates statistical relationship information 34a related to position A and statistical relationship information 34b related to position B. That is, the position information associated with the statistical relationship information 34a indicates position A, and the position information associated with the statistical relationship information 34b indicates position B.

[0067] The noise information generation unit 130 places the statistical-related information 34a at a position corresponding to position A in the change amount distribution information 35. That is, the noise information generation unit 130 assigns the values ​​indicated by the multiple pixels constituting the statistical-related information 34a to the corresponding multiple pixels in the change amount distribution information 35. The noise information generation unit 130 also places the statistical-related information 34b at a position corresponding to position B in the change amount distribution information 35. That is, the noise information generation unit 130 assigns the values ​​indicated by the multiple pixels constituting the statistical-related information 34b to the corresponding multiple pixels in the change amount distribution information 35. Here, if the areas corresponding to the multiple pieces of statistical-related information 34 partially overlap each other in the change amount distribution information 35, values ​​based on the multiple pieces of statistical-related information 34 will be assigned to one pixel in the change amount distribution information 35. When multiple values ​​are assigned to one pixel in this way, the average of these multiple values ​​is taken as the value for that pixel in the change amount distribution information 35.

[0068] In this way, the noise information generation unit 130 can generate the change amount distribution information 35 by applying the pixel values ​​constituting each of the multiple pieces of statistical relationship information 34 to the pixels of the change amount distribution information 35. It can be said that the change amount distribution information 35 is information that indicates the distribution of noise change amounts in one or more images 30. The number of pixels in the generated change amount distribution information 35 is equal to or less than the number of pixels of the image 30 acquired by the extraction unit 110.

[0069] Furthermore, the noise information generation unit 130 generates noise distribution information as noise information by adding an offset value to the entire noise variation distribution. The variation distribution information 35 described above indicates the relative relationship of noise between pixels. Here, by adding an appropriate offset value to each pixel value of the variation distribution information 35, the noise value of each pixel can be estimated.

[0070] For example, the noise information generation unit 130 identifies the lowest pixel in the reference image that has the lowest pixel value. The noise information generation unit 130 can then identify an offset value so that the value of the pixel corresponding to the lowest pixel in the noise variation distribution matches the pixel value of the lowest pixel. It can be said that the lower the pixel value in the image, the less light the pixel receives in the image sensor. In other words, the pixel value of the lowest pixel in the reference image has the lowest content of signal components resulting from the light most incident on the image sensor, and conversely, the highest content of noise components. Therefore, by identifying an offset value so that the value of the pixel corresponding to the lowest pixel in the noise variation distribution matches the pixel value of the lowest pixel, the noise value of each pixel can be accurately estimated.

[0071] The noise information generation unit 130 uses an image obtained using the target imaging sensor as a reference image. When noise information is used to reduce noise in an image, it is preferable to use an image of the target for noise reduction as the reference image. As another example, the reference image may be any of the images 30 acquired by the extraction unit 110, or may be an average image 31 generated by the extraction unit 110. It is preferable that the reference image be a time-averaged image in which random noise has been reduced.

[0072] When the change amount distribution information 35 is information for a portion of the pixels of the target imaging sensor, the noise information generator 130 identifies the offset value using a reference image corresponding to the portion of the pixels. For example, the noise information generator 130 may generate the reference image by cutting out a portion of an image formed from the light reception results of all the pixels of the target imaging sensor.

[0073] The noise information generating unit 130 obtains the coordinates and pixel value V of the lowest pixel in the reference image. ｍｉｎ Furthermore, the noise information generating unit 130 determines the value V indicated in the change amount distribution information 35 for the pixel at the same coordinate as the lowest pixel. ｒ Then, the noise information generating unit 130 determines V ｍｉｎ From V ｒ The noise information generation unit 130 specifies the value obtained by subtracting .theta. from .theta. as the offset value. The noise information generation unit 130 adds the specified offset value to the value for all pixels in the change amount distribution information 35. That is, the same offset value is added to all pixels in the change amount distribution information 35. In this way, noise information is obtained as an estimation result. The noise information indicates an estimated noise value for each of the multiple pixels.

[0074] FIG. 10 is a flowchart illustrating in detail the flow of processing executed by the information processing device 10 according to this disclosure.

[0075] In S101, the extraction unit 110 of the information processing device 10 acquires one or more images 30 captured by a target imaging sensor. When the extraction unit 110 acquires two or more images 30 in S101, these images 30 are images of the same scene but captured at different times.

[0076] In S102 following S101, the extraction unit 110 generates an average image 31 using one or more images 30 acquired in S101.

[0077] In S103 following S102, the extraction unit 110 extracts one or more partial images 32 from the average image 31. When the extraction unit 110 extracts two or more partial images 32 in S103, the positions (position information) of these partial images 32 in the average image 31 are different from each other.

[0078] In S104 following S103 , the noise information generating unit 130 generates the relationship information 33 for each partial image 32 .

[0079] Steps S101 to S104 constitute a loop process A for each imaging scene. The loop process A is repeated until processing is completed for a predetermined number of imaging scenes. As another example, the information processing device 10 may end the loop process A when processing is completed for all images 30 that can be acquired by the extraction unit 110.

[0080] One or more images 30 acquired by the extraction unit 110 may be associated with scene information indicating the captured scene. The extraction unit 110 and the noise information generation unit 130 can perform loop processing A based on the scene information associated with the images 30. Scene information may also be associated with the relationship information 33. The noise information generation unit 130 may simply associate the same scene information with the relationship information 33 as the scene information associated with the images 30 used to generate the relationship information 33.

[0081] One or more pieces of relationship information 33 are obtained by the loop process A. When a plurality of pieces of relationship information 33 are obtained, the pieces of relationship information 33 differ from each other in at least one of the scene information and the position information.

[0082] In S105 following the loop process A, the noise information generation unit 130 generates statistical relationship information 34 using one or more pieces of relationship information 33. As described above, the noise information generation unit 130 generates statistical relationship information 34 for each piece of position information.

[0083] In S106 following S105, the noise information generation unit 130 generates change amount distribution information 35 using one or more pieces of statistical relationship information 34 generated in S105. When statistical relationship information 34 is generated for each of a plurality of pieces of position information, the noise information generation unit 130 can generate change amount distribution information 35 by integrating the plurality of pieces of statistical relationship information 34 as described above. When only one piece of statistical relationship information 34 is generated in S105, the noise information generation unit 130 can simply treat that statistical relationship information 34 as the change amount distribution information 35.

[0084] In S107 following S106, the noise information generation unit 130 generates noise information using the reference image and the change amount distribution information 35, as described above. The noise information generation unit 130 may acquire the reference image from an imaging device equipped with a target imaging sensor, may acquire it from a device other than the information processing device 10, or may read and acquire it from a storage unit provided inside or outside the information processing device 10. When this storage unit is provided inside the information processing device 10, this storage unit is realized by the storage device 1080 of the computer 1000 that realizes the information processing device 10.

[0085] The noise information generation unit 130 can output the generated noise information. The noise information generation unit 130 may output the noise information as an image, or as a table showing the relationship between pixel positions and noise values. The noise information generation unit 130 may output the noise information by displaying it on a display connected to the information processing device 10, or may output it to a device other than the information processing device 10. Alternatively, the noise information generation unit 130 may store the noise information in a storage unit provided inside or outside the information processing device 10. When this storage unit is provided inside the information processing device 10, this storage unit is realized by a storage device 1080 of the computer 1000 that realizes the information processing device 10.

[0086] The information processing device 10 may generate noise information by performing the processes of S101 to S107 at predetermined time intervals, or may generate noise information each time a predetermined condition is satisfied. The latter example will be described in detail in the third embodiment. Alternatively, the information processing device 10 may generate noise information by performing the processes of S101 to S107 in response to a user operation or a signal received from the outside.

[0087] As described above, according to this embodiment, the extraction unit 110 extracts one or more partial images based on one or more images. A partial image has fewer pixels than each of the one or more images. The noise information generation unit 130 generates noise information using one or more partial images. Therefore, by processing a partial image having fewer pixels than each of the one or more images, noise estimation is possible without the need to capture an image in a state without light input.

[0088] (Modification) The information processing device 10 according to the modification is the same as the information processing device 10 according to the first embodiment, except that the noise information generating section 130 generates noise information for each of red, green, and blue.

[0089] The one or more images 30 acquired by the extraction unit 110 according to this modification are RGB images (after demosaicing) acquired by an RGB camera. The RGB image indicates luminance values ​​(pixel values) for each of red (R), green (G), and blue (B). In this modification, when the extraction unit 110 acquires the RGB image, the extraction unit 110 generates an R image, a G image, and a B image from the RGB image. The pixel values ​​constituting the R image indicate luminance values ​​for red. The pixel values ​​constituting the G image indicate luminance values ​​for green. The pixel values ​​constituting the B image indicate luminance values ​​for blue. The extraction unit 110 can generate the R image, the G image, and the B image by extracting the luminance values ​​of each color for each pixel from the RGB image.

[0090] Then, the extraction unit 110 and the noise information generation unit 130 perform the processes from S101 to S107 described in the first embodiment for each of the R image, G image, and B image, thereby obtaining R noise information indicating the estimated results of noise contained in the R image, G noise information indicating the estimated results of noise contained in the G image, and B noise information indicating the estimated results of noise contained in the B image.

[0091] Although the example described above illustrates the case where the one or more images 30 acquired by the extraction unit 110 are RGB images after demosaicing, the one or more images 30 acquired by the extraction unit 110 may also be RGB images before demosaicing. In an RGB camera, a color filter array determines the transmission color for each pixel. That is, image data (one-channel image) is obtained in which each pixel indicates a luminance value of one of R, B, and G. Demosaicing is a process of interpolating pixel values ​​of missing pixels for each color. Demosaicing results in image data (three-channel image) in which all luminance values ​​of R, B, and G are associated with each pixel.

[0092] When one or more images 30 acquired by the extraction unit 110 are RGB images before demosaicing, the extraction unit 110 can generate an R image, a G image, and a B image by selecting, for each color, pixel values ​​that indicate the luminance value of that color from the RGB image. The number of pixels in the R image, G image, and B image thus obtained will each be smaller than that of the RGB image.

[0093] Then, the extraction unit 110 and the noise information generation unit 130 perform the processes from S101 to S107 described in the first embodiment for each of the R image, G image, and B image, thereby obtaining R noise information, G noise information, and B noise information.

[0094] The R noise information, G noise information, and B noise information are preferably used for noise reduction of RGB images. The noise information generating unit 130 generates noise information for each of red, green, and blue, thereby enabling noise reduction for RGB images. The noise reduction process using the R noise information, G noise information, and B noise information will be described in detail in the second embodiment.

[0095] 11 is a diagram illustrating the configuration of an information processing device 10 and an imaging system 20 that include a noise reduction unit 150. The information processing device 10 and the imaging system 20 according to the second embodiment are the same as the information processing device 10 and the imaging system 20 according to the first embodiment, except that the information processing device 10 further includes a noise reduction unit 150. The noise reduction unit 150 performs noise reduction processing on a target image using noise information.

[0096] The hardware configuration of the computer that realizes this information processing device 10 is similar to that of the information processing device 10 according to the first embodiment and is shown in Fig. 2, for example. However, the storage device 1080 of the computer 1000 that realizes the information processing device 10 according to this embodiment further stores a program module that realizes the function of the noise reduction unit 150.

[0097] The target image is a target for noise reduction processing. The noise reduction unit 150 performs noise reduction processing on the target image using the noise information generated by the noise information generation unit 130. Here, the imaging sensor used to obtain the target image is the target imaging sensor in the noise information used for noise reduction processing of the target image. Note that the target image may be one of one or more images 30 used to generate the noise information.

[0098] Like the one or more images 30 acquired by the extraction unit 110, the target image may be a color image such as an RGB image, or may be a black and white image. The target image may be a frame image constituting a video. Alternatively, the target image may be an infrared image obtained by an infrared camera such as a short-wave infrared camera. The target image is preferably a time-averaged image in which random noise has been reduced.

[0099] The noise reduction unit 150 may acquire the target image from an imaging device equipped with a target imaging sensor, may acquire the target image from a device other than the information processing device 10, or may read and acquire the target image from a storage unit provided inside or outside the information processing device 10. When this storage unit is provided inside the information processing device 10, this storage unit is realized by the storage device 1080 of the computer 1000 that realizes the information processing device 10.

[0100] An example of the noise reduction process performed by the noise reduction unit 150 will be described below.

[0101] For example, the noise reduction unit 150 can reduce noise in a target pixel by subtracting, from the pixel value of the target image, the noise value of the pixel corresponding to that pixel in the noise information.

[0102] As another example, the noise reduction unit 150 can use the noise information to identify a processing target pixel with strong noise and change the pixel value corresponding to the processing target pixel in the target image, thereby reducing noise in the target image. Specifically, the noise reduction unit 150 identifies, as the processing target pixel, a pixel that exhibits a noise value equal to or greater than a predetermined threshold value from among the multiple pixels that make up the noise information. Then, the noise reduction unit 150 interpolates the value of the pixel corresponding to the processing target pixel in the target image from the surrounding pixels. For example, an existing technique such as a median filter can be used for this interpolation.

[0103] As described in the modified example, a case will be described in which the noise information generation unit 130 generates R noise information, G noise information, and B noise information. In this case, the noise reduction unit 150 can use an RGB image as a target image. The noise reduction unit 150 generates an R image, a G image, and a B image from the RGB image as the target image. The method by which the noise reduction unit 150 generates an R image, a G image, and a B image from the RGB image is the same as the method by which the extraction unit 110 generates an R image, a G image, and a B image from the RGB image.

[0104] Specifically, when the R noise information, G noise information, and B noise information used by 150 are information generated from a demosaiced RGB image, 150 uses the demosaiced RGB image as the target image. 150 can generate each of the R image, G image, and B image by extracting the luminance value of each color for each pixel from the target image. On the other hand, when the R noise information, G noise information, and B noise information used by 150 are information generated from an RBG image before demosaicing, 150 uses the RBG image before desaicing as the target image. 150 can generate each of the R image, G image, and B image by picking out, for each color, the pixel values ​​at which the luminance value of that color is indicated from the target image.

[0105] The noise reduction unit 150 uses the R noise information to perform noise reduction processing on the R image. This noise reduction processing can use the methods described above (for example, a method of subtracting a noise value from a pixel value, or a method of complementing a pixel value). Similarly, the noise reduction unit 150 uses the G noise information to perform noise reduction processing on the G image. Similarly, the noise reduction unit 150 uses the B noise information to perform noise reduction processing on the B image.

[0106] The noise reduction unit 150 then generates an RGB image using the R, G, and B images with reduced noise. Specifically, if the R noise information, G noise information, and B noise information used by the noise reduction unit 150 are information generated from the RGB image after demosaicing, the noise reduction unit 150 integrates the R, G, and B images with reduced noise to generate an RGB image. In this way, an RGB image with reduced noise can be obtained. If the R noise information, G noise information, and B noise information used by the noise reduction unit 150 are information generated from the RGB image before demosaicing, the noise reduction unit 150 performs demosaicing processing using the R, G, and B images with reduced noise. Then, the RGB image is generated by integrating the R, G, and B images after demosaicing.

[0107] The noise reduction unit 150 can output the target image after noise reduction processing. The noise reduction unit 150 may output the processed target image, for example, by displaying it on a display connected to the information processing device 10, or may output it to a device other than the information processing device 10. Alternatively, the noise reduction unit 150 may store the processed target image in a storage unit provided inside or outside the information processing device 10. When this storage unit is provided inside the information processing device 10, this storage unit is realized by the storage device 1080 of the computer 1000 that realizes the information processing device 10.

[0108] In this embodiment, the same functions and effects as those in the first embodiment can be obtained. In addition, in this embodiment, the information processing device 10 further includes a noise reduction unit 150. Therefore, it is possible to reduce noise in the target image.

[0109] Third Embodiment An information processing device 10 and an imaging system 20 according to a third embodiment are the same as the information processing device 10 and the imaging system 20 according to the second embodiment, except for the points described below.

[0110] As described above in the first embodiment, the one or more images 30 used by the information processing device 10 are images obtained using the same imaging sensor, i.e., the target imaging sensor. In this embodiment, the noise information generator 130 generates noise information for each combination of the temperature and exposure time of the target imaging sensor.

[0111] Since fixed pattern noise varies depending on the temperature and exposure time of the image sensor, the noise information generator 130 can estimate noise according to the imaging conditions by generating noise information for each combination of the temperature and exposure time of the target image sensor.

[0112] FIG. 12 is a diagram illustrating an example of the configuration of an information processing device 10 and an imaging system 20 that include a determination unit 100. In the example of FIG. 12, an imaging sensor 200 is a target imaging sensor. The imaging system 20 includes a temperature sensor 201. The temperature sensor 201 is configured to measure the temperature of the imaging sensor 200. The imaging sensor 200 and the temperature sensor 201 are included in an imaging device 290. The imaging device 290 is, for example, a camera that captures images using the imaging sensor 200. The imaging device 290 is communicatively connected to the information processing device 10 via an input / output interface 1100 or a network interface 1120 of a computer 1000 that implements the information processing device 10. The information processing device 10 acquires the measurement results of the temperature sensor 201 from the imaging device 290. The information processing device 10 also acquires information indicating the exposure time during imaging from the imaging device 290.

[0113] The hardware configuration of the computer that realizes this information processing device 10 is similar to that of the information processing device 10 according to the first embodiment and is shown in Fig. 2. However, a program module that realizes the function of the determination unit 100 is further stored in the storage device 1080 of the computer 1000 that realizes the information processing device 10 according to this embodiment.

[0114] FIG. 13 is a flowchart illustrating the flow of processing in which the information processing device 10 generates noise information for each combination of temperature and exposure time.

[0115] In S201, the extraction unit 110 acquires one or more images 30 from the imaging device 290. Then, the extraction unit 110 and the noise information generation unit 130 perform the processes from S101 to S107 described in the first embodiment to generate noise information.

[0116] In S202 following S201, the determination unit 100 acquires from the imaging device 290 and stores the temperature and exposure time of the imaging sensor 200 when the extraction unit 110 captured one or more images 30. Note that S202 may be performed before S201.

[0117] Next, in S203, the noise reduction unit 150 acquires a target image. The noise reduction unit 150 may acquire each of the one or more images 30 used in S201 as the target image. The noise reduction unit 150 may also acquire an image other than the one or more images 30 used in S201 from the imaging device 290 as the target image.

[0118] In S204 following S203, the noise reduction unit 150 performs noise reduction processing on the target image. This noise reduction processing is as described in the second embodiment.

[0119] It is preferable that the noise reduction unit 150 first sets each of the one or more images 30 used in S201 as a target image. Then, after completing the noise reduction process for all of those images 30, it is preferable to acquire new images other than those used in S201 as target images from the image capture device 290. In this way, any image captured by the image capture device 290 can become a target image. In other words, when the image capture device 290 is used for monitoring or inspection, noise reduction can be performed on the image while the monitoring or inspection is continued. This can ultimately improve the accuracy of the monitoring or inspection.

[0120] In S205 following S204, the determination unit 100 determines whether a noise information regeneration condition is satisfied. The regeneration condition may be, for example, that "at least one of the temperature and exposure time of the image sensor 200 has changed by more than a predetermined standard."

[0121] For the purpose of making the determination, the determination unit 100 acquires the latest temperature and exposure time from the image capture device 290. Then, the determination unit 100 determines whether the absolute value of the value obtained by subtracting the latest temperature from the temperature stored in S202 is equal to or greater than a predetermined standard. The determination unit 100 also determines whether the absolute value of the value obtained by subtracting the latest exposure time from the exposure time stored in S202 is equal to or greater than a predetermined standard. The standard is set for each of the temperature and the exposure time. If the calculated absolute value for at least one of the temperature and the exposure time is equal to or greater than the predetermined standard, the determination unit 100 determines that the regeneration condition is satisfied. Otherwise, the determination unit 100 determines that the regeneration condition is not satisfied.

[0122] If the regeneration condition is satisfied (Yes in S205), the process of the information processing device 10 returns to S201, and noise information is generated again. On the other hand, if the regeneration condition is not satisfied (No in S205), the process of the information processing device 10 returns to S203, and noise reduction of the target image is further performed using the noise information that has already been generated. Therefore, noise reduction processing can be performed using noise information that is suited to the conditions under which the target image was obtained, and noise can be reduced with high accuracy.

[0123] This embodiment provides the same functions and effects as the first embodiment. Additionally, in this embodiment, the noise information generator 130 generates noise information for each combination of the temperature of the target image sensor and the exposure time. Therefore, noise estimation according to the image capturing conditions is possible.

[0124] 14 is a diagram illustrating the configuration of an information processing device 10 and an imaging system 20 that include an authentication unit 170. The information processing device 10 and the imaging system 20 according to the fourth embodiment are the same as the information processing device 10 and the imaging system 20 according to the second embodiment, respectively, except that the information processing device 10 further includes an authentication unit 170. The authentication unit 170 performs authentication using a target image that has been subjected to noise reduction processing. Therefore, the influence of noise on authentication can be reduced.

[0125] The hardware configuration of the computer that realizes this information processing device 10 is similar to that of the information processing device 10 according to the first embodiment and is shown in Fig. 2. However, the storage device 1080 of the computer 1000 that realizes the information processing device 10 according to this embodiment further stores a program module that realizes the function of the authentication unit 170.

[0126] Examples of authentication performed by the authentication unit 170 include biometric authentication, such as face authentication, iris authentication, fingerprint authentication, vein authentication, and palm print authentication.

[0127] FIG. 15 is a flowchart illustrating the flow of a process in which the information processing device 10 performs authentication.

[0128] In S301, the extraction unit 110 acquires one or more images 30. Then, the extraction unit 110 and the noise information generation unit 130 perform the processes from S101 to S107 described in the first embodiment to generate noise information.

[0129] In S302 following S301, the noise reduction unit 150 acquires a target image in the same manner as in the second embodiment. The target image is an image to be authenticated. The target image includes, for example, at least one of a face, iris, fingerprint, vein, palm print, and the area around the eye (e.g., eyelids, eyelashes, etc.).

[0130] In S303 following S302, the noise reduction unit 150 performs noise reduction processing on the target image, as in the second embodiment, thereby obtaining a target image with reduced noise.

[0131] In S304, the authentication unit 170 performs authentication processing using the target image after noise reduction processing. In the example of FIG. 14 , the information processing device 10 further includes an authentication information storage unit 101. The authentication information storage unit 101 stores authentication information that associates personal identification information with features used for authentication. The authentication unit 170 can read out the authentication information from the authentication information storage unit 101 and use it for authentication processing. The authentication information storage unit 101 is realized, for example, by a storage device 1080 of the computer 1000 that realizes the information processing device 10. Note that the authentication information storage unit 101 may be provided outside the information processing device 10.

[0132] The authentication unit 170 can perform authentication processing using existing technology. For example, the authentication unit 170 identifies a region of interest in the target image. The region of interest is an area used for authentication. For example, the region of interest is an area of ​​the target image occupied by a face, iris, fingerprint, veins, palm print, or the area around the eyes. The authentication unit 170 also extracts features by analyzing the region of interest in the target image. The authentication unit 170 identifies features that are highly similar to the features extracted from the target image from the authentication information stored in the authentication information storage unit 101. Then, the authentication unit 170 identifies personal identification information associated with the identified features in the authentication information. The authentication unit 170 can output the identified personal identification information as an authentication result.

[0133] Alternatively, the authentication unit 170 may output information indicating whether or not there is a feature that matches the target image in the authentication information stored in the authentication information storage unit 101. In this case, the authentication unit 170 determines whether or not there is a feature in the authentication information stored in the authentication information storage unit 101 whose similarity to the feature extracted from the target image is equal to or greater than a predetermined threshold. Then, the authentication unit 170 outputs the determination result.

[0134] As in the third embodiment, the information processing device 10 according to this embodiment may further include a determination unit 100. The imaging device 290 may also include a temperature sensor 201. The noise information generation unit 130 may generate noise information for each combination of the temperature of the target imaging sensor and the exposure time.

[0135] In this embodiment, the same effects and advantages as those of the first embodiment can be obtained. In addition, in this embodiment, the authentication unit 170 performs authentication using a target image that has been subjected to noise reduction processing. Therefore, the influence of noise on authentication can be reduced.

[0136] Fifth Embodiment An information processing device 10 according to a fifth embodiment is the same as the information processing device 10 according to at least one of the first to fourth embodiments, except for the points described below. An imaging system 20 according to the fifth embodiment is the same as the imaging system 20 according to at least one of the first to fourth embodiments, except for the points described below. In the fifth embodiment, the extraction unit 110 extracts one or more partial images 32 based on an area of ​​one or more images 30 that is likely to contain a region of interest. By limiting the area and performing processing in this manner, it is possible to reduce the number of partial images 32 or the number of pixels in the partial images 32. As a result, it is possible to reduce the processing load on the information processing device 10.

[0137] For example, assume that one or more images 30 or target images are images used for biometric authentication. In such cases, the position and size of the area of ​​the image occupied by the face, iris, fingerprint, veins, palm print, or the area around the eyes, etc., used for authentication, are often roughly fixed. Therefore, the area occupied by the face, iris, fingerprint, veins, palm print, or the area around the eyes, etc., used for authentication, can be called the area of ​​interest.

[0138] As another example, suppose one or more images 30 or target images are images used for surveillance. For example, in an image of a hallway, the location and size of the area of ​​particular interest, e.g., the area occupied by a person passing through the hallway, is often roughly fixed. Therefore, the area of ​​particular interest can be referred to as an area of ​​interest.

[0139] As another example, assume that one or more images 30 or target images are images used for inspection. A portion of an image obtained by capturing an image of an item may be the area to be inspected. Furthermore, the position and size of the area to be inspected within image 30 may be roughly determined. Therefore, the area to be inspected in particular may be referred to as the area of ​​interest.

[0140] In this embodiment, the extraction unit 110 determines, as the extraction target region, a region in which the above-described regions of interest are likely to occur. When extracting one or more partial images 32 from the average image 31, the extraction unit 110 extracts one or more partial images 32 from a region of the average image 31 that corresponds to the extraction target region of the image 30. On the other hand, the extraction unit 110 does not need to extract partial images 32 from a region of the average image 31 other than the region that corresponds to the extraction target region of the image 30.

[0141] The target area information indicating the extraction target area can be determined in advance. The target area information is information for identifying the extraction target area in the image 30. For example, if the shape of the extraction target area is a rectangle, the target area information includes information indicating the vertical length of the rectangle, the horizontal length of the rectangle, and the position of the rectangle in the image 30.

[0142] The target region information can be prepared, for example, by analyzing one or more images 30 acquired by the extraction unit 110 and multiple images obtained in advance in the same manner. Specifically, a region of interest is extracted from each of the multiple images obtained in advance, and pixels whose frequency of inclusion in the region of interest is equal to or greater than a predetermined standard are identified. The set of identified pixels is then designated as the extraction target region.

[0143] The target region information is stored in a storage unit (for example, storage device 1080) that is accessible from extraction unit 110. Extraction unit 110 can read the target region information from this storage unit and use it to extract partial image 32.

[0144] In this embodiment, the noise information generating unit 130 generates noise information in the same manner as in the first embodiment, using one or more partial images 32 extracted by the extracting unit 110 based on the extraction target region.

[0145] The noise information thus obtained indicates the noise distribution in the extraction target region, i.e., the region where the region of interest is likely to occur. By using this noise information, it is possible to reduce noise in the region of the target image where the region of interest is likely to occur.

[0146] In this embodiment, the same actions and effects as those of the first embodiment can be obtained. Additionally, in this embodiment, the extraction unit 110 extracts one or more partial images 32 based on an area of ​​one or more images 30 where a region of interest is likely to occur. By limiting the area and performing processing in this manner, it is possible to reduce the number of partial images 32 and the number of pixels in the partial images 32. As a result, it is possible to reduce the processing load on the information processing device 10.

[0147] Sixth Embodiment An information processing device 10 according to a sixth embodiment is the same as the information processing device 10 according to the fifth embodiment, except for the points described below. An imaging system 20 according to the sixth embodiment is the same as the imaging system 20 according to the fifth embodiment, except for the points described below. In the sixth embodiment, an extraction unit 110 extracts one or more partial images 32 based on the detected region of interest.

[0148] In the fifth embodiment, the extraction target region is determined as a region where a region of interest is likely to occur, whereas in the sixth embodiment, the extraction target region is determined by a detected region of interest. Here, the "detected region of interest" may be a region of interest detected in at least one of the one or more images 30, or may be a region of interest detected in the target image. The extraction unit 110 extracts one or more partial images 32 from the one or more images 30 based on the region of interest.

[0149] 16 is a flowchart illustrating a process flow in which the extraction unit 110 extracts one or more partial images 32 based on the detected region of interest. In the example of Fig. 16, the information processing device 10 includes a noise reduction unit 150 and an authentication unit 170, similar to the example shown in Fig. 14.

[0150] In S401, the authentication unit 170 acquires a target image. The target image is an image to be authenticated, as described in the fourth embodiment. The authentication unit 170 may acquire the target image from an imaging device equipped with a target imaging sensor, from a device other than the information processing device 10, or by reading it from a storage unit provided inside or outside the information processing device 10. If this storage unit is provided inside the information processing device 10, this storage unit is realized by the storage device 1080 of the computer 1000 that realizes the information processing device 10.

[0151] In S402 following S401, the authentication unit 170 detects an area of ​​interest in the target image. The area of ​​interest is an area used for authentication, such as a face, iris, fingerprint, veins, palm print, or the area around the eyes. This detected area of ​​interest is set as the extraction target area. In other words, the authentication unit 170 generates information indicating the detected area of ​​interest as target area information.

[0152] In S403, the extraction unit 110 and the noise information generation unit 130 generate noise information by performing the processes from S101 to S107 described in the first embodiment. Here, the extraction unit 110 acquires a predetermined number of the most recent images acquired by the target imaging sensor as the one or more images 30, for example, and uses these images to extract the partial image 32. The one or more images 30 used to extract the partial image 32 may include the target image.

[0153] In S404, the noise reduction unit 150 performs noise reduction processing on the target image using the noise information generated in S403. In this way, noise in the target image is reduced. The noise reduction processing is as described in the second embodiment.

[0154] In S405, the authentication unit 170 performs authentication processing using the target image in which noise in the attention area has been reduced. The authentication processing is as described in the fourth embodiment. Since noise in the attention area used for authentication in the target image is reduced, authentication processing with little influence of noise becomes possible. Furthermore, by limiting the area in which noise reduction is performed, the processing load on the information processing device 10 can be reduced.

[0155] Although an example in which the information processing device 10 performs authentication processing on a target image has been described with reference to FIG. 16, the processing of the information processing device 10 is not limited to this example.

[0156] For example, the information processing device 10 may perform an inspection process instead of an authentication process. In this case, an image obtained by capturing an image of an inspection target is used as a target image. The area to be inspected is used as a region of interest. Then, the information processing device 10 performs an inspection process, such as detecting foreign matter or defects, using the target image in which noise in the region of interest has been reduced.

[0157] In this embodiment, the same effects and advantages as those of the first embodiment can be obtained. In addition, in this embodiment, the extraction unit 110 extracts one or more partial images 32 based on the detected region of interest. Therefore, it is possible to estimate noise in a necessary region and reduce the processing load on the information processing device 10.

[0158] Seventh Embodiment An information processing device 10 according to the seventh embodiment is the same as the information processing device 10 according to at least one of the first to sixth embodiments, except for the points described below. An imaging system 20 according to the seventh embodiment is the same as the imaging system 20 according to at least one of the first to sixth embodiments, except for the points described below. In the seventh embodiment, the extraction unit 110 identifies the shapes of one or more partial images 32 based on estimated noise characteristics. In this way, highly accurate noise information can be obtained.

[0159] The characteristics of the fixed pattern noise that occurs may vary depending on the type of sensor used as the target imaging sensor. For example, stripe-shaped fixed pattern noise may occur depending on the readout direction of the image sensor used as the target imaging sensor. When the image sensor reads out vertically in columns, vertical stripe-shaped fixed pattern noise is likely to occur. When the image sensor reads out horizontally in rows, horizontal stripe-shaped fixed pattern noise is likely to occur.

[0160] FIG. 17 is a diagram illustrating an example of the relationship between striped fixed pattern noise and the shape of a partial image 32. In FIG. 17, the shape of the partial image 32 is indicated by a thick rectangle. While FIG. 17 illustrates the relationship between striped fixed pattern noise in an average image 31 and the shape of the partial image 32, the relationship between striped fixed pattern noise in one or more images 30 and the shape of the partial image 32 is also illustrated. In this embodiment, the one or more images 30 are images obtained using an image sensor. The shape of the one or more partial images 32 is a rectangle whose short side corresponds to the readout direction of the image sensor.

[0161] That is, if vertical stripe fixed pattern noise is likely to occur in the image, the extraction unit 110 sets the shape of the partial image 32 to a rectangle with its long sides extending in the horizontal direction of the image. On the other hand, if horizontal stripe fixed pattern noise is likely to occur in the image, the extraction unit 110 sets the shape of the partial image 32 to a rectangle with its long sides extending in the vertical direction of the image. By doing so, the relationship between pixel values ​​between stripes can be effectively reflected in the estimation result.

[0162] FIG. 18 is a flowchart illustrating the flow of processing by the information processing device 10 to acquire information from the target imaging sensor and identify the shape of the partial image 32.

[0163] In S501, the extraction unit 110 acquires information about the target imaging sensor. The extraction unit 110 can acquire information about the target imaging sensor based on, for example, an operation performed by a user on the information processing device 10. The user can input information to the information processing device 10 using a keyboard, a mouse, or the like connected to a computer 1000 that implements the information processing device 10. For example, the user specifies information about the target imaging sensor by selecting from options displayed on a display connected to the information processing device 10. The extraction unit 110 accepts the information about the target imaging sensor input to the information processing device 10.

[0164] When the information processing device 10 is connected to an imaging device that includes a target imaging sensor, the extraction unit 110 may acquire information about the target imaging sensor from the imaging device.

[0165] The information about the target imaging sensor may be information indicating the type of the target imaging sensor or information indicating estimated noise characteristics. Examples of the type of the target imaging sensor include a CMOS sensor, a CCD sensor, and the relationship between the long side direction of the image and the readout direction (perpendicular or parallel). Examples of estimated noise characteristics include the direction of estimated stripe noise (perpendicular or parallel to the long side of the image).

[0166] In S502, the extraction unit 110 identifies the shape of the partial image 32 based on information about the target imaging sensor. A storage unit (e.g., storage device 1080) accessible from the extraction unit 110 holds reference information that associates information about the target imaging sensor with information indicating the shape of the partial image 32. The extraction unit 110 can use the reference information to identify the shape of the partial image 32. That is, the extraction unit 110 identifies, among the reference information, information indicating the shape of the partial image 32 that is associated with the information about the target imaging sensor acquired in S501 as the shape to be adopted.

[0167] In S503, the extraction unit 110 acquires one or more images 30. Then, the extraction unit 110 and the noise information generation unit 130 generate noise information by performing the processes from S101 to S107 described in the first embodiment. Here, when the extraction unit 110 extracts one or more partial images 32 in S503, the extraction unit 110 extracts partial images 32 of the shape identified in S502.

[0168] In this embodiment, the same functions and effects as those in the first embodiment can be obtained. In addition, in this embodiment, the extraction unit 110 identifies the shapes of one or more partial images 32 based on the estimated noise characteristics. Therefore, accurate noise information can be obtained.

[0169] Eighth Embodiment An information processing device 10 according to an eighth embodiment is the same as the information processing device 10 according to at least one of the first to seventh embodiments, except for the points described below. An imaging system 20 according to the eighth embodiment is the same as the imaging system 20 according to at least one of the first to seventh embodiments, except for the points described below.

[0170] Fig. 19 is a diagram illustrating an information processing device 10 and an imaging system 20 each including a control unit 190. The imaging device 290 is a camera including a target imaging sensor. In the example of Fig. 19, the information processing device 10 includes the control unit 190. In the eighth embodiment, the control unit 190 controls the imaging device 290. Specifically, the control unit 190 controls the focus of the imaging device 290.

[0171] The hardware configuration of the computer that realizes this information processing device 10 is similar to that of the information processing device 10 according to the first embodiment and is shown in Fig. 2. However, the storage device 1080 of the computer 1000 that realizes the information processing device 10 according to this embodiment further stores a program module that realizes the functions of the control unit 190.

[0172] In this embodiment, the control unit 190 controls the imaging device 290 to switch between an in-focus state and an out-of-focus state. For example, when the information processing device 10 is to generate noise information, the control unit 190 sets the imaging device 290 to an out-of-focus state. The extraction unit 110 acquires one or more images 30 captured by the imaging device 290 in that state. The extraction unit 110 and the noise information generation unit 130 then generate noise information using the acquired one or more images 30.

[0173] One or more images 30 captured out of focus will be blurred, which reduces the influence of the captured scene in each image 30. As a result, fixed pattern noise can be estimated with high accuracy.

[0174] The control unit 190 keeps the imaging device 290 in focus except when obtaining one or more images 30 for generating noise information.

[0175] It is preferable that images captured out of focus are not used for monitoring, inspection, authentication, or the like.

[0176] In this embodiment, the same functions and effects as those in the first embodiment can be obtained. In addition, in this embodiment, the control unit 190 of the information processing device 10 generates noise information using one or more images 30 captured in an out-of-focus state. Therefore, fixed pattern noise can be estimated with high accuracy.

[0177] Ninth Embodiment An information processing device 10 according to a ninth embodiment is the same as the information processing device 10 according to at least one of the first to seventh embodiments, except for the points described below. An imaging system 20 according to a ninth embodiment is the same as the imaging system 20 according to at least one of the first to seventh embodiments, except for the points described below.

[0178] FIG. 20 is a diagram showing an example of an imaging system 20 equipped with a diffuser. In the example of FIG. 20, a diffuser 210 is provided in front of the lens of an imaging device 290. The imaging device 290 is a camera equipped with a target imaging sensor. In the example of FIG. 20, the information processing device 10 includes a control unit 190. In the ninth embodiment, the control unit 190 controls the diffuser 210. That is, when the imaging device 290 captures an image, the control unit 190 can switch between a state in which the diffuser 210 is disposed in front of the lens and a state in which the diffuser 210 is not disposed in front of the lens. The diffuser 210 has a function of diffusing light.

[0179] The hardware configuration of the computer that realizes this information processing device 10 is similar to that of the information processing device 10 according to the first embodiment and is shown in Fig. 2. However, the storage device 1080 of the computer 1000 that realizes the information processing device 10 according to this embodiment further stores a program module that realizes the functions of the control unit 190.

[0180] In this embodiment, the control unit 190 controls the control unit 190 so that one or more images 30 captured with the diffuser 210 placed in front of the lens are used for noise estimation in the information processing device 10. For example, when the information processing device 10 is to generate noise information, the control unit 190 places the diffuser 210 in front of the lens of the imaging device 290. The extraction unit 110 acquires one or more images 30 captured by the imaging device 290 in this state. Then, the extraction unit 110 and the noise information generation unit 130 generate noise information using the acquired one or more images 30.

[0181] One or more images 30 captured with the diffuser 210 placed in front of the lens will be blurred, i.e., the influence of the captured scene in each image 30 will be reduced. As a result, fixed pattern noise can be estimated with high accuracy.

[0182] The control unit 190 does not place the diffuser 210 in front of the lens except when one or more images 30 for generating noise information are obtained.

[0183] It is preferable that images captured with the diffuser 210 placed in front of the lens are not used for monitoring, inspection, authentication, or the like.

[0184] In this embodiment, the same functions and effects as those in the first embodiment can be obtained. In addition, in this embodiment, the control unit 190 of the information processing device 10 generates noise information using one or more images 30 captured with the diffuser 210 placed in front of the lens. Therefore, fixed pattern noise can be estimated with high accuracy.

[0185] (Tenth embodiment) An information processing device 10 according to a tenth embodiment is the same as the information processing device 10 according to at least one of the first to seventh embodiments, except for the points described below. An imaging system 20 according to a tenth embodiment is the same as the imaging system 20 according to at least one of the first to seventh embodiments, except for the points described below.

[0186] Fig. 21 is a diagram showing an example of an imaging system 20 including a driving unit 220. In the example of Fig. 21, an imaging device 290 can be driven by the driving unit 220. The imaging device 290 is a camera including a target imaging sensor. In the example of Fig. 21, the information processing device 10 includes a control unit 190.

[0187] In the tenth embodiment, the control unit 190 controls the driving unit 220. For example, the driving unit 220 is a support unit that can change the angle of the imaging device 290, and the control unit 190 can control the orientation of the imaging device 290. Specifically, the control unit 190 can control the pan and tilt of the imaging device 290. Alternatively, the driving unit 220 may be a stage that can drive the imaging device 290 in at least one direction, and the control unit 190 may be able to control the position of the imaging device 290 in addition to or instead of the orientation of the imaging device 290.

[0188] The hardware configuration of the computer that realizes this information processing device 10 is similar to that of the information processing device 10 according to the first embodiment and is shown in Fig. 2. However, the storage device 1080 of the computer 1000 that realizes the information processing device 10 according to this embodiment further stores a program module that realizes the functions of the control unit 190.

[0189] The driving unit 220 is communicably connected to the information processing device 10 via an input / output interface 1100 or a network interface 1120 of the computer 1000 that implements the information processing device 10 .

[0190] In this embodiment, the control unit 190 controls the control unit 190 so that one or more images 30 captured while changing at least one of the orientation and position of the imaging device 290 are used for noise estimation in the information processing device 10. For example, when the information processing device 10 is to generate noise information, the control unit 190 changes at least one of the orientation and position of the imaging device 290. The extraction unit 110 acquires one or more images 30 captured by the imaging device 290 while at least one of the orientation and position is changing. Then, the extraction unit 110 and the noise information generation unit 130 generate noise information using the acquired one or more images 30.

[0191] Blurring occurs in one or more images 30 captured while changing at least one of the orientation and position of the imaging device 290. In other words, the influence of the captured scene is reduced in each image 30. As a result, fixed pattern noise can be estimated with high accuracy.

[0192] It is preferable that the imaging device 290 changes at least one of the orientation and position of the imaging device 290 at a speed that causes blurring of the image. The control unit 190 may change at least one of the orientation and position of the imaging device 290 so as to vibrate it with a small amplitude within a range that allows one or more obtained images 30 to be considered to be of the same captured scene.

[0193] It is preferable that images used for monitoring, inspection, authentication, etc. are not blurred.

[0194] In this embodiment, the same actions and effects as those in the first embodiment can be obtained. Additionally, in this embodiment, the control unit 190 controls the control unit 190 so that one or more images 30 captured while changing at least one of the orientation and position of the imaging device 290 are used for noise estimation in the information processing device 10. Therefore, fixed pattern noise can be estimated with high accuracy.

[0195] Although this disclosure has been described above with reference to the embodiments, this disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of this disclosure within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0196] In addition, although the flowcharts used in the above description show a sequence of steps (processes), the order of steps executed in each embodiment is not limited to the sequence shown in the flowcharts. In each embodiment, the order of steps shown in the diagrams can be changed as long as it does not cause any problems in terms of the content.

[0197] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. 1-1. An information processing device comprising: an extraction means for extracting, based on one or more images, one or more partial images each having a smaller number of pixels than an image included in the one or more images; and a noise information generation means for generating, using the one or more partial images, information indicating noise included in the one or more images, wherein the noise information generation means generates, for each partial image included in the one or more partial images, relationship information indicating relative values ​​of each of a plurality of pixel values ​​included in the partial image, with reference to a pixel value of a reference pixel included in the partial image, and generates the information indicating the noise using the generated relationship information. 1-2. The information processing device described in 1-1., wherein the one or more images are images obtained using the same imaging sensor, and the noise information generation means generates the information indicating the noise for each combination of temperature and exposure time of the imaging sensor. 1-3. The information processing device described in 1-1. or 1-2., wherein the extraction means extracts the one or more partial images based on an area of ​​the one or more images that is likely to contain a region of interest. 1-4. The information processing device described in any one of 1-1. to 1-3., further comprising noise reduction means for performing noise reduction processing on a target image using information indicating the noise. 1-5. The information processing device described in any one of 1-1. to 1-4., wherein the noise information generation means generates information indicating the noise for each of red, green, and blue. 1-6. The information processing device described in any one of 1-1. to 1-5., wherein the extraction means generates an average image by averaging the one or more images obtained by capturing images at different times, and extracts the one or more partial images from the generated average image. 1-7. The information processing device described in any one of 1-1. to 1-6., wherein the extraction means identifies the shape of the one or more partial images based on estimated noise characteristics.1-8. The information processing device described in 1-1. or 1-2., wherein the extraction means extracts the one or more partial images based on a detected region of interest. 1-9. The information processing device described in 1-1., wherein the extraction means uses a plurality of images of different imaging scenes and extracts the one or more partial images for each imaging scene. 1-10. The information processing device described in 1-9., wherein the noise information generation means generates the relationship information for each imaging scene, and estimates a noise change distribution in the one or more images using a plurality of pieces of relationship information obtained from a plurality of partial images that are at the same position in the one or more images. 1-11. The information processing device described in 1-10., wherein the noise information generation means generates statistical relationship information for each position in the one or more images by performing statistical processing on the plurality of pieces of relationship information, and estimates the noise change distribution using the generated statistical relationship information. 1-12. The information processing device described in 1-11. 1-13. The information processing device described in 1-11. or 1-12., wherein the statistical processing is processing to average the relative values ​​for each pixel, or processing to identify the most frequent value of the relative values. 1-13. The information processing device described in 1-11. or 1-12., wherein the noise information generation means generates estimated noise distribution information as information indicating the noise by adding an offset value to the entire noise variation distribution. 1-14. The information processing device described in 1-13., wherein the noise information generation means identifies a lowest pixel having a lowest pixel value in a reference image, and identifies the offset value so that the value of a pixel corresponding to the lowest pixel in the noise variation distribution matches the pixel value of the lowest pixel. 1-15. The information processing device described in 1-8., wherein the one or more images are images obtained using an image sensor, and the shape of the one or more partial images is a rectangle with its shorter side in a direction corresponding to the readout direction of the image sensor.1-16. The information processing device described in 1-4., further comprising authentication means for performing authentication using the target image that has been subjected to the noise reduction processing. 2-1. An information processing method in which one or more computers extract, based on one or more images, one or more partial images having a smaller number of pixels than one image included in the one or more images, generate relationship information for each partial image included in the one or more partial images indicating relative values ​​of each of a plurality of pixel values ​​included in the partial image, with the pixel value of a reference pixel included in the partial image as a reference, and generate information indicating noise included in the one or more images using the generated relationship information. 2-2. The information processing method described in 2-1., in which the one or more images are images obtained using the same imaging sensor, and the one or more computers generate information indicating the noise for each combination of temperature and exposure time of the imaging sensor. 2-3. The information processing method described in 2-1. or 2-2., in which the one or more computers extract the one or more partial images based on an area of ​​the one or more images that is likely to contain a region of interest. 2-4. 2-1. The information processing method described in any one of 2-1. to 2-3., wherein the one or more computers further perform noise reduction processing on the target image using the information indicating the noise. 2-5. The information processing method described in any one of 2-1. to 2-4., wherein the one or more computers generate information indicating the noise for each of red, green, and blue. 2-6. The information processing method described in any one of 2-1. to 2-5., wherein the one or more computers generate an average image by averaging the one or more images obtained by capturing images at different times, and extract the one or more partial images from the generated average image. 2-7. The information processing method described in any one of 2-1. to 2-6., wherein the one or more computers identify the shape of the one or more partial images based on estimated noise characteristics.2-8. The information processing method described in 2-1. or 2-2., wherein the one or more computers extract the one or more partial images based on the detected region of interest. 2-9. The information processing method described in 2-1., wherein the one or more computers use a plurality of images of different captured scenes and extract the one or more partial images for each captured scene. 2-10. The information processing method described in 2-9., wherein the one or more computers generate the relationship information for each captured scene, and estimate the distribution of noise change in the one or more images using a plurality of pieces of relationship information obtained from a plurality of the partial images that are at the same position in the one or more images. 2-11. The information processing method described in 2-10., wherein the one or more computers generate statistical relationship information for each position in the one or more images by performing statistical processing on the plurality of pieces of relationship information, and estimate the distribution of noise change using the generated statistical relationship information. 2-12. 2-11. The information processing method described in 2-11, wherein the statistical processing is processing of averaging the relative values ​​for each pixel, or processing of identifying the most frequent value of the relative values. 2-13. The information processing method described in 2-11. or 2-12, wherein the one or more computers generate estimated noise distribution information as information indicating the noise by adding an offset value to the entire noise variation distribution. 2-14. The information processing method described in 2-13, wherein the one or more computers identify a lowest pixel having the lowest pixel value in a reference image, and identify the offset value so that the value of a pixel corresponding to the lowest pixel in the noise variation distribution matches the pixel value of the lowest pixel. 2-15. The information processing method described in 2-8, wherein the one or more images are images obtained using an image sensor, and the shape of the one or more partial images is a rectangle with its shorter side in a direction corresponding to the readout direction of the image sensor.2-16. An information processing method according to 2-4., wherein the one or more computers further perform authentication using the target image that has been subjected to the noise reduction processing. 3-1. A program that causes a computer to function as: an extraction means that extracts, based on one or more images, one or more partial images having a smaller number of pixels than one image included in the one or more images; and a noise information generation means that generates information indicative of noise included in the one or more images using the one or more partial images, wherein the noise information generation means generates, for a partial image included in the one or more partial images, relationship information that indicates the relative values ​​of each of a plurality of pixel values ​​included in the partial image, using the pixel value of a reference pixel included in the partial image as a reference, and generates the information indicative of the noise using the generated relationship information. 3-2. A program according to 3-1., wherein the one or more images are images obtained using the same imaging sensor, and the noise information generation means generates the information indicative of the noise for each combination of temperature and exposure time of the imaging sensor. 3-3. 3-1. or 3-2. 3-4. A program according to any one of 3-1. to 3-3., wherein the extraction means extracts the one or more partial images based on an area from the one or more images that is likely to contain a region of interest. 3-4. A program according to any one of 3-1. to 3-3., wherein the computer is further caused to function as noise reduction means that performs noise reduction processing on a target image using information indicating the noise. 3-5. A program according to any one of 3-1. to 3-4., wherein the noise information generation means generates information indicating the noise for each of red, green, and blue. 3-6. A program according to any one of 3-1. to 3-5., wherein the extraction means generates an average image by averaging the one or more images obtained by capturing images at different times, and extracts the one or more partial images from the generated average image. 3-7. A program according to any one of 3-1. to 3-6., wherein the extraction means identifies the shape of the one or more partial images based on estimated noise characteristics.3-8. A program described in 3-1. or 3-2., wherein the extraction means extracts the one or more partial images based on a detected region of interest. 3-9. A program described in 3-1., wherein the extraction means uses a plurality of images of different captured scenes and extracts the one or more partial images for each captured scene. 3-10. A program described in 3-9., wherein the noise information generation means generates the relationship information for each captured scene, and estimates a noise change distribution in the one or more images using a plurality of pieces of relationship information obtained from a plurality of partial images that are at the same position in the one or more images. 3-11. A program described in 3-10., wherein the noise information generation means generates statistical relationship information for each position in the one or more images by performing statistical processing on the plurality of pieces of relationship information, and estimates the noise change distribution using the generated statistical relationship information. 3-12. A program described in 3-11. 3-13. A program according to claim 3-11, wherein the statistical processing is a process of averaging the relative values ​​for each pixel, or a process of identifying the most frequent value of the relative values. 3-13. A program according to claim 3-11 or 3-12, wherein the noise information generation means generates estimated noise distribution information as information indicating the noise by adding an offset value to the entire noise variation distribution. 3-14. A program according to claim 3-13, wherein the noise information generation means identifies a lowest pixel having the lowest pixel value in a reference image, and identifies the offset value so that the value of a pixel corresponding to the lowest pixel in the noise variation distribution matches the pixel value of the lowest pixel. 3-15. A program according to claim 3-8, wherein the one or more images are images obtained using an image sensor, and the shape of the one or more partial images is a rectangle with its shorter side in a direction corresponding to the readout direction of the image sensor.3-16. A program according to 3-4., further causing the computer to function as authentication means for performing authentication using the target image that has been subjected to the noise reduction processing. 4-1. A computer-readable recording medium having a program recorded thereon, wherein the program causes a computer to function as: extraction means for extracting, based on one or more images, one or more partial images having a smaller number of pixels than one image included in the one or more images; and noise information generation means for generating information indicating noise included in the one or more images using the one or more partial images, wherein the noise information generation means generates, for a partial image included in the one or more partial images, relationship information indicating the relative values ​​of each of a plurality of pixel values ​​included in the partial image, using the pixel value of a reference pixel included in the partial image as a reference, and generates the information indicating the noise using the generated relationship information. 4-2. A computer-readable recording medium having a program according to any one of 3-1. to 3-16. recorded thereon. 5-1. 5-2. An imaging system comprising: an imaging sensor; and an information processing device, wherein the information processing device comprises: extraction means for extracting, based on one or more images obtained using the imaging sensor, one or more partial images having a smaller number of pixels than an image included in the one or more images; and noise information generation means for generating information indicating noise included in the one or more images using the one or more partial images, wherein the noise information generation means generates, for a partial image included in the one or more partial images, relationship information indicating relative values ​​of each of a plurality of pixel values ​​included in the partial image, with reference to a pixel value of a reference pixel included in the partial image, and generates information indicating the noise using the generated relationship information. 5-2. An imaging system comprising: an imaging sensor; and the information processing device described in any one of 1-1. to 1-16.6-1. An imaging method in which one or more images are captured with an imaging sensor, and one or more computers extract, based on the one or more images, one or more partial images having a smaller number of pixels than one of the images included in the one or more images, and for each partial image included in the one or more partial images, generate relationship information indicating the relative values ​​of each of a plurality of pixel values ​​included in the partial image, using the pixel value of a reference pixel included in the partial image as a reference, and generate information indicating noise included in the one or more images using the generated relationship information. 6-2. An imaging method in which one or more images are captured with an imaging sensor, and the one or more computers execute the information processing method described in any one of 2-1 to 2-16 using the one or more images.

[0198] Furthermore, some or all of the configurations described in Supplementary Notes 1-2 to 1-16, which are dependent on Supplementary Note 1-1, may also be dependent on Supplementary Notes 4-1, 5-1, and 6-1 in the same dependency relationship as Supplementary Notes 1-2 to 1-16. Furthermore, not limited to Supplementary Notes 4-1, 5-1, and 6-1, some or all of the configurations described as supplementary notes may be similarly made dependent on various hardware, software, various recording means for recording software, or systems, within the scope of each of the above-mentioned embodiments.

[0199] REFERENCE SIGNS LIST 10 Information processing device 20 Imaging system 30 Image 31 Average image 32 Partial image 33 Relationship information 34 Statistical relationship information 35 Change amount distribution information 100 Determination unit 101 Authentication information storage unit 110 Extraction unit 130 Noise information generation unit 150 Noise reduction unit 170 Authentication unit 190 Control unit 200 Imaging sensor 201 Temperature sensor 210 Diffusion plate 220 Driving unit 290 Imaging device 1000 Computer

Claims

1. An information processing apparatus comprising: extraction means for extracting one or more partial images having a smaller number of pixels than one of the one or more images based on the one or more images; and noise information generation means for generating information indicating noise included in the one or more images using the one or more partial images, wherein the noise information generation means generates relationship information indicating relative values of a plurality of pixel values included in a partial image included in the one or more partial images with respect to the pixel value of a reference pixel included in the partial image, and generates information indicating the noise using the generated relationship information.

2. The information processing apparatus according to claim 1, wherein the one or more images are images obtained using the same imaging sensor, and the noise information generation means generates information indicating the noise for each combination of the temperature and exposure time of the imaging sensor.

3. The information processing apparatus according to claim 1 or 2, wherein the extraction means extracts the one or more partial images based on an area in which a region of interest is likely to occur among the one or more images.

4. The information processing apparatus according to any one of claims 1 to 3, further comprising noise reduction means for performing noise reduction processing on a target image using the information indicating the noise.

5. The information processing apparatus according to any one of claims 1 to 4, wherein the noise information generation means generates information indicating the noise for each of red, green, and blue.

6. The information processing apparatus according to any one of claims 1 to 5, wherein the extraction means generates an average image obtained by averaging the one or more images captured at different timings, and extracts the one or more partial images from the generated average image.

7. The information processing apparatus according to any one of claims 1 to 6, wherein the extraction means specifies the shape of the one or more partial images based on an estimated noise characteristic.

8. The information processing apparatus according to claim 1 or 2, wherein the extraction means extracts the one or more partial images based on a detected region of interest.

9. The information processing apparatus according to claim 1, wherein the extraction means uses a plurality of images with different imaging scenes and extracts the one or more partial images for each imaging scene.

10. The information processing apparatus according to claim 9, wherein the noise information generation means generates the relationship information for each imaging scene, and estimates the change amount distribution of noise in the one or more images using a plurality of the relationship information obtained from a plurality of the partial images having the same positions in the one or more images.

11. The information processing apparatus according to claim 10, wherein the noise information generation means generates statistical relationship information for each position in the one or more images by performing statistical processing on the plurality of relationship information, and estimates the change amount distribution of noise using the generated statistical relationship information.

12. The information processing apparatus according to claim 11, wherein the statistical processing is a process of averaging the relative values for each pixel or a process of specifying the mode value of the relative values.

13. The information processing apparatus according to claim 11 or 12, wherein the noise information generation means generates estimated noise distribution information as information indicating the noise by adding an offset value to the entire change amount distribution of the noise.

14. The information processing apparatus according to claim 13, wherein the noise information generation means specifies a lowest pixel having the lowest pixel value in a reference image, and specifies the offset value so that the value of the pixel corresponding to the lowest pixel in the change amount distribution of the noise matches the pixel value of the lowest pixel.

15. The information processing apparatus according to claim 8, wherein the one or more images are images obtained using an image sensor, and the shape of the one or more partial images is a rectangle having a direction corresponding to the readout direction of the image sensor as a short side.

16. The information processing apparatus according to claim 4, further comprising authentication means for performing authentication using the target image on which the noise reduction processing has been performed.

17. An information processing method in which one or more computers extract one or more partial images having a smaller number of pixels than one of the images included in the one or more images based on the one or more images, and for the partial images included in the one or more partial images, generate relationship information indicating relative values of a plurality of pixel values included in the partial image with respect to the pixel value of a reference pixel included in the partial image, and generate information indicating noise included in the one or more images using the generated relationship information.

18. A computer-readable recording medium recording a program, wherein the program causes a computer to function as extraction means for extracting one or more partial images having a smaller number of pixels than one of the images included in the one or more images based on the one or more images, and noise information generation means for generating information indicating noise included in the one or more images using the one or more partial images, and the noise information generation means generates relationship information indicating relative values of a plurality of pixel values included in the partial image with respect to the pixel value of a reference pixel included in the partial image for the partial images included in the one or more partial images, and generates the information indicating the noise using the generated relationship information.

19. An imaging system including an imaging sensor and an information processing apparatus, wherein the information processing apparatus includes extraction means for extracting one or more partial images having a smaller number of pixels than one of the images included in the one or more images based on the one or more images obtained using the imaging sensor, and noise information generation means for generating information indicating noise included in the one or more images using the one or more partial images, and the noise information generation means generates relationship information indicating relative values of a plurality of pixel values included in the partial image with respect to the pixel value of a reference pixel included in the partial image for the partial images included in the one or more partial images, and generates the information indicating the noise using the generated relationship information.

20. An imaging method, comprising: capturing one or more images with an imaging sensor; extracting, by one or more computers, one or more partial images having a pixel count less than that of one of the images included in the one or more images based on the one or more images; generating, for the partial images included in the one or more partial images, relationship information indicating relative values of a plurality of pixel values included in the partial image with respect to a pixel value of a reference pixel included in the partial image as a reference; and generating information indicating noise included in the one or more images using the generated relationship information.

Images