Information processing device, information processing method, and program

Abstract

A polarized light rendering setting unit 30 sets a plurality of parameters used for generating a polarized light rendering image of an abnormality detection target. A polarized light rendering image generation unit 40 generates, on the basis of the parameters set by the polarized light rendering setting unit 30, the polarized light rendering image of the abnormality detection target. An abnormality detection unit 50 detects an abnormal region of the abnormality detection target, on the basis of a difference between a captured polarized-light image acquired by imaging the abnormality detection target and the polarized light rendering image generated by the polarized light rendering image generation unit 40. An abnormality difficult to be detected on the basis of luminance information or color information can be detected on the basis of the polarized-light image.

Description

[Technical Field]

[0001] This technology relates to an information processing device, an information processing method, and a program, and enables detection of abnormalities that do not appear in color information or luminance information. [Background technology]

[0002] Conventionally, brightness information and color information shown in captured images have been used to detect abnormalities. For example, in Patent Document 1, defects in a target object are detected based on brightness information of each image contained in images captured by irradiating the target object with light from multiple directions. In addition, in Patent Document 2, a rendering image corresponding to the field of view of an imaging unit is generated, and image features of an image region corresponding to an object model in this rendering image are compared with image features of a change region extracted from the captured image to detect abnormalities in the object corresponding to the object model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2016-166842 [Patent Document 2] Japanese Patent Application Laid-Open No. 2013-211740 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when detecting an abnormality based on the brightness and color information shown in a captured image, it is difficult to detect an abnormality where there is little change in the brightness and color information. For example, if a deformation occurs on the front of an object, and the contour of the deformed part is smooth and there is little change in the brightness and color information, it is impossible to detect an abnormality from the front of the object.

[0005] Therefore, an object of this technology is to provide an information processing device, an information processing method, and a program that can detect abnormalities that are difficult to detect based on luminance information or color information. [Means for solving the problem]

[0006] The first aspect of this technology is a polarization rendering setting unit that sets a plurality of parameters used to generate a polarization rendering image of an abnormality detection target; a polarization rendering image generation unit that generates a polarization rendering image of the abnormality detection target based on the parameters set by the polarization rendering setting unit; and an abnormality detection unit that detects an abnormal region of the abnormality detection target based on a difference between a polarized captured image acquired by capturing an image of the abnormality detection target and the polarized rendering image generated by the polarized rendering image generation unit; The information processing device includes:

[0007] In this technology, the polarization rendering setting unit sets multiple parameters to be used in generating a polarization rendering image of an abnormality detection target. The multiple parameters include light source parameters related to the light source, geometry parameters related to the abnormality detection target, material parameters related to the polarization characteristics of the abnormality detection target, and camera parameters of a polarization imaging image acquisition unit that acquires the polarization imaging image. For example, when multiple parameters are measured, the polarization rendering setting unit sets the measured multiple parameters as parameters to be used in generating the polarization rendering image. Furthermore, when, for example, some or all of the multiple parameters are not measured, the polarization rendering setting unit performs an optimization process for the unmeasured parameters. In the parameter optimization process, the unmeasured parameters are set as parameters that can minimize the difference between the polarization imaging image and the polarization rendering image generated by the polarization imaging image generation unit. The polarization rendering setting unit, for example, repeatedly updates the parameters using the difference, and sets the converged parameter values ​​as optimized parameter values, thereby enabling adjustment of the parameter convergence characteristics. The polarization rendering setting unit sets the measured parameters and parameters calculated by the optimization process, or, if no measured parameters are available, the parameters calculated by the optimization process, as parameters to be used in generating the polarization rendering image. At least the material parameters may be measured in advance, and the polarization rendering setting unit may use the parameters measured in advance as fixed parameters.

[0008] The polarization rendering image generating unit generates a polarization rendering image of the abnormality detection target based on the parameters set by the polarization rendering setting unit.

[0009] The anomaly detection unit detects an abnormal region of the anomaly detection target on a pixel-by-pixel basis based on the difference between a polarized captured image acquired by capturing an image of the anomaly detection target and a polarized rendering image generated by the polarized rendering image generation unit. The anomaly detection unit may also estimate the cause of the anomaly in the detected abnormal region based on information indicating whether the parameter is a measured parameter or a parameter calculated by an optimization process. Furthermore, the anomaly detection unit may detect an abnormal region of the anomaly detection target based on the difference in polarization information using polarization information calculated from the polarized captured image and polarization information calculated from the polarized rendering image.

[0010] The second aspect of this technology is setting a plurality of parameters used to generate a polarization rendering image of an abnormality detection target in a polarization rendering setting unit; generating a polarization rendering image of the abnormality detection target by a polarization rendering image generation unit based on the parameters set by the polarization rendering setting unit; an abnormality detection unit detects an abnormal region of the abnormality detection target based on a difference between a polarized captured image acquired by capturing an image of the abnormality detection target and the polarized rendering image generated by the polarized rendering image generation unit. The present invention relates to an information processing method including the steps of:

[0011] The third aspect of this technology is A program for causing a computer to execute anomaly detection for an anomaly detection target, setting a plurality of parameters used to generate a polarization rendering image of the anomaly detection target; generating a polarization rendering image of the anomaly detection target based on the set plurality of parameters; detecting an abnormal region of the abnormality detection target based on a difference between a polarized captured image acquired by capturing an image of the abnormality detection target and the generated polarized rendering image; The program is executed by the computer.

[0012] The program of the present technology is, for example, a program that can be provided in a computer-readable format to a general-purpose computer capable of executing various program codes via a storage medium or communication medium, such as an optical disk, a magnetic disk, or a semiconductor memory, or a communication medium such as a network. By providing such a program in a computer-readable format, processing according to the program is realized on the computer. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing device. [Figure 2] FIG. 2 is a diagram illustrating the configuration of a polarization captured image acquisition unit. [Figure 3] FIG. 10 is a diagram illustrating pixel configurations for a plurality of polarization directions. [Figure 4] 10 is a flowchart illustrating an operation of the information processing device. [Figure 5] FIG. 2 is a diagram illustrating an example of the configuration of an environmental imaging unit. [Figure 6] 10A and 10B are diagrams illustrating examples of polarized images generated by the environmental imaging unit. [Figure 7] FIG. 10 is a diagram illustrating division of a polarization-captured image. [Figure 8] FIG. 10 is a diagram illustrating an example of a light source parameter L. [Figure 9] FIG. 10 is a diagram for explaining material polarization characteristics. [Figure 10] 10 is a flowchart (part 1) illustrating an operation for obtaining material parameters. [Figure 11] 10 is a flowchart (part 2) illustrating an example of the operation of acquiring material parameters. [Figure 12] FIG. 10 is a diagram showing a list of operation examples. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present technology will be described in the following order. 1. Polarized Rendering and This Technology 2. Configuration of the embodiment 3. Operation of the embodiment 3-1. Acquiring light source parameters 3-2. Obtaining Geometry Parameters 3-3. Obtaining material parameters 3-4. Acquiring camera parameters 3-5. Image difference and parameter optimization processing 3-6.About Anomaly Detection 4. Operational Example of the Embodiment

[0015] 1. Polarized Rendering and This Technology In polarization rendering, a rendering image (hereinafter referred to as a "polarized rendering image") is generated using polarization characteristics, etc. For example, a polarization rendering image I is generated based on the function f shown in Equation (1) using a light source parameter L, a geometry parameter G of the non-light source (subject), a material parameter M indicating the polarization characteristics of the non-light source, and a camera parameter C. f φ Generate.

[0016]

number

[0017] The light source parameter L is a vector of parameters indicating the position and direction of the light source and the Stokes vector of the irradiating light emitted from the light source. The non-light source geometry parameter G is a vector of parameters indicating the shape, position, and orientation of the subject. The non-light source material parameter M is a parameter indicating the Mueller matrix of the subject. The camera parameter C is a vector of the internal and external parameters of the camera. Furthermore, in equation (1), the parameter φ indicates the angle (also referred to as the polarization direction) of the linear polarizer (also simply referred to as the "polarizing plate") used when acquiring a polarized image (hereinafter referred to as a "polarized captured image") with the camera.

[0018] This technology generates a polarization rendering image I f φ and a polarized image I obtained by capturing an image using a camera equipped with a polarizing plate at an angle φ. r φ Using the polarized rendering image I f φ and polarized image I r φ The difference between the two is calculated for each region in a predetermined pixel unit, and a region where the difference is greater than a predetermined determination threshold is determined to be an abnormal region.

[0019] <2. Configuration of the embodiment> 1 illustrates an example of the configuration of an information processing device 10. The information processing device 10 includes a polarized captured image acquisition unit 20, a polarized rendering setting unit 30, a polarized rendering image generation unit 40, and an abnormality detection unit 50.

[0020] The polarized image acquisition unit 20 captures an image of an abnormality detection target and acquires a polarized image. FIG. 2 illustrates an example of the configuration of the polarized image acquisition unit. The polarized image acquisition unit 20 generates an image by capturing an image using a polarizing plate 202 configured with pixels of multiple polarization directions arranged on an image sensor 201, as shown in FIG. 2(a), for example. Note that FIG. 2(a) illustrates a case where the polarizing plate 202, each pixel of which has one of four different polarization directions (the polarization directions are indicated by arrows), is arranged in front of the image sensor 201. Alternatively, as shown in FIG. 2(b), the polarized image acquisition unit 20 may generate multiple polarized images with different polarization directions using a multi-lens array configuration. For example, multiple lenses 203 (four in the figure) are arranged in front of the image sensor 201, and each lens 203 forms an optical image of the subject on the imaging surface of the image sensor 201. Furthermore, a polarizing plate 204 is provided in front of each lens 203, and the polarization direction (the polarization direction is indicated by the arrow) of the polarizing plate 204 is set to a different direction, thereby generating a plurality of polarized images with different polarization directions. By configuring the polarized image acquisition unit 20 in this manner, a plurality of polarized images can be acquired in a single imaging session, thereby enabling rapid anomaly detection. Furthermore, as shown in FIG. 2(c), polarizing plates 212-1 to 212-4 with different polarization directions may be provided in front of the imaging units 210-1 to 210-4, thereby generating a plurality of polarized images with different polarization directions (the polarization direction is indicated by the arrow) from a plurality of different viewpoints.

[0021] In addition, when the movement of the target for abnormality detection is slow or when the target for abnormality recognition moves in a stepwise manner, a polarizing plate 211 may be provided in front of the imaging unit 210, as shown in (d) of Fig. 2. In this case, the polarizing plate 211 is rotated to capture images in a plurality of different polarization directions, thereby obtaining a plurality of polarized captured images with different polarization directions.

[0022] The different polarization directions can be any combination of angles as long as they are all different. For example, if three polarization directions are used, 0 degrees, 60 degrees, and 120 degrees are used, and if four polarization directions are used, 0 degrees, 45 degrees, 90 degrees, and 135 degrees are used.

[0023] When the image sensor 201 does not use a color filter, the polarization image acquisition unit 20 can acquire a luminance-polarized image. In the case of (a) in FIG. 2, an image equivalent to a normal-luminance unpolarized image can be acquired by averaging the luminance of four adjacent pixels with different polarization directions. In the cases of (b) and (c) in FIG. 2, if the positional spacing between the lenses 203 and the image capture units 210-1 to 210-4 is negligibly short relative to the distance to the abnormality detection target, parallax can be ignored between multiple polarization images with different polarization directions. Therefore, an image equivalent to a normal-luminance unpolarized image can be acquired by averaging the luminance of the polarization images with different polarization directions. In the case of (d) in FIG. 2, an image equivalent to a normal-luminance unpolarized image can be acquired by aligning the polarization images with different polarization directions according to the amount of parallax and averaging the luminance of the aligned polarization images.

[0024] Furthermore, the polarization image acquisition section 20 may generate not only a luminance polarization captured image but also a three-primary color image by providing a color filter to the image sensor 201, or may simultaneously generate an infrared image, etc. The polarization image acquisition section 20 may also generate a luminance image by calculating the luminance from the three-primary color image.

[0025] FIG. 3 illustrates pixel configurations for multiple polarization directions, with the configuration shown in FIG. 3 repeated in the horizontal and vertical directions. (a) and (b) of FIG. 3 illustrate pixel configurations for capturing monochrome images. (a) of FIG. 3 illustrates a case where a 2×2 pixel polarization pixel block is configured with polarization pixels having polarization directions (polarization angles) of 0 degrees, 45 degrees, 90 degrees, and 135 degrees, for example. (b) of FIG. 3 illustrates a case where a 4×4 pixel polarization pixel block is configured with polarization pixels having polarization directions of 0 degrees, 45 degrees, 90 degrees, and 135 degrees, for example, with 2×2 pixels as the polarization direction unit. When the polarization component unit of the polarizer is 2×2 pixels as shown in (b) of FIG. 3, the proportion of polarization components leaking in from adjacent polarization component regions with different polarization component units relative to the polarization components obtained for each polarization component unit is smaller than that for the 1×1 pixel shown in (a) of FIG. 3. Furthermore, when a polarizing plate uses a wire grid, polarized light whose electric field component is perpendicular to the direction of the grid (wire direction) is transmitted, and the longer the wire, the higher the transmittance. Therefore, when the polarization component unit is 2x2 pixels, the transmittance is higher than when it is 1x1 pixel. Therefore, when the polarization component unit is 2x2 pixels, the transmittance is higher than when it is 1x1 pixel, and the extinction ratio can be improved.

[0026] Figures 3(c) to 3(g) show examples of pixel configurations when acquiring a color image. Figure 3(c) shows a case where the 2x2 pixel polarization pixel block shown in Figure 3(a) is used as one color unit, and the three primary color pixels (red pixel, green pixel, red pixel) are arranged in a Bayer array.

[0027] FIG. 3(d) illustrates an example in which three primary color pixels are arranged in a Bayer array for each pixel block of 2×2 pixels with the same polarization direction shown in FIG. 3(b).

[0028] Figure 3(e) illustrates an example in which three primary color pixels are arranged in a Bayer array for each 2x2 pixel block with the same polarization direction, and 2x2 pixel blocks with different polarization directions are pixels of the same color.

[0029] (f) of Figure 3 shows a case where, for a 2x2 pixel block in a Bayer array with the same polarization direction, the phase difference in the polarization direction between horizontally adjacent pixel blocks is 90 degrees and the phase difference in the polarization direction between vertically adjacent pixel blocks is ±45 degrees.

[0030] (g) of Figure 3 shows a case where, for a 2x2 pixel block in a Bayer array with the same polarization direction, the phase difference in the polarization direction between vertically adjacent pixel blocks is 90 degrees and the phase difference in the polarization direction between horizontally adjacent pixel blocks is ±45 degrees.

[0031] In addition, the pixel configuration may be such that a 2x2 pixel block is composed of three primary color pixels and a white pixel, or a three primary color pixel block and a white pixel block. Furthermore, a 2x2 pixel block may be composed of polarized pixels and non-polarized pixels with different polarization directions, or a polarized pixel block and a non-polarized pixel block with different polarization directions. When using such a pixel configuration, the polarized captured image acquisition unit 20 generates a polarized captured image with a resolution in pixel units, i.e., in pixel block units of a predetermined number of pixels, for each polarization direction. Furthermore, if the pixel blocks are arranged in a Bayer array, a polarized captured image with a resolution in pixel units may be generated for each polarization direction using existing demosaic processing.

[0032] If multiple parameters used to generate a polarization rendering image have been measured, the polarization rendering setting unit 30 sets the measured parameters as the parameters used to generate the polarization rendering image. Furthermore, if some or all of the multiple parameters have not been measured, the polarization rendering setting unit 30 performs an optimization process for the unmeasured parameters and sets the measured parameters and parameters calculated through the optimization process, or, if no measured parameters exist, the parameters calculated through the optimization process, as the parameters used to generate the polarization rendering image. The polarization rendering setting unit 30 performs a parameter optimization process and calculates parameters that can minimize the difference between the polarization captured image and the polarization rendering image. For example, the polarization rendering setting unit uses the difference to repeatedly update the parameters and uses the converged parameter values ​​as the optimized parameter values. Furthermore, the polarization rendering setting unit 30 can adjust the convergence characteristics of the parameters by using an update rate, which will be described later.

[0033] The multiple parameters used to generate the polarization rendering image include light source parameters related to the light source, geometry parameters related to the target for abnormality detection, material parameters related to the polarization characteristics of the target for abnormality detection, and camera parameters of the polarization image acquisition unit that acquires the polarization image.As shown in Figure 1, the polarization rendering setting unit 30 has a light source parameter setting unit 31, a geometry parameter setting unit 32, a material parameter setting unit 33, and a camera parameter setting unit 34.

[0034] The light source parameter setting unit 31 sets the position and direction of the light source, the Stokes vector of the illumination light emitted from the light source, and other light source parameters to generate a polarization rendering image. The light source parameter setting unit 31 may fix and use light source parameters measured in advance, or may fix and use the results of light source parameter measurement performed before the rendering process. Furthermore, if the light source parameters have not been measured, the light source parameter setting unit 31 performs a parameter optimization process, starts the rendering process using preset initial values, and calculates and uses light source parameters that can minimize the difference between the polarization capture image and the polarization rendering image.

[0035] The geometry parameter setting unit 32 sets the shape, position, and orientation of the anomaly detection target as geometry parameters. The geometry parameter setting unit 32 may fix and use geometry parameters measured in advance, or may fix and use geometry parameter measurement results performed before the rendering process. If the geometry parameters have not been measured, the unit 32 performs parameter optimization processing, starts the rendering process using preset initial values, and calculates and uses geometry parameters that can minimize the difference between the polarization captured image and the polarization rendered image.

[0036] The material parameter setting unit 33 sets a Mueller matrix indicating the polarization characteristics of the object to be detected as the material parameters. The material parameter setting unit 33 may use fixed material parameters measured in advance, or may use fixed material parameter measurement results performed before the rendering process. If the material parameters have not been measured, the material parameter setting unit 33 performs parameter optimization processing, starts the rendering process using preset initial values, and calculates and uses material parameters that can minimize the difference between the polarized captured image and the polarized rendered image.

[0037] The camera parameter setting unit 34 sets the internal and external parameters of the camera as camera parameters. The camera parameter setting unit 34 may fix and use camera parameters measured in advance, or may fix and use the results of measuring the camera parameters performed before the rendering process. If the camera parameters have not been measured, the unit 34 performs a parameter optimization process, starts the rendering process using preset initial values, and calculates and uses camera parameters that can minimize the difference between the polarization captured image and the polarization rendered image.

[0038] The polarization rendering setting unit 30 outputs the parameters set by the light source parameter setting unit 31, the geometry parameter setting unit 32, the material parameter setting unit 33, and the camera parameter setting unit 34 to the polarization rendering image generation unit 40. The polarization rendering setting unit 30 may also generate information indicating for each parameter whether the parameter is a measured parameter, a parameter calculated by optimization processing, whether the optimization processing has been completed, etc. (hereinafter referred to as "parameter attribute information") and output this information to the anomaly detection unit 50.

[0039] The polarization rendering image generating unit 40 generates a polarization rendering image based on the parameters set by the polarization rendering setting unit 30 , and outputs the generated polarization rendering image to the abnormality detecting unit 50 .

[0040] The anomaly detection unit 50 calculates the difference between the polarized captured image acquired by the polarized captured image acquisition unit 20 and the polarized rendered image generated by the polarized rendered image generation unit 40. Furthermore, when parameter optimization processing is performed based on the parameter attribute information generated by the polarized rendering setting unit 30, the anomaly detection unit 50 outputs the calculated difference to the polarized rendering setting unit 30. Furthermore, when the parameter attribute information indicates that all optimization processing has been completed, the anomaly detection unit 50 compares the difference between the polarized captured image and the polarized rendered image with a determination threshold, and detects pixel regions where the difference is greater than the determination threshold as abnormal regions.

[0041] <3. Operation of the embodiment> 4 is a flowchart illustrating the operation of the information processing device. In step ST1, the information processing device acquires a polarized image. The polarized image acquisition unit 20 of the information processing device 10 captures an image of the abnormality detection target to acquire the polarized image, and the process proceeds to step ST2.

[0042] In step ST2, the information processing device sets a parameter update rate. The polarization rendering setting unit 30 of the information processing device 10 sets an update rate α L Furthermore, the polarization rendering setting unit 30 sets the update rate α G ,α M ,α C and proceed to step ST3.

[0043] The update rate is information for setting the parameter correction amount based on the difference between the polarization captured image and the polarization rendered image when performing parameter optimization processing. If the update rate is "0," the polarization rendering setting unit 30 does not perform optimization processing because the parameter correction amount is "0." Furthermore, if the parameter correction amount is greater than "0," the polarization rendering setting unit 30 calculates the parameter correction amount based on the abnormality detection result and performs optimization processing. Details of the operation using the update rate will be described later.

[0044] In step ST3, the information processing device sets parameters with an update rate of 0. For parameters with an update rate of 0, the polarization rendering setting unit 30 sets the parameter values ​​to be used in rendering as the parameter values ​​that have been acquired in advance, or, if the parameter values ​​have not been acquired in advance, as the parameter values ​​calculated by performing a parameter measurement process, and then proceeds to step ST4.

[0045] In step ST4, the information processing device sets initial values ​​for parameters with update rates > 0. Since the polarization rendering setting unit 30 performs optimization processing for parameters with update rates > 0 and automatically calculates optimal parameter values, it sets the initial values ​​for parameters with update rates > 0 to arbitrary values ​​or pre-specified values ​​and proceeds to step ST5.

[0046] In step ST5, the information processing device generates a polarization rendering image. The polarization rendering image generation unit 40 of the information processing device 10 performs rendering processing using the parameter values ​​set in steps ST3 and ST4, or the parameter values ​​set in step ST3 and the parameter values ​​updated in step ST8 (described later), to generate a polarization rendering image, and then proceeds to step ST6.

[0047] In step ST6, the information processing device calculates the difference. The abnormality detection unit 50 of the information processing device 10 calculates the difference between the polarized captured image acquired in step ST1 and the polarized rendering image generated in step ST5, and then the process proceeds to step ST7.

[0048] In step ST7, the information processing device determines whether the optimization process has ended. The polarization rendering setting unit 30 performs an optimization process for parameters with an update rate > 0, updating the parameters using the differential result for each parameter with respect to the difference at a rate according to the update rate, thereby converging the parameters. If each parameter with an update rate > 0 has converged, the polarization rendering setting unit 30 determines that the optimization process has ended and proceeds to step ST9. If there are any parameters that have not converged, the polarization rendering setting unit 30 proceeds to step ST8.

[0049] In step ST8, the information processing device updates the parameters. The polarization rendering setting unit 30 calculates the correction amount for each parameter whose update rate is greater than 0 using the differential result for each parameter with respect to the difference and the update rate. The polarization rendering setting unit 30 then corrects the parameters using the calculated correction amount and returns to step ST5.

[0050] In step ST9, the information processing device performs a discrimination process. The abnormality detection unit 50 of the information processing device 10 compares the difference with a judgment threshold on a pixel-by-pixel basis, and determines that a pixel region with a difference smaller than the judgment threshold is a normal region, and a pixel region with a difference equal to or greater than the judgment threshold is an abnormal region.

[0051] Next, the operation of the information processing device will be described individually for obtaining the light source parameter L, the geometry parameter G, the material parameter M, the camera parameter, the parameter optimization process, and the anomaly detection process. When the parameters are measured in advance, the parameter obtaining unit may be provided separately from the polarization rendering setting unit 30. When the parameters are measured at the site where anomaly detection is performed, the function of the parameter obtaining unit may be provided in the parameter setting unit of the polarization rendering setting unit 30.

[0052] <3-1. Acquiring light source parameters> Next, the acquisition of light source parameters will be described. The light source parameter acquisition unit that acquires the light source parameters is configured using, for example, an image capture unit and an environment image capture unit that has a polarizing plate in front of the image capture unit so that the polarization direction can be changed. The light source parameter acquisition unit calculates a Stokes vector based on polarized captured images in the multiple polarization directions obtained by capturing images of the light source for each of the multiple polarization directions, and uses the Stokes vector as the light source parameter.

[0053] FIG. 5 illustrates the configuration of the environment imaging unit. The environment imaging unit 311 includes, for example, multiple imaging units 3111 with different imaging directions, and polarizing plates 3112 in front of each imaging unit so that the polarization direction can be changed. Note that the polarization directions of the polarizing plates 3112 are the same. The environment imaging unit 311 captures an image of the environment when capturing an image of an abnormality detection target, and generates, for example, polarized images of the celestial sphere for each of multiple polarization directions. Note that the environment imaging unit 311 may use a fisheye lens or the like to acquire polarized images of the celestial sphere for each of multiple polarization directions using one imaging unit 3111 and polarizing plate 3112. The environment imaging unit 311 is not limited to generating polarized images of the celestial sphere. For example, if a light source is provided only in a limited range, polarized images of the limited range may be generated.

[0054] Fig. 6 shows an example of a polarized image captured by the environmental imaging unit, where Fig. 6(a) shows a fisheye image representing the entire celestial sphere, and Fig. 6(b) shows an image obtained by expanding the fisheye image onto a cylindrical surface.

[0055] The light source parameter calculation unit divides the polarized captured image generated by the environment imaging unit 311 into the zenith direction and the azimuth direction, and uses the average incident Stokes vector within each divided region as the light source parameter. The light source parameter calculation unit may also calculate the average incident direction of light rays and include it in the light source parameters.

[0056] Fig. 7 shows an example of division of a polarization-captured image, where Fig. 7(a) shows an example of division of the fisheye image shown in Fig. 6(a), and Fig. 7(b) shows an example of division of the unfolded image shown in Fig. 6(b).

[0057] Here, as shown in (c) of FIG. 7, when the light source position of the polarized incident light with the incident direction ωi is included in the region ARi, the light source parameter calculation unit calculates the average incident direction in the region ARi and the average incident Stokes vector SLi as shown in Equation (2). Similarly, the average incident direction and average incident Stokes vector are calculated for each of the other regions. Note that in Equation (2), the observed value IL(0°) is the observed value obtained by imaging the light source when the polarization direction is 0°, the observed value IL(45°) is the observed value obtained by imaging the light source when the polarization direction is 45°, the observed value IL(90°) is the observed value obtained by imaging the light source when the polarization direction is 90°, and the observed value IL(135°) is the observed value obtained by imaging the light source when the polarization direction is 135°. FIG. 8 also illustrates the light source parameter L.

[0058]

number

[0059] <3-2. Obtaining Geometry Parameters> Next, acquisition of geometry parameters will be described. When acquiring geometry parameters before rendering processing, various existing shape measurement methods may be used. For example, parameters indicating the shape, position, and orientation of the anomaly detection target may be acquired using a ToF (Time Of Flight) sensor or a stereo camera with multiple viewpoints. Furthermore, if the anomaly detection target is a primitive geometry, for example, a cone, cube, sphere, or the like, the geometry parameters may be mathematical expressions indicating the shape, position, and orientation of the anomaly detection target. In this case, rendering processing can be performed without performing shape measurement.

[0060] <3-3. Acquiring material parameters> Next, the acquisition of material parameters will be described. Fig. 9 is a diagram for explaining material polarization characteristics. Light emitted from a light source LT is irradiated onto a measurement object OB via, for example, a polarizing plate PL1, and an imaging unit (hereinafter referred to as a "measurement object imaging unit") CM that images the measurement object for material parameters images the measurement object OB via, for example, a polarizing plate PL2. Note that the Z direction indicates the zenith direction, and the angle θ is the zenith angle.

[0061] When the polarization direction of the polarizers (PL1 and PL2) is, for example, 0°, 45°, 90°, and 135°, and the pixel value obtained by capturing an image of the measurement object with the measurement object imaging unit CM is the observation value I, the relationship between the observation value I (0°) when the polarization direction is 0°, the observation value I (45°) when the polarization direction is 45°, the observation value I (90°) when the polarization direction is 90°, and the observation value I (135°) when the polarization direction is 135° is expressed as the Stokes vector S = [s 0 ,s 1 ,s 2 ] T The relationship between the Stokes vector and the observation value is given by equation (3).

[0062]

number

[0063] In the Stokes vector, the component s 0 represents the unpolarized light intensity or average light intensity. 1 is the difference in intensity between the 0° and 90° polarization directions, and component s 2 indicates the difference in intensity between the 45° and 135° polarization directions. That is, the Stokes vector at 0° is [1,1,0] T , the Stokes vector at 45° is [1,0,1] T , the Stokes vector at 90° is [1,-1,0] T , the Stokes vector for 135° is [1,0,-1] T This becomes:

[0064] Here, if the Stokes vector of the light irradiated onto the measurement object OB in the incident direction ωi is "Si," the Stokes vector of the light observed by the measurement object imaging unit CM in the outgoing direction ωo is "So," and the Mueller matrix for the incident direction ωi and outgoing direction ωo is M(ωo, ωi), then equation (4) holds. Note that equation (4) expresses equation (3) as a determinant.

[0065] M(ωo,ωi) Si = So (3)

[0066]

number

[0067] Equation (4) becomes equation (5) when the polarization direction of the incident light irradiated onto the measurement object OB is 0°. Equation (4) also becomes equation (6) when the polarization direction of the incident light is 45°, equation (7) when the polarization direction of the incident light is 90°, and equation (8) when the polarization direction of the incident light is 135°.

[0068]

number

[0069] Therefore, based on equations (5) to (8), the Mueller matrix M(ωo, ωi) shown in equation (9) can be calculated. Furthermore, normalization is performed to eliminate the influence of luminance on the Mueller matrix M(ωo, ωi). Equation (10) shows the Mueller matrix M(ωo, ωi) after normalization.

[0070]

number

[0071] The Mueller matrix calculated in this way indicates the polarization reflection characteristics specific to the material being measured, and the calculated Mueller matrix is ​​used as the material parameters. Note that the polarization reflection characteristics are independent of the external environment, so once they are measured, they can be used anywhere, eliminating the need to repeatedly obtain the polarization reflection characteristics. Therefore, if the material parameters are measured in advance, it becomes easier to set the parameters required for polarization rendering images.

[0072] 10 and 11 are flowcharts illustrating the operation of acquiring material parameters. Note that FIGS. 10 and 11 show a case where light source parameters are used at positions every angle θa in the azimuth direction and every angle θb in the zenith direction. Furthermore, the measurement object imaging unit that images the object to be measured for material parameters moves the imaging direction every angle θc in the zenith direction, and the polarization direction is switched between "0°, 45°, 90°, 135°." Note that the light source parameters are measured in advance.

[0073] In step ST11, the material parameter acquisition unit initializes the measurement object imaging unit. The material parameter acquisition unit calibrates the measurement object imaging unit that captures an image of the measurement object for material parameters, sets the azimuth angle and zenith angle to 0°, and proceeds to step ST12.

[0074] In step ST12, the material parameter acquisition unit initializes the zenith angle. The material parameter acquisition unit sets the direction where the zenith angle of the measurement object imaging unit is 0° as the direction where the zenith angle of the imaging unit used to acquire the light source parameters (hereinafter referred to as the "light source imaging unit") is 0°, and then proceeds to step ST13.

[0075] In step ST13, the material parameter acquisition unit initializes the azimuth angle. The material parameter acquisition unit initializes the measurement object imaging unit, sets the direction where the orientation of the measurement object imaging unit is 0° as the direction of the azimuth angle of 0° in the light source imaging unit, and proceeds to step ST14.

[0076] In step ST14, the material parameter acquisition unit initializes the polarizing plate on the light source side, sets the polarization direction of the polarizing plate used in the light source imaging unit to 0°, and then proceeds to step ST15.

[0077] In step ST15, the material parameter acquisition unit initializes the polarizing plate of the measurement object imaging unit, sets the polarization direction of the polarizing plate used in the measurement object imaging unit to 0°, and then proceeds to step ST16.

[0078] In step ST16, the material parameter acquisition unit captures an image of the object to be measured for material parameters. The geodetic object capturing unit captures an image of the object to be measured to generate a polarized image, and the process proceeds to step ST17.

[0079] In step ST17, the material parameter acquisition unit rotates the polarizing plate of the measurement object imaging unit by 45°. The material parameter acquisition unit rotates the polarization direction of the polarizing plate by 45°, and then proceeds to step ST18.

[0080] In step ST18, the material parameter acquisition unit determines whether the polarization direction of the measurement object imaging portion is smaller than 180°. If the polarization direction after rotation is smaller than 180°, the material parameter acquisition unit returns to step ST16, and if it is 180° or larger, the material parameter acquisition unit proceeds to step ST19.

[0081] In step ST19, the material parameter acquisition unit acquires the exit Stokes vector. Since the polarization images with polarization directions of 0°, 45°, 90°, and 135° are generated by performing the processes in steps ST16 to ST18, the material parameter acquisition unit calculates the exit Stokes vector based on the generated polarization images, and then proceeds to step ST20.

[0082] In step ST20, the material parameter acquisition unit rotates the polarizing plate on the light source side by 45°. The material parameter acquisition unit rotates the polarization direction of the polarizing plate by 45°, and then proceeds to step ST21.

[0083] In step ST21, the material parameter acquisition unit determines whether the polarization direction on the light source side is smaller than 180°. If the polarization direction after rotation is smaller than 180°, the material parameter acquisition unit returns to step ST15, and if it is 180° or larger, the material parameter acquisition unit proceeds to step ST22.

[0084] In step ST22, the material parameter acquisition unit calculates the polarization reflection characteristics. The material parameter acquisition unit calculates the Mueller matrix based on the output Stokes vector when the polarization direction of the polarized incident light to the measurement object is "0°, 45°, 90°, 135°." That is, the material parameter acquisition unit calculates the Mueller matrix shown in equation (9) or equation (10) based on the above equations (5) to (8), and then proceeds to step ST23.

[0085] In step ST23, the material parameter acquisition unit saves the material parameters. The material parameter acquisition unit generates material parameters by associating the incident direction ωi indicating the light source direction and the emission direction ωo indicating the direction of the measurement target imaging unit with the Mueller matrix calculated in step ST22, saves the material parameters in a database unit or the like, and proceeds to step ST24.

[0086] In step ST24, the material parameter acquisition unit moves the light source azimuth angle by θa°. The material parameter acquisition unit moves the azimuth angle of the light source imaging unit by θa°, and then proceeds to step ST25.

[0087] In step ST25, the material parameter acquisition unit determines whether the light source azimuth angle is smaller than 360°. If the light source azimuth angle is smaller than 360°, the material parameter acquisition unit returns to step ST14, and if the light source azimuth angle is 360° or greater, the material parameter acquisition unit proceeds to step ST26.

[0088] In step ST26, the material parameter acquisition unit moves the light source zenith angle by θb°. The material parameter acquisition unit moves the zenith angle of the light source imaging unit by θb°, and then proceeds to step ST27.

[0089] In step ST27, the material parameter acquisition unit determines whether the light source zenith angle is smaller than 90°. If the light source zenith angle is smaller than 90°, the material parameter acquisition unit returns to step ST13, and if the light source azimuth angle is 90° or greater, proceeds to step ST28. That is, by performing the processes from step ST13 to step ST27, material parameters for each incident direction, with a resolution of θa° in the azimuth direction and a resolution of θb° in the zenith direction for one emission direction, are stored in a database unit or the like.

[0090] In step ST28, the material parameter acquisition unit moves the zenith angle of the measurement object imaging unit by θc°. The material parameter acquisition unit moves the zenith angle of the measurement object imaging unit by θc°, and then proceeds to step ST29.

[0091] In step ST29, the material parameter acquisition unit determines whether the zenith angle of the measurement object imaging unit is smaller than 90°. If the zenith angle of the measurement object imaging unit is smaller than 90°, the material parameter acquisition unit returns to step ST12, and if the azimuth angle is 90° or greater, ends the processing. Therefore, material parameters for each incident direction in which the azimuth resolution is angle θa and the zenith resolution is angle θb, and for each emission direction in which the zenith resolution is angle θc, are stored in a database unit or the like.

[0092] <3-4. Acquiring camera parameters> Next, how to obtain the camera parameters will be described. The camera parameters can be obtained by using an existing measurement method. For example, the intrinsic parameters and extrinsic parameters can be obtained using the method disclosed in Japanese Patent Application Laid-Open No. 2001-264037 or Japanese Patent Application Laid-Open No. 2008-131176.

[0093] <3-5. Image Difference and Parameter Optimization Processing> Next, the image difference and parameter optimization process will be described. The anomaly detection unit 50 detects the polarization rendering image I f φ and polarized image I r φ Difference E Iφ The abnormality detection unit 50 calculates the difference E Iφ For (L, G, M, C), the mean square error (MSE) may be calculated as shown in equation (11), or an arbitrary norm function may be used as shown in equation (12). Also, the maximum pixel value that an image can have and the peak signal to noise ratio (PSNR) calculated using the mean square error may be used as the difference.

[0094]

number

[0095] Polarized rendering image I f φ The parameters used to generate the polarization image I r φ If the parameters match, the difference E Iφ (L, G, M, C) = 0. Therefore, the polarization rendering setting unit 30 generates the polarization rendering image I f φ The parameters used to generate the polarization image I r φ The difference E is used to match the parameters obtained in Iφ For example, the polarization rendering setting unit 30 performs parameter optimization processing using the multivariate function difference E Iφ Calculate the partial derivatives of each parameter (L, G, M, C) and use an existing optimization method (such as the steepest descent method) to obtain the difference E Iφ Parameter L that can minimize (L, G, M, C) em ,G em ,M em ,C em Calculate.

[0096]

number

[0097] The polarization rendering setting unit 30 converges the light source parameter L to an optimal value by repeating the calculation of equation (14). The polarization rendering setting unit 30 also converges the geometry parameter G to an optimal value by repeating the calculation of equation (15), and converges the material parameter M to an optimal value by repeating the calculation of equation (16). Furthermore, the polarization rendering setting unit 30 converges the camera parameter C to an optimal value by repeating the calculation of equation (17). The update rate α used in equations (14) to (17) L ,α G ,α M ,α C is a parameter that adjusts the amount of correction when the parameter is converged using the calculation result of the partial derivative. The update rate may be set in advance, or may be made changeable so that the time required for convergence can be adjusted. Furthermore, the calculation of the partial derivative may be performed automatically using a technique called automatic differentiation, or may be performed manually by the user.

[0098]

number

[0099] By performing such processing, the anomaly detection unit 50 can calculate the image difference. In addition, the polarization rendering setting unit 30 can set each parameter required to generate a polarization rendering image by performing optimization processing on parameters that are not measured in advance or at the site where anomaly detection is performed.

[0100] <3-6. Anomaly detection> Next, we will explain anomaly detection. Polarized rendering image I f φ and polarized image I r φ If they match, the difference E Iφ(L, G, M, C) are "0". In addition, if the parameters used for rendering are unclear, parameter optimization processing can be performed to obtain the polarization captured image I r φ It is possible to approximate the imaging environment and imaging conditions when the image was acquired.

[0101] Therefore, the anomaly detection unit 50 generates a polarization rendering image I generated using parameters measured in advance, parameters measured at the site where the anomaly detection is performed, and parameters calculated in the optimization process. f φ and polarized image I r φ The difference between the two is calculated in pixel units, and the difference E Iφ A pixel region where (L, G, M, C) is smaller than the determination threshold Jth is determined to be a normal region, and a pixel region where (L, G, M, C) is equal to or greater than the determination threshold Jth is determined to be an abnormal region.

[0102] In this way, the difference E Iφ By comparing (L, G, M, C) with the determination threshold Jth, the abnormality detection unit 50 can detect an abnormal area occurring in the object of abnormality detection based on the polarization characteristics.

[0103] Furthermore, the anomaly detection unit 50 may estimate the cause of the detected anomaly based on parameter attribute information indicating whether the parameter is a parameter measured in advance or a parameter calculated in the optimization process. For example, if the material parameter M is measured in advance and fixed by the polarization rendering setting unit 30, and the other parameters are automatically estimated by the optimization process, the cause of the detected anomaly region may be estimated to be an anomaly in the material polarization characteristics corresponding to the material parameter M that was measured in advance and fixed.

[0104] Furthermore, the anomaly detection unit 50 detects the polarization rendering image I f φ and polarized image I r φPolarization information (e.g., degree of polarization, polarization phase, normal, etc.) may be calculated from each of the above, and the difference in the polarization information may be output. By outputting the difference in polarization information in this way, the user can easily identify an abnormal area based on the polarization information, even when it is difficult to identify an abnormality using brightness information or color information. Note that the polarization information, such as the degree of polarization, polarization phase, normal, etc., may be calculated using a method such as that disclosed in Patent Document WO 2019 / 116708.

[0105] By detecting an abnormal area using polarization information in this way, it becomes possible to accurately detect an abnormal area that is difficult to distinguish visually.

[0106] <4. Operational Example of the Embodiment> Next, an example of the operation of the embodiment will be described. In the information processing device 10, when the material parameter M of the target for anomaly detection is measured in advance, the imaging environment and imaging conditions of the target for anomaly detection are set to a pre-specified state, and the light source parameter L, geometry parameter G, and camera parameter C are measured in advance, the polarization rendering setting unit 30 fixes each parameter to the pre-measured parameters.

[0107] The polarization rendering image generator 40 generates a polarization rendering image I using the parameters fixed by the polarization rendering setting unit 30. f φ The anomaly detection unit 50 generates the polarization rendering image I generated by the polarization rendering setting unit 30. f φ and the polarized image I acquired by the polarized image acquisition unit 20. r φ Difference E Iφ (L, G, M, C) is calculated pixel by pixel and compared with the determination threshold Jth to detect an abnormal region of the object of abnormality detection.

[0108] By performing such processing, an abnormal region in the object of abnormality detection can be detected based on the polarization characteristics.

[0109] However, the light source parameter L, the geometry parameter G, the material parameter M, and the camera parameter C are not necessarily measurable in advance. For example, when the polarization image I is captured under different imaging environments or imaging conditions, r φ When a polarization image I is acquired, r φ Polarized rendering image I is created using parameters corresponding to the imaging environment and conditions at the time of acquisition. f φ If you do not generate the difference E Iφ It is not possible to detect abnormal regions based on (L, G, M, C). In this case, for parameters that cannot be measured in advance, the initial parameter values ​​are used to generate the polarization rendering image I f φ Generation and difference E Iφ Calculate (L, G, M, C) and calculate the difference E Iφ The parameter values ​​are updated using (L, G, M, C). The updated parameter values ​​are then used to generate the polarization rendering image I f φ Generation and difference E Iφ Calculate (L, G, M, C) and calculate the difference E Iφ The parameters are optimized by repeatedly updating the parameter values ​​using (L, G, M, C). Furthermore, the polarization rendering image I generated using the optimized parameters is f φ and the polarized image I acquired by the polarized image acquisition unit 20. r φ Difference E Iφ (L, G, M, C) is calculated pixel by pixel and compared with the determination threshold Jth to detect an abnormal region in the object to be detected for abnormality.

[0110] By performing this process, the polarization rendering image I f φ Even if the parameters used to generate the parameter α cannot be measured in advance, an abnormal region in the target for anomaly detection can be detected.

[0111] An example of operation is shown in Fig. 12. The example of operation shown in Fig. 12 shows a case where an abnormality in the surface coating state is detected for each abnormality detection target on a production line or the like.

[0112] Case 1 shows a case where abnormality detection targets with different shapes are included, and an abnormality in the surface coating condition is detected for each abnormality detection target. In this case, the material parameter M of the abnormality detection target is measured in advance. In addition, the light source parameter L, geometry parameter G, and camera parameter C are fixed parameters that correspond to the imaging environment and imaging conditions when acquiring polarized images of the abnormality detection targets on the line, so the light source parameter L, geometry parameter G, and camera parameter C are acquired on-site. By setting the parameters in this way, abnormal areas in the surface coating condition can be detected based on the difference between the generated polarization rendering image and the acquired polarized image.

[0113] Case 2 shows the case where abnormalities in the surface coating condition are detected for each anomaly detection target that has the same shape but different orientations. In this case, the material parameter M of the anomaly detection target is measured in advance. Furthermore, the light source parameter L and camera parameter C are fixed parameters that correspond to the imaging environment and imaging conditions when acquiring polarized images of the anomaly detection target on the line, so they are acquired on-site. Furthermore, because the anomaly detection targets have different orientations, the geometry parameter G is automatically set through optimization processing. By setting the parameters in this way, abnormal areas in the surface coating condition can be detected based on the difference between the generated polarization rendering image and the acquired polarized image.

[0114] Case 3 shows a case where abnormalities in the surface coating condition are detected for each target with the same shape and orientation, without obtaining information about the polarization image acquisition unit. In this case, the material parameter M of the target is measured in advance. Furthermore, the light source parameter L and the geometry parameter G are acquired on-site to be fixed parameters that correspond to the imaging environment and imaging conditions when acquiring the polarization image of the target on the production line. Furthermore, because information about the polarization image acquisition unit is not available, the camera parameter C is automatically set through optimization processing. By setting the parameters in this way, abnormal areas in the surface coating condition can be detected based on the difference between the generated polarization rendering image and the acquired polarization image.

[0115] Case 4 shows a case where information about the light source is unavailable and anomalies in the surface coating condition are detected for each anomaly detection target with the same shape and orientation. In this case, the material parameter M of the anomaly detection target is measured in advance. Furthermore, the geometry parameter G and camera parameter C are fixed parameters that correspond to the imaging environment and imaging conditions when acquiring polarized images of the anomaly detection target on the line, so the geometry parameter G and camera parameter C are acquired on-site. Furthermore, because information about the light source is unavailable, the light source parameter L is automatically set through optimization processing. By setting the parameters in this way, abnormal areas in the surface coating condition can be detected based on the difference between the generated polarization rendering image and the acquired polarized image.

[0116] Case 5 shows a case where abnormalities in the surface coating condition are detected for each target with the same shape, without obtaining information about its orientation and light source. In this case, the material parameter M of the target is measured in advance. The camera parameter C is acquired on-site to be a fixed parameter that corresponds to the imaging environment and conditions when acquiring polarized images of the target on the production line. Furthermore, because information about its orientation and light source is not available, the light source parameter L and geometry parameter G are automatically set through optimization processing. By setting the parameters in this way, abnormal areas in the surface coating condition can be detected based on the difference between the generated polarization rendering image and the acquired polarized image.

[0117] Case 6 shows a case where abnormalities in the surface coating condition are detected for each target with the same shape and orientation, without obtaining information about the light source and the polarization image acquisition unit. In this case, the material parameter M of the target is measured in advance. The geometry parameter G is acquired on-site because it is a fixed parameter that corresponds to the imaging environment and imaging conditions when acquiring the polarization image of the target on the production line. Furthermore, because information about the light source and the polarization image acquisition unit is not available, the light source parameter L and the camera parameter C are automatically set through optimization processing. By setting the parameters in this way, abnormal areas in the surface coating condition can be detected based on the difference between the generated polarization rendering image and the acquired polarization image.

[0118] Case 7 shows a case where abnormalities in the surface coating condition are detected for each target object with the same shape, without obtaining information about the light source, the polarized image acquisition unit, and its orientation. In this case, the material parameter M of the target object is measured in advance. Furthermore, because information about the light source, the polarized image acquisition unit, and its orientation is not available, the light source parameter L, the geometry parameter G, and the camera parameter C are automatically set by optimization processing. By setting the parameters in this way, abnormal areas in the surface coating condition can be detected based on the difference between the generated polarized rendering image and the acquired polarized image.

[0119] It should be noted that Fig. 12 is merely an example, and the operation is not limited to the case shown in Fig. 12. For example, rotten parts may be detected from the surface condition of the fruit.

[0120] The series of processes described in this specification can be executed by hardware, software, or a combination of both. When executing processes by software, a program recording the processing sequence is installed in the memory of a computer incorporated in dedicated hardware and executed. Alternatively, the program can be installed and executed on a general-purpose computer capable of executing various processes.

[0121] For example, the program can be pre-recorded on a recording medium such as a hard disk, a solid state drive (SSD), or a read-only memory (ROM). Alternatively, the program can be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, a compact disc read-only memory (CD-ROM), a magneto optical (MO) disc, a digital versatile disc (DVD), a Blu-ray disc (BD (registered trademark)), a magnetic disk, or a semiconductor memory card. Such removable recording media can be provided as so-called packaged software.

[0122] In addition to being installed on a computer from a removable recording medium, the program may also be transferred wirelessly or by wire from a download site to a computer via a network such as a LAN (Local Area Network) or the Internet. The computer can receive the program transferred in this manner and install it on a recording medium such as a built-in hard disk.

[0123] Note that the effects described in this specification are merely examples and are not limiting, and additional effects not described may exist. Furthermore, the present technology should not be interpreted as being limited to the above-described embodiments of the technology. The embodiments of the technology disclose the present technology in the form of examples, and it is obvious that a person skilled in the art can modify or substitute the embodiments without departing from the gist of the present technology. In other words, the scope of the claims should be taken into consideration when determining the gist of the present technology.

[0124] Furthermore, the information processing device of the present technology can also have the following configuration. (1) a polarization rendering setting unit that sets a plurality of parameters used to generate a polarization rendering image of an object to be detected for anomaly; a polarization rendering image generation unit that generates a polarization rendering image of the abnormality detection target based on the parameters set by the polarization rendering setting unit; and an abnormality detection unit that detects an abnormal region of the abnormality detection target based on a difference between a polarized captured image acquired by capturing an image of the abnormality detection target and the polarized rendering image generated by the polarized rendering image generation unit; An information processing device comprising: (2) the plurality of parameters are measured; The information processing device according to (1), wherein the polarization rendering setting unit sets the measured parameters as parameters to be used in generating the polarization rendering image. (3) If some or all of the parameters are not measured, The information processing device described in (1) in which the polarization rendering setting unit performs an optimization process on the unmeasured parameters, and sets the measured parameters and the parameters calculated by the optimization process, or, if there are no measured parameters, the parameters calculated by the optimization process, as parameters to be used for generating the polarization rendering image. (4) The information processing device according to (3), wherein the parameter optimization process calculates parameters that can minimize the difference between the polarized captured image and the polarized rendering image generated by the polarized rendering image generation unit. (5) The information processing device described in (4) is configured such that the convergence characteristics of the parameters can be adjusted by repeatedly updating the parameters using the difference, and the converged parameter value is set as an optimized parameter value. (6) The information processing device according to (3) or (5), wherein the abnormality detection unit estimates the cause of the abnormality in the detected abnormal area based on information indicating whether the parameter is the measured parameter or a parameter calculated by the optimization process. (7) An information processing device described in any of (1) to (6), wherein the abnormality detection unit uses polarization information calculated from the polarized captured image and polarization information calculated from the polarized rendering image to detect an abnormal area of ​​the abnormality detection target based on the difference in polarization information. (8) The information processing device according to any one of (1) to (7), wherein the abnormality detection unit detects the abnormal area of ​​the abnormality detection target on a pixel-by-pixel basis. (9) An information processing device according to any one of (1) to (8), wherein the plurality of parameters include light source parameters relating to a light source, geometry parameters relating to the abnormality detection target, material parameters relating to the polarization characteristics of the abnormality detection target, and camera parameters of a polarization image acquisition unit that acquires the polarization image. (10) The information processing device according to (9), wherein at least the material parameters are measured in advance, and the polarization rendering setting unit uses the parameters measured in advance as fixed parameters. [Explanation of symbols]

[0125] 10. Information processing device 20 Polarized image acquisition unit 30 Polarization rendering setting section 31 Light source parameter setting section 32 Geometry parameter setting section 33 Material parameter setting section 34. Camera parameter setting section 40. Polarized rendering image generation unit 50 Abnormality detection unit

Claims

1. a polarization rendering setting unit that sets a plurality of parameters used to generate a polarization rendering image of an abnormality detection target; a polarization rendering image generation unit that generates a polarization rendering image of the abnormality detection target based on the parameters set by the polarization rendering setting unit; and an abnormality detection unit that detects an abnormal region of the abnormality detection target based on a difference between a polarized captured image acquired by capturing an image of the abnormality detection target and the polarized rendering image generated by the polarized rendering image generation unit; Equipped with If some or all of the plurality of parameters are not measured, the polarization rendering setting unit performs an optimization process for the unmeasured parameters, and sets the measured parameters and the parameters calculated by the optimization process, or, if there are no measured parameters, the parameters calculated by the optimization process, as parameters to be used for generating the polarization rendering image; In the parameter optimization process, parameters that can minimize the difference between the polarized captured image and the polarized rendering image generated by the polarized rendering image generation unit are calculated. Information processing device.

2. a polarization rendering setting unit that sets a plurality of parameters used to generate a polarization rendering image of an abnormality detection target; a polarization rendering image generation unit that generates a polarization rendering image of the abnormality detection target based on the parameters set by the polarization rendering setting unit; and an abnormality detection unit that detects an abnormal region of the abnormality detection target based on a difference between a polarized captured image acquired by capturing an image of the abnormality detection target and the polarized rendering image generated by the polarized rendering image generation unit, The plurality of parameters include light source parameters relating to a light source, geometry parameters relating to the abnormality detection target, material parameters relating to polarization characteristics of the abnormality detection target, and camera parameters of a polarization image acquisition unit that acquires the polarization image. Information processing device.

3. The polarization rendering setting unit sets the measured parameters as parameters to be used in generating the polarization rendering image.

3. The information processing device according to claim 1.

4. The polarization rendering setting unit is configured to adjust the convergence characteristics of the parameters by repeatedly updating the parameters using the difference and setting the converged parameter values ​​as optimized parameter values. The information processing device according to claim 1 .

5. The abnormality detection unit estimates a cause of the abnormality in the detected abnormal region based on information indicating whether the parameter is the measured parameter or the parameter calculated by the optimization process. The information processing device according to claim 1 .

6. The anomaly detection unit detects an abnormal region of the anomaly detection target based on a difference between polarization information calculated from the polarized captured image and polarization information calculated from the polarized rendered image.

3. The information processing device according to claim 1.

7. The abnormality detection unit detects an abnormal area of ​​the abnormality detection target in pixel units.

3. The information processing device according to claim 1.

8. At least the material parameters are measured in advance, and the polarization rendering setting unit uses the parameters measured in advance as fixed parameters. The information processing device according to claim 2 .

9. setting a plurality of parameters used to generate a polarization rendering image of an abnormality detection target in a polarization rendering setting unit; generating a polarization rendering image of the abnormality detection target by a polarization rendering image generation unit based on the parameters set by the polarization rendering setting unit; detecting an abnormal region of the abnormality detection target by an abnormality detection unit based on a difference between a polarized captured image acquired by capturing an image of the abnormality detection target and the polarized rendering image generated by the polarized rendering image generation unit, If some or all of the plurality of parameters are not measured, the polarization rendering setting unit performs an optimization process for the unmeasured parameters, and sets the measured parameters and the parameters calculated by the optimization process, or, if there are no measured parameters, the parameters calculated by the optimization process, as parameters to be used for generating the polarization rendering image; the parameter optimization process calculates parameters that can minimize the difference between the polarized captured image and the polarized rendering image generated by the polarized rendering image generation unit; Information processing methods.

10. setting a plurality of parameters used to generate a polarization rendering image of an abnormality detection target in a polarization rendering setting unit; generating a polarization rendering image of the abnormality detection target by a polarization rendering image generation unit based on the parameters set by the polarization rendering setting unit; detecting an abnormal region of the abnormality detection target by an abnormality detection unit based on a difference between a polarized captured image acquired by capturing an image of the abnormality detection target and the polarized rendering image generated by the polarized rendering image generation unit, an information processing method, wherein the plurality of parameters include light source parameters related to a light source, geometry parameters related to the object of abnormality detection, material parameters related to polarization characteristics of the object of abnormality detection, and camera parameters of a polarization image acquisition unit that acquires the polarization image.

11. A program for causing a computer to execute anomaly detection for an anomaly detection target, setting a plurality of parameters used to generate a polarization rendering image of the anomaly detection target; generating a polarization rendering image of the anomaly detection target based on the set plurality of parameters; detecting an abnormal region of the abnormality detection target based on a difference between a polarized captured image acquired by capturing an image of the abnormality detection target and the generated polarized rendering image; If some or all of the plurality of parameters are not measured, In the step of generating the polarization rendering image, the computer is caused to execute a process of optimizing the unmeasured parameters, and setting the measured parameters and the parameters calculated by the optimization process, or, if there are no measured parameters, the parameters calculated by the optimization process, as parameters to be used in generating the polarization rendering image; In the parameter optimization process, the program causes the computer to execute a process of calculating parameters that can minimize the difference between the polarized captured image and the polarized rendering image generated in the polarized rendering image generation procedure.

12. A program for causing a computer to execute anomaly detection for an anomaly detection target, setting a plurality of parameters used to generate a polarization rendering image of the anomaly detection target; generating a polarization rendering image of the anomaly detection target based on the set plurality of parameters; detecting an abnormal region of the abnormality detection target based on a difference between a polarized captured image acquired by capturing an image of the abnormality detection target and the generated polarized rendering image; The program includes light source parameters related to a light source, geometry parameters related to the object of abnormality detection, material parameters related to the polarization characteristics of the object of abnormality detection, and camera parameters of a polarization image acquisition unit that acquires the polarization image.

Images